Search Results (57)

Human communication in daily life entails not only talking about what we are currently doing or will do, but also speculating about future possibilities that may (or may not) occur, i.e., “anticipatory speech”. Such conversations are central to social cooperation and social cohesion in humans. This suggests that such capabilities may also be critical for developing improved speech systems for artificial agents, e.g., human–agent interaction (HAI) and human–robot interaction (HRI). However, to do so successfully, it is imperative that we understand how anticipatory speech may affect the behavior of human users and, subsequently, the behavior of the agent/robot. Moreover, it is possible that such effects may vary across cultures and languages. To that end, we conducted an experiment where a human and autonomous 3D virtual avatar interacted in a cooperative gameplay environment. The experiment included 40 participants, comparing different languages (20 English, 20 Korean), where the artificial agent had anticipatory speech either enabled or disabled. The results showed that anticipatory speech significantly altered the speech patterns and turn-taking behavior of both the human and the agent, but those effects varied depending on the language spoken. We discuss how the use of such novel communication forms holds potential for enhancing HAI/HRI, as well as the development of mixed reality and virtual reality interactive systems for human users. Full article

Dialogue systems must understand children’s utterance intentions by considering their unique linguistic characteristics, such as syntactic incompleteness, pronunciation inaccuracies, and creative expressions, to enable natural conversational engagement in child–robot interactions. Even state-of-the-art large language models (LLMs) for language understanding and contextual awareness cannot comprehend children’s intent as accurately as humans because of their distinctive features. An LLM-based dialogue system should acquire the manner by which humans understand children’s speech to enhance its intention reasoning performance in verbal interactions with children. To this end, we propose a fine-tuning methodology that utilizes the LLM–human judgment discrepancy and interactive response data. The former data represent cases in which the LLM and human judgments of the contextual appropriateness of a child’s answer to a robot’s question diverge. The latter data involve robot responses suitable for children’s utterance intentions, generated by the LLM. We developed a fine-tuned dialogue system using these datasets to achieve human-like interpretations of children’s utterances and to respond adaptively. Our system was evaluated through human assessment using the Robotic Social Attributes Scale (RoSAS) and Sensibleness and Specificity Average (SSA) metrics. Consequently, it supports the effective interpretation of children’s utterance intentions and enables natural verbal interactions, even in cases with syntactic incompleteness and mispronunciations. Full article

In this paper, we detail the technical development of a conversation design that is sensitive to group dynamics and adaptable, taking into account the subtleties of linguistic variations between dyadic (i.e., one human and one agent) and group interactions in human–robot interaction (HRI) using the German language as a case study. The paper details the implementation of robust person and group detection with YOLOv5m and the expansion of knowledge databases using large language models (LLMs) to create adaptive multi-party interactions (MPIs) (i.e., group–robot interactions (GRIs)). We describe the use of LLMs to generate training data for socially interactive agents including social robots, as well as a self-developed synthesis tool, knowledge expander, to accurately map the diverse needs of different users in public spaces. We also outline the integration of a LLM as a fallback for open-ended questions not covered by our knowledge database, ensuring it can effectively respond to both individuals and groups within the MPI framework. Full article

Background/Objectives: Artificial Intelligence (AI) technology is gaining attention for its potential in cognitive function assessment and intervention. AI robots and agents can offer continuous dialogue with the elderly, helping to prevent social isolation and support cognitive health. Speech-based evaluation methods are promising as they reduce the burden on elderly participants. AI agents could replace human questioners, offering efficient and consistent assessments. However, existing research lacks sufficient comparisons of elderly speech content when interacting with AI versus human partners, and detailed analyses of factors like cognitive function levels and dialogue partner effects on speech elements such as proper nouns and fillers. Methods: This study investigates how elderly individuals’ cognitive functions influence their communication patterns with both human and AI conversational partners. A total of 34 older people (12 men and 22 women) living in the community were selected from a silver human resource centre and day service centre in Tokyo. Cognitive function was assessed using the Mini-Mental State Examination (MMSE), and participants engaged in semi-structured daily conversations with both human and AI partners. Results: The study examined the frequency of fillers, proper nouns, and “listen back” in conversations with AI and humans. Results showed that participants used more fillers in human conversations, especially those with lower cognitive function. In contrast, proper nouns were used more in AI conversations, particularly by those with higher cognitive function. Participants also asked for explanations more often in AI conversations, especially those with lower cognitive function. These findings highlight differences in conversation patterns based on cognitive function and the conversation partner being either AI or human. Conclusions: These results suggest that there are differences in conversation patterns depending on the cognitive function of the participants and whether the conversation partner is a human or an AI. This study aims to provide new insights into the effective use of AI agents in dialogue with the elderly, contributing to the improvement of elderly welfare. Full article

As humans’ additional arms or legs, supernumerary robotic limbs (SRLs) have gained great application prospects in many fields. However, current SRLs lack both rigidity/flexibility adaptability and arm/leg function conversion. Inspired by the muscular hydrostat characteristics of octopus tentacles, fiber-reinforced actuators (FRAs) were employed to develop SRLs simultaneously realizing flexible operation and stable support. In this paper, an SRL with FRAs was designed and implemented. The analytic model of the FRA was established to formulate the movement trajectory and stiffness profile of the SRL. A hierarchical hidden Markov model (HHMM) was proposed to recognize the wearer’s motion intention and control the SRL to complete the specific working mode and motion type. Experiments were conducted to exhibit the feasibility and superiority of the proposed robot. Full article

This study investigates the effects of humanoid robot appearance, emotional expression, and interaction skills on the uncanny valley phenomenon among university students using the social humanoid robot (SHR) Ameca. Two fundamental studies were conducted within a university setting: Study 1 assessed student expectations of SHRs in a hallway environment, emphasizing the need for robots to integrate seamlessly and engage effectively in social interactions; Study 2 compared the humanlikeness of three humanoid robots, ROMAN, ROBIN, and EMAH (employing the EMAH robotic system implemented on Ameca). The initial findings from corridor interactions highlighted a diverse range of human responses, from engagement and curiosity to indifference and unease. Additionally, the online survey revealed significant insights into expected non-verbal communication skills, continuous learning, and comfort levels during hallway conversations with robots. Notably, certain humanoid robots evoked stronger emotional reactions, hinting at varying degrees of humanlikeness and the influence of interaction quality. The EMAH system was frequently ranked as most humanlike before the study, while post-study perceptions indicated a shift, with EMAH and ROMAN showing significant changes in perceived humanlikeness, suggesting a re-evaluation by participants influenced by their interactive experiences. This research advances our understanding of the uncanny valley phenomenon and the role of humanoid design in enhancing human–robot interaction, marking the first direct comparison between the most advanced, humanlike research robots. Full article

Pantomimes are human actions that simulate ideas, objects, and events, commonly used in conversation, performance art, and gesture-based interfaces for computing and controlling robots. Yet, their underlying neurocognitive mechanisms are not well understood. In this review, we examine pantomimes through two parallel lines of research: (1) the two visual systems (TVS) framework for visually guided action, and (2) the neuropsychological literature on limb apraxia. Historically, the TVS framework has considered pantomime actions as expressions of conscious perceptual processing in the ventral stream, but an emerging view is that they are jointly influenced by ventral and dorsal stream processing. Within the apraxia literature, pantomimes were historically viewed as learned motor schemas, but there is growing recognition that they include creative and improvised actions. Both literatures now recognize that pantomimes are often created spontaneously, sometimes drawing on memory and always requiring online cognitive control. By highlighting this convergence of ideas, we aim to encourage greater collaboration across these two research areas, in an effort to better understand these uniquely human behaviors. Full article

Freedom of movement of the hands is the most desired hope of stroke patients. However, stroke recovery is a long, long road for many patients. If artificial intelligence can assist human arm movement, the possibility of stroke patients returning to normal hand movement might be significantly increased. This study uses the artificial neuromolecular system (ANM system) developed in our laboratory as the core of motion control, in an attempt to learn to control the mechanical arm to produce actions similar to human rehabilitation training and the transition between different activities. This research adopts two methods. The first is hypothetical exploration, the so-called “artificial world” simulation method. The detailed approach uses the V-REP (Virtual Robot Experimentation Platform) to conduct different experimental runs to capture relevant data. Our policy is to establish an action database systematically to a certain extent. From these data, we use the ANM system with self-organization and learning capabilities to develop the relationship between these actions and establish the possibility of conversion between different activities. The second method of this study is to use the data from a hospital in Toronto, Canada. Our experimental results show that the ANM system can continuously learn for problem-solving. In addition, our three experimental results of adaptive learning, transfer learning, and cross-task learning further confirm that the ANM system can use previously learned systems to complete the delivered tasks through autonomous learning (instead of learning from scratch). Full article

This paper presents the architecture of a multimodal human–robot interaction control platform that leverages the advanced language capabilities of ChatGPT to facilitate more natural and engaging conversations between humans and robots. Implemented on the Pepper humanoid robot, the platform aims to enhance communication by providing a richer and more intuitive interface. The motivation behind this study is to enhance robot performance in human interaction through cutting-edge natural language processing technology, thereby improving public attitudes toward robots, fostering the development and application of robotic technology, and reducing the negative attitudes often associated with human–robot interactions. To validate the system, we conducted experiments measuring negative attitude robot scale and their robot anxiety scale scores before and after interacting with the robot. Statistical analysis of the data revealed a significant improvement in the participants’ attitudes and a notable reduction in anxiety following the interaction, indicating that the system holds promise for fostering more positive human–robot relationships. Full article

Affective communication, encompassing verbal and non-verbal cues, is crucial for understanding human interactions. This study introduces a novel framework for enhancing emotional understanding by fusing speech emotion recognition (SER) and sentiment analysis (SA). We leverage diverse features and both classical and deep learning models, including Gaussian naive Bayes (GNB), support vector machines (SVMs), random forests (RFs), multilayer perceptron (MLP), and a 1D convolutional neural network (1D-CNN), to accurately discern and categorize emotions in speech. We further extract text sentiment from speech-to-text conversion, analyzing it using pre-trained models like bidirectional encoder representations from transformers (BERT), generative pre-trained transformer 2 (GPT-2), and logistic regression (LR). To improve individual model performance for both SER and SA, we employ an extended dynamic Bayesian mixture model (DBMM) ensemble classifier. Our most significant contribution is the development of a novel two-layered DBMM (2L-DBMM) for multimodal fusion. This model effectively integrates speech emotion and text sentiment, enabling the classification of more nuanced, second-level emotional states. Evaluating our framework on the EmoUERJ (Portuguese) and ESD (English) datasets, the extended DBMM achieves accuracy rates of 96% and 98% for SER, 85% and 95% for SA, and 96% and 98% for combined emotion classification using the 2L-DBMM, respectively. Our findings demonstrate the superior performance of the extended DBMM for individual modalities compared to individual classifiers and the 2L-DBMM for merging different modalities, highlighting the value of ensemble methods and multimodal fusion in affective communication analysis. The results underscore the potential of our approach in enhancing emotional understanding with broad applications in fields like mental health assessment, human–robot interaction, and cross-cultural communication. Full article

Bio-inspired hydrogel robots have become promising due to their advantage of the interaction safety and comfort between robots and humans, while current hydrogel robots mainly focus on underwater movement due to the hydration–dehydration process of thermo-responsive hydrogels, which greatly limits their practical applications. To expand the motion of the thermo-responsive hydrogel robot to the ground, we constructed a hydrogel robot inspired by a caterpillar, which has an anisotropic double-layered structure by the interfacial diffusion polymerization method. Adding PVA and SA to PNIPAm will cause different conformation transitions. Therefore, sticking the two layers of hydrogel together will form a double-layer anisotropic structure. The ultra-high hydrophilicity of PVA and SA significantly reduces the contact angle of the hydrogel from 53.1° to about 10° and reduces its hydration time. The responsive time for bending 30° of the hydrogel robot has been greatly reduced from 1 h to half an hour through the enhancement of photo-thermal conversion and thermal conductivity via the addition of Fe₃O₄ nanoparticles. As a result, the fabricated hydrogel robot can achieve a high moving speed of 54.5 mm·h⁻¹ on the ground. Additionally, the fabricated hydrogel has excellent mechanical strength and can endure significant flexibility tests. This work may pave the road for the development of soft robots and expand their applications in industry. Full article

Anthropomorphized robots are increasingly integrated into human social life, playing vital roles across various fields. This study aimed to elucidate the neural dynamics underlying users’ perceptual and emotional responses to robots with varying levels of anthropomorphism. We investigated event-related potentials (ERPs) and event-related spectral perturbations (ERSPs) elicited while participants viewed, perceived, and rated the affection of robots with low (L-AR), medium (M-AR), and high (H-AR) levels of anthropomorphism. EEG data were recorded from 42 participants. Results revealed that H-AR induced a more negative N1 and increased frontal theta power, but decreased P2 in early time windows. Conversely, M-AR and L-AR elicited larger P2 compared to H-AR. In later time windows, M-AR generated greater late positive potential (LPP) and enhanced parietal-occipital theta oscillations than H-AR and L-AR. These findings suggest distinct neural processing phases: early feature detection and selective attention allocation, followed by later affective appraisal. Early detection of facial form and animacy, with P2 reflecting higher-order visual processing, appeared to correlate with anthropomorphism levels. This research advances the understanding of emotional processing in anthropomorphic robot design and provides valuable insights for robot designers and manufacturers regarding emotional and feature design, evaluation, and promotion of anthropomorphic robots. Full article

During the transition from Industry 4.0 to Industry 5.0, industrial robotics technology faces the need for intelligent and highly integrated development. Metaverse technology creates immersive and interactive virtual environments, allowing technicians to perform simulations and experiments in the virtual world, and overcoming the limitations of traditional industrial operations. This paper explores the application and evolution of metaverse technology in the field of industrial robotics, focusing on the realization of virtual–real integration and human–machine collaboration. It proposes a design framework for a virtual–real interaction system based on the ROS and WEB technologies, supporting robot connectivity, posture display, coordinate axis conversion, and cross-platform multi-robot loading. This paper emphasizes the study of two key technologies for the system: virtual–real model communication and virtual–real model transformation. A general communication mechanism is designed and implemented based on the ROS, using the ROS topic subscription to achieve connection and real-time data communication between physical robots and virtual models, and utilizing URDF model transformation technology for model invocation and display. Compared with traditional simulation software, i.e., KUKA Sim PRO (version 1.1) and RobotStudio (version 6.08), the system improves model loading by 45.58% and 24.72%, and the drive response by 41.50% and 28.75%. This system not only supports virtual simulation and training but also enables the operation of physical industrial robots, provides persistent data storage, and supports action reproduction and offline data analysis and decision making. Full article

Social robots are being conceived with different characteristics and being used in different applications. The growth of social robotics benefits from advances in fabrication, sensing, and actuation technologies, as well as signal processing and artificial intelligence. This paper presents a design and implementation of the humanoid robotic platform Adam, consisting of a motorized human-like head with precise movements of the eyes, jaw, and neck, together with capabilities of face tracking and vocal conversation using ChatGPT. Adam relies on 3D-printed parts together with a microphone, a camera, and proper servomotors, and it has high structural integrity and flexibility. Adam’s control framework consists of an adequate signal exploitation and motor command strategy that allows efficient social interactions. Adam is an innovative platform that combines manufacturability, user-friendliness, low costs, acceptability, and sustainability, offering advantages compared with other platforms. Indeed, the platform’s hardware and software components are adjustable and allow it to increase its abilities and adapt them to different applications in a variety of roles. Future work will entail the development of a body for Adam and the addition of skin-like materials to enhance its human-like appearance. Full article

The vertebrate basal ganglia play an important role in action selection—the resolution of conflicts between alternative motor programs. The effective operation of basal ganglia circuitry is also known to rely on appropriate levels of the neurotransmitter dopamine. We investigated reducing or increasing the tonic level of simulated dopamine in a prior model of the basal ganglia integrated into a robot control architecture engaged in a foraging task inspired by animal behaviour. The main findings were that progressive reductions in the levels of simulated dopamine caused slowed behaviour and, at low levels, an inability to initiate movement. These states were partially relieved by increased salience levels (stronger sensory/motivational input). Conversely, increased simulated dopamine caused distortion of the robot’s motor acts through partially expressed motor activity relating to losing actions. This could also lead to an increased frequency of behaviour switching. Levels of simulated dopamine that were either significantly lower or higher than baseline could cause a loss of behavioural integration, sometimes leaving the robot in a ‘behavioral trap’. That some analogous traits are observed in animals and humans affected by dopamine dysregulation suggests that robotic models could prove useful in understanding the role of dopamine neurotransmission in basal ganglia function and dysfunction. Full article