Situation
Charles River Analytics (Charles River), a leader in artificial intelligence, robotics, and human-centered technologies, sought to develop intuitive human-robot interaction systems through their Person Aware Liaison (PAL) project, funded by a Phase II Small Business Innovation Research (SBIR) contract from NASA. PAL aims to build trust between humans and robots by interpreting both verbal and nonverbal cues, enabling robots to effectively respond to intuitive instructions and seamlessly collaborate with human partners.
Problem
Collaborative robots offer tremendous potential to enhance productivity and safety across various industries. However, their effectiveness relies heavily on the robot’s ability to accurately interpret and respond to subtle human signals. Traditional robotic systems often fall short, as they typically require explicit instructions and lack the capacity to interpret nuanced nonverbal human interactions such as gestures, gaze, posture, and context. Without the ability to accurately contextualize tasks and environments, collaborative robots struggle to achieve seamless, natural, and trustworthy interactions with their human counterparts.
Solution
Charles River developed PAL, an advanced system designed specifically to bridge this communication gap. PAL incorporates a sophisticated combination of speech recognition, gesture interpretation, gaze tracking, and posture analysis to reliably interpret user intent. For example, when a user instructs a robot, “Pass that to me,” combined with a focused gaze and an outstretched hand, PAL accurately determines the target object and desired action.
PAL reliably discretizes these intuitive human interactions into explicit, actionable robotic tasks. It generates detailed instructions for MoveIt Pro, providing clearly defined positions, orientations, translation, and rotation commands. MoveIt Pro’s graph-based planning approach then leverages multiple inverse kinematic solutions and grasp poses, ensuring that the robot requires only the object’s position to autonomously handle all motion planning, grasping strategies, and execution logic.
To demonstrate PAL’s capabilities on real-world hardware, Charles River leveraged PickNik Robotics’ MoveIt Pro platform. MoveIt Pro provided essential robotic manipulation functions, such as collision-free motion planning, accurate grasping, and reliable object placement. MoveIt Pro’s modular and hardware-agnostic design simplified the integration of PAL across multiple robotic platforms. For instance, Charles River seamlessly transitioned between different robotic hardware setups, such as replacing a standard Robotiq two-finger gripper with a Robotiq Hand-E gripper on a Universal Robots UR5e arm.
Results
Utilizing MoveIt Pro, Charles River successfully demonstrated PAL’s robust capability to interpret and execute intuitive human instructions on physical robotic hardware. PAL’s nuanced interpretation of both verbal and nonverbal cues significantly enhanced the naturalness and reliability of robot interactions, greatly fostering human-robot trust in collaborative environments.
Beyond basic motion planning, MoveIt Pro provided advanced capabilities essential for robust operation. The platform’s behavior tree architecture enabled Charles River to implement sophisticated decision-making logic, such as motion-planning fallbacks and real-time adjustments. MoveIt Pro could dynamically respond to prompts from PAL, modifying, changing, or stopping tasks based on evolving interactions and user feedback. This allowed Charles River to demonstrate more nuanced and intelligent robot behavior, further enhancing trust and collaboration between humans and robots.
MoveIt Pro’s flexibility allowed Charles River to clearly showcase PAL’s applicability across diverse robotic setups and scenarios, confirming its viability for real-world applications in fields ranging from industrial manufacturing and maintenance to complex space missions. Dr. Madison Clark-Turner, Robotics Scientist at Charles River and Principal Investigator on PAL, highlighted MoveIt Pro’s integral role: “MoveIt Pro has enabled us to effectively showcase PAL as a reliable partner that smoothly merges human and robotic capabilities.”
As Charles River continues advancing PAL by incorporating additional interaction modalities and more sophisticated manipulation tasks, MoveIt Pro remains a pivotal component. It allows Charles River to clearly demonstrate advanced, intuitive, and practical human-robot collaboration across diverse and challenging environments.
MoveIt Pro is PickNik Robotics’ commercial robotics software platform that extends the capabilities of the open-source MoveIt project. Designed for unstructured environments, MoveIt Pro provides powerful tools for motion planning, grasping, perception, and behavior sequencing. With pre-built robot behaviors and a modular runtime architecture, MoveIt Pro significantly reduces engineering effort and time-to-deployment. Its hardware-agnostic design supports a wide range of commercial and custom robotic arms, making it ideal for scalable automation across space, manufacturing, and logistics sectors.
