Research ArticleHUMAN-ROBOT INTERACTION

Dynamic locomotion synchronization of bipedal robot and human operator via bilateral feedback teleoperation

See allHide authors and affiliations

Science Robotics  30 Oct 2019:
Vol. 4, Issue 35, eaav4282
DOI: 10.1126/scirobotics.aav4282
  • Fig. 1 Bilateral teleoperation of a bipedal robot for dynamic locomotion.

    (A) The human operator controls the small bipedal robot, Little HERMES, to take steps in place. (B) The core components of the locomotion dynamics are mapped from operator to the robot using a simplified model, the LIP. Simultaneously, a feedback force [red arrows in (B1) and (B2)] is applied to the torso of the operator, near the CoM, to synchronize the motion of operator and robot. This feedback force is proportional to the relative instantaneous velocity between the operator and robot. (C) The machine uses human reference to reproduce stepping motions in real time.

  • Fig. 2 Dynamic similarity of the simplified models for the human and robot.

    In the ideal scenario, the horizontal motion of the robot DCM [ξR(t)=xR(t)+ẋR(t)ωR] and CoP pR are dynamically similar to those of the human operator, which means that their dimensionless trajectories match [ξH(t)=ξR(t) and pH(t)=pR(t)]. The CoMs are not required to coincide because manipulating the DCM is sufficient to control locomotion. Time dependency of the state variables is omitted for clarity.

  • Fig. 3 Results for stepping-in-place teleoperation experiment.

    Dynamic synchronization of human and robot DCM and CoP, the two fundamental components of the LIP. (A) Comparison between human and robot DCM normalized by the distance between the feet dH and dR. The high-frequency component of the robot DCM is an artifact of the foot compliance. (B) Dimensionless CoP position for the operator (blue) and the robot (red). The areas shaded in magenta indicate right foot support, as illustrated by the cartoon in the magenta box, when the relative CoP trajectory is flat at pR=0.5. Left foot support is analogously represented by green shaded areas and the cartoon in the green box (pR=0.5). The cartoons also illustrate the robot swing foot trajectory δR(t) scaled from the human reference δH(t) and the feedback force FBFI(t) in red. (C) Time evolution of the feedback force FBFI(t) applied to the CoM of the operator during teleoperation.

  • Fig. 4 Constrained dynamic walking and jumping experiments.

    (A) Operator commands the robot to walk by stepping in place while leaning forward and applying a static force against a string. The robot produces a similar tangential force with the stance foot, propelling the machine forward. Walking speed ẏR is controlled by the tangential ground contact force FyR in combination with the proper stride length δyR. (B) Robot jumps by modulating the magnitude of the net contact force similarly to the operator. (B1) to (B3) illustrate this procedure and are indicated by the vertical yellow, green, and magenta lines in (B4) and (B5). (B4) Dimensionless vertical component of the net contact forces from the operator and robot. (B5) Vertical displacement of the CoM estimated from leg kinematics and from the external gantry.

  • Fig. 5 Dynamic mobile manipulation.

    We intend to combine previous results for manipulation, enabled by HERMES, and locomotion, introduced here by Little HERMES, to develop a capable robotic responder. This robot will leverage human motor control and perception skills to perform physically demanding tasks that require whole-body coordination in addition to balance regulation.

  • Fig. 6 The bipedal robot Little HERMES.

    The design of the small-scale robot is based on principles specific for agile legged locomotion. (A) Custom actuators were designed for impact mitigation and high-bandwidth torque control. (B) Lightweight limbs have negligible inertia and allow fast leg swing. A timing belt transmits the torque to the knee joint from the motor mounted coaxial to the hip axis. (C) Impact-robust and lightweight foot sensors measure three-axis contact forces and were used as contact switches. (D) A ruggedized IMU estimates the robot’s torso posture, angular rate, and linear acceleration at 250 Hz. (E) Real-time computer sbRIO 9606 from National Instruments controls the robot at 600 Hz. (F) The robot is powered by two three-cell lithium-polymer batteries in series. (G) A rigid and lightweight frame minimizes the robot mass.

  • Fig. 7 The BFI.

    (A) Custom HMI was designed to be transparent to the operator and to capture human motion data at high speed (1 kHz). (B) BFI has two underactuated modules that together track the position and orientation of the torso and apply forces to the operator’s CoM. (C) Each actuation module has three DoFs, one of which is a push/pull rod actuated by a DC brushless motor (Maxon EC90 Flat). A pair of load cells measures the actual forces applied to the operator. These sensors were not used for closed-loop force control. Force control was achieved via current-based torque control of the motors. (D) A series of linkages with passive joints were connected to the operator’s feet and track their spatial translation. (E) Real-time controller cRIO 9082 from National Instruments closed the BFI control loop and sampled data at 1 kHz. (F) A 3 feet–by–3 feet force plate estimated the operator’s CoP position and measured the shear and normal components of the operator’s net contact force.

  • Fig. 8 High-level teleoperation control loop with bilateral feedback.

    Human kinematic and dynamic data on the far left were transformed into a reference for the robot using the reduced model, the LIP. This transformation computed the desired DCM state ξH and the appropriate contact forces FxRref and FzRref that the robot must apply to the ground to reproduce human motion. With this information, the robot-embedded controller computed the required contact force to be applied by each foot. The feedback FBFI applied to the operator is proportional to the relative motion velocity between operator and robot.

  • Movie 1. Summary of the bilateral teleoperation strategy adopted in this work.

Supplementary Materials

  • robotics.sciencemag.org/cgi/content/full/4/35/eaav4282/DC1

    Fig. S1. Stances during teleoperation.

    Fig. S2. Swing foot vertical trajectory.

    Fig. S3. Right-to-left motion teleoperation via bilateral feedback.

    Fig. S4. Motion synchronization using force feedback.

    Fig. S5. Gantries used to constrain the robot during experiments.

    Fig. S6. Soft force sensors used for the robot feet.

    Fig. S7. Bipedal robot contact force control.

    Fig. S8. BFI force control.

    Movie S1. Teleoperation of stepping in place.

    Movie S2. Robot autonomous balancing controller.

    Movie S3. Teleoperation of constrained walking.

    Movie S4. Teleoperation of consecutive jumps.

    Movie S5. Compilation of unsuccessful stepping experiments.

  • Supplementary Materials

    The PDF file includes:

    • Fig. S1. Stances during teleoperation.
    • Fig. S2. Swing foot vertical trajectory.
    • Fig. S3. Right-to-left motion teleoperation via bilateral feedback.
    • Fig. S4. Motion synchronization using force feedback.
    • Fig. S5. Gantries used to constrain the robot during experiments.
    • Fig. S6. Soft force sensors used for the robot feet.
    • Fig. S7. Bipedal robot contact force control.
    • Fig. S8. BFI force control.

    Download PDF

    Other Supplementary Material for this manuscript includes the following:

    • Movie S1 (.mp4 format). Teleoperation of stepping in place.
    • Movie S2 (.mp4 format). Robot autonomous balancing controller.
    • Movie S3 (.mp4 format). Teleoperation of constrained walking.
    • Movie S4 (.mp4 format). Teleoperation of consecutive jumps.
    • Movie S5 (.mp4 format). Compilation of unsuccessful stepping experiments.

    Files in this Data Supplement:

Navigate This Article