Research ArticleHUMAN-ROBOT INTERACTION

When a robot teaches humans: Automated feedback selection accelerates motor learning

See allHide authors and affiliations

Science Robotics  20 Feb 2019:
Vol. 4, Issue 27, eaav1560
DOI: 10.1126/scirobotics.aav1560
  • Fig. 1 Normalized errors over the course of the study for one participant of the experimental group and his matched control.

    Training blocks (T) are indicated by dark gray background. Baseline (BL) and retention tests at the beginning of day 2 (RE2) and day 3 (RE3) are indicated in white background. No-feedback trials (NF) are indicated in light gray background. BL, RE2, and NF were used as test conditions to assess the spatial error during catch, drive, release, recovery (see Fig. 4), and the velocity error. Triangle face color represents the dominant error based on which feedback was provided.

  • Fig. 2 Development of the six primary outcomes over the main test conditions.

    Group means are connected between the baseline test on day 1 (BL) and the retention tests on day 2 (RE2) and day 3 (RE3). For each box, the central horizontal line is the median, the edges of the box are the 25th and 75th percentiles, and the whiskers extend to the most extreme data points and cover 99.3% of the data. Outliers are plotted individually (marked by a cross in the color of the corresponding group). Horizontal lines in group colors above the boxes indicate significant main effects of test within the groups. Small vertical bars attached to those horizontal lines indicate Bonferroni-corrected significant differences in the test conditions. Small black lines ending with group symbols and a star above lie between the test conditions and indicate significant group differences in learning rate between the two test conditions.

  • Fig. 3 Coauthor demonstrating the rowing simulator.

    Participants were seated in a real rowing boat placed in a CAVE. The participant’s task was to perform a predefined rowing oar movement only with arms and upper body (roll seat was strapped back to prevent the rower from bending the legs). When the virtually prolonged oar was moved through the virtual water, the rower saw, heard, and felt realistic oar-water interactions. In addition to rendering haptic oar-water interactions, the tendon-based parallel robot was used to haptically guide the participant if the related feedback design was selected. The current photo shows visual feedback during the drive phase (green and red dots drawn right above and below the blade).

  • Fig. 4 Desired oar movement.

    The arrowheads indicate the desired direction of the cyclic oar movement. The desired oar movement was divided into four phases: the region around the horizontal turning point before the oar enters the water (catch), the drive phase, the region around the horizontal turning point after the oar exits the water (release), and the region where the oar moves horizontally through air (recovery phase).

  • Fig. 5 Visualization of performance metrics mapping to the reference movement.

    DTW is used to correlate the measured spatial (A) and velocity (B) data to the reference trajectory. The two measured datasets show the worst and best performance measured during this study. The best performance was measured during position control. The spatial error (A) is indicated by the mean value over the entire rowing cycle in the color of the corresponding performance. The velocity error (B) is indicated by the mean value over the entire rowing cycle in the color of the corresponding performance.

  • Fig. 6 Study protocol for the three consecutive training/measurement days.

    The no-feedback trials lasted for 1 min (indicated by small block). All other measurement blocks lasted for 3 min. I, instruction; BL, baseline measurement; T, training; NF, no feedback trial; RE2, retention test on day 2; RE3, retention test on day 3.

  • Table 1 Dominant error for each participant for each training condition.

    Augmented feedback was only provided during the training conditions T1 to T5 on day 1 and T6 to T10 on day 2. Cat, spatial error at catch; Drv, spatial error during drive phase; Rel, spatial error at release; Rec, spatial error during recovery phase; Vel, velocity error over the entire cycle. Bold font indicates a match, meaning that both participants had the same dominant error by chance, and therefore, the control participant received feedback focusing on his/her dominant error as well.

    Day 1Day 2
    ParticipantT1T2T3T4T5T6T7T8T9T10
    Exp-1/Ctrl-1Drv/VelDrv/VelDrv/VelDrv/VelDrv/VelDrv/VelVel/VelDrv/VelDrv/VelVel/Vel
    Exp-2/Ctrl-2Vel/RecRec/RecRec/VelRec/VelVel/VelVel/DrvRec/RecVel/VelRec/VelVel/Drv
    Exp-3/Ctrl-3Vel/RecRec/RecRec/DrvVel/VelVel/DrvDrv/DrvRec/VelVel/VelVel/DrvVel/Drv
    Exp-4/Ctrl-4Vel/VelRec/VelCat/VelDrv/VelVel/DrvCat/VelVel/CatRec/CatCat/CatCat/Drv
    Exp-5/Ctrl-5Vel/VelDrv/RecVel/VelDrv/RecVel/VelVel/VelDrv/RecDrv/RecVel/RecVel/Rec
    Exp-6/Ctrl-6Vel/VelRec/VelVel/VelVel/VelRec/DrvVel/VelVel/DrvVel/VelVel/VelVel/Vel
    Exp-7/Ctrl-7Vel/VelVel/VelVel/VelVel/VelVel/VelVel/VelRec/VelVel/VelVel/VelVel/Vel
    Exp-8/Ctrl-8Rec/RecRec/DrvRec/RecRec/DrvRec/RecRec/DrvRec/DrvDrv/RecVel/VelRec/Vel
  • Table 2 Learning effects from the baseline test on day 1 (BL) to the retention tests on day 2 (RE2) and day 3 (RE3) between and within groups.
    VariableInteraction and between group effectsWithin experimental groupWithin control group
    Group(2)*Test(3) groupPEmbedded ImageMain effect post hocPEmbedded ImageMain effect post hocPEmbedded Image
    Spatial error at catchF1.40,19.57 = 0.02
    F1,14 = 0.08
    0.943
    0.782
    0.002
    0.006
    F2,14 = 0.610.5590.080F2,14 = 0.860.4440.110
    Spatial error recovery phaseF1.33,18.61 = 0.45
    F1,14 = 0.00
    0.566
    0.992
    0.031
    0.000
    F2,14 = 10.79
    BL–RE2
    BL-RE3
    0.001
    0.019
    0.069
    0.606F1.23,8.57 = 2.700.1340.278
    Spatial error drive phaseF2,28 = 1.56
    F1,14 = 0.69
    0.228
    0.421
    0.100
    0.047
    F2,14 = 7.07
    BL-RE2
    BL-RE3
    0.008
    0.045
    0.116
    0.502F2,14 = 0.880.4380.111
    Spatial error at releaseF1.41,19.76 = 1.06
    F1,14 = 0.01
    0.341
    0.943
    0.071
    0.000
    F2,14 = 21.57
    BL-RE2
    BL-RE3
    <0.001
    0.006
    0.004
    0.755F2,14 = 4.01
    BL-RE2
    BL-RE3
    0.042
    0.223
    0.215
    0.364
    Velocity error entire cycleF2,28 = 2.52
    F1,14 = 0.09
    0.099
    0.768
    0.152
    0.006
    F2,14 = 23.18
    BL-RE2
    BL-RE3
    <0.001
    0.003
    0.004
    0.768F2,14 = 10.16
    BL-RE2
    BL-RE3
    0.002
    0.122
    0.005
    0.592
    Spatial error entire cycleF1.405,19.67 = 0.54
    F1,14 = 0.07
    0.532
    0.801
    0.037
    0.005
    F2,14 = 6.78
    BL-RE2
    BL-RE3
    0.009
    0.031
    0.196
    0.492F1.12,7.86 = 1.990.1990.221

Supplementary Materials

  • Supplementary Materials

    The PDF file includes:

    • Table S1. Between- and within-group differences during training with and without feedback.

    Download PDF

    Other Supplementary Material for this manuscript includes the following:

    • Movie S1 (.mp4 format). M3-rowing simulator

    Files in this Data Supplement:

Stay Connected to Science Robotics

Navigate This Article