Research ArticleCOMPUTER VISION

Efficient nonparametric belief propagation for pose estimation and manipulation of articulated objects

See allHide authors and affiliations

Science Robotics  22 May 2019:
Vol. 4, Issue 30, eaaw4523
DOI: 10.1126/scirobotics.aaw4523
  • Fig. 1 A robot needs to estimate the pose of a cabinet to operate on it.

  • Fig. 2 Probabilistic graphical model of an articulated object.

    A cabinet with three drawers is converted to a probabilistic graphical model with hidden nodes Xs representing the poses of different parts and observed nodes Ys connected to each of the hidden nodes. Edges between the hidden nodes capture the articulation constraints between them.

  • Fig. 3 Pose estimation results of a cabinet in two different scenes using PMPNBP and particle filter.

    (A and G) Observed scene with the cabinet. (B and H) Point cloud observation of the cabinet. (D and J) Beliefs at iteration 0. (E and K) Beliefs at iteration 100. (F and L) MLE of the cabinet pose at iteration 100. (C and I) Error comparison of PMPNBP and particle filter—400 particles, over 10 runs.

  • Fig. 4 Pose estimation of articulated objects under occlusions using PMPNBP.

    (A and E) RGB observations. (B and F) Point cloud observations. (C and G) Pose estimation results from the first viewpoint. (D and H) Pose estimation results from the second viewpoint.

  • Fig. 5 An example of maintaining beliefs aid in robot planning.

    The robot’s task is to open the bottom drawer. (A and D) RGB observation of the scene. (B and E) MLE of the cabinet in the scene. (C and F) Confidence ellipsoids of the pose estimation generated by the covariance of beliefs. (G and H) Robot operating the bottom drawer using the MLE from (E), because confidence ellipsoids in (F) are within the provided thresholds.

  • Fig. 6 2D articulation pattern and its graphical model.

    The pattern used for the experiments has nine nodes with one circle at the center and four arms with two links each as shown in (A). This forms the graphical model shown in (B), where hidden nodes Xs are connected to their neighbors and informed by observed nodes Ys. Geometrically, the circle and links are defined by their location (xs, ys), orientation and dimensions as shown in (A).

  • Fig. 7 PMPNBP convergence on 2D patterns and its comparison with PAMPAS.

    (A) 2D observation without occlusion (circle is visible). (B to D) MLE of the 2D pattern at iterations 1, 10, and 24, respectively. (E) 2D observation with occlusion (circle is not visible). (F to H) MLE of the 2D pattern at iterations 1, 10, and 34, respectively. (I) Average error of convergence of PMPNBP and PAMPAS with respect to the number of iterations. (J) CPU run time per message update iteration with respect to the number of particles. (Best viewed in color).

Supplementary Materials

  • robotics.sciencemag.org/cgi/content/full/4/30/eaaw4523/DC1

    Fig. S1. Unary potential illustration.

    Fig. S2. Pairwise potential illustration.

    Fig. S3. More results on pose estimation of a cabinet under partial occlusion.

    Fig. S4. Pose estimation of a Fetch robot.

    Fig. S5. Pose estimation of a Fetch robot with simulated occlusion.

    Fig. S6. Illustrative overview of Message and Belief update algorithms.

    Fig. S7. PMPNBP results with circle node observed in the 2D articulated pattern estimation.

    Fig. S8. PMPNBP results with circle node “occluded” in the 2D articulated pattern estimation.

    Movie S1. Research summary.

  • Supplementary Materials

    The PDF file includes:

    • Fig. S1. Unary potential illustration.
    • Fig. S2. Pairwise potential illustration.
    • Fig. S3. More results on pose estimation of a cabinet under partial occlusion.
    • Fig. S4. Pose estimation of a Fetch robot.
    • Fig. S5. Pose estimation of a Fetch robot with simulated occlusion.
    • Fig. S6. Illustrative overview of Message and Belief update algorithms.
    • Fig. S7. PMPNBP results with circle node observed in the 2D articulated pattern estimation.
    • Fig. S8. PMPNBP results with circle node “occluded” in the 2D articulated pattern estimation.

    Download PDF

    Other Supplementary Material for this manuscript includes the following:

    • Movie S1 (.mp4 format). Research summary.

    Files in this Data Supplement:

Stay Connected to Science Robotics

Navigate This Article