Research ArticleMEDICAL ROBOTS

Intelligent magnetic manipulation for gastrointestinal ultrasound

See allHide authors and affiliations

Science Robotics  19 Jun 2019:
Vol. 4, Issue 31, eaav7725
DOI: 10.1126/scirobotics.aav7725
  • Fig. 1 System description.

    (A) A conceptual image of the in vivo RCE showing real-time acquisition of localized μUS images where ultrasonic features consistent with the histological layers of the bowel wall have been annotated. The RCE contains (a) μUS transducers, (b) an LED light source, (c) an irrigation channel, (d) a camera, (e) an IPM with circuitry that facilitates real-time pose estimation of the device, and (f) a soft, flexible tether. (B) The benchtop setup showing key components, as well as a closeup view of the RCE prototype.

  • Fig. 2 B-scans generated during RCE benchtop trials.

    (A) B-scan images with the corresponding tilt and tissue-coupling force, collected during a force sensitivity test. The black arrow identifies a wall echo from our phantom. (B) Autonomous echo detection “echo maintenance” routine with three disturbances. The wall echo from the phantom is indicated by black arrows. The change in the observed depth of the back wall is attributed to variability in the phantom thickness, RCE contact force, RCE tilt, and thickness of coupling medium.

  • Fig. 3 Experimental setup and results for in vivo evaluation of our RCE.

    (A) The in vivo experimental setup in the endoscopy suite. (B) A B-scan acquired during an in vivo trial and the corresponding ESR with the set threshold (red line). Features, characteristic of bowel wall layers, are annotated with black arrows. A deeper tissue feature is shown with a white arrow.

  • Fig. 4 System schematic.

    A diagram that summarizes the major components in the system and the data flow between them.

  • Fig. 5 Block diagram control schematic of our RCE system.

    The US system components are shown in yellow, the magnetic control system is shown in blue, and the auxiliary inputs and outputs are shown in green.

  • Table 1 Autonomous RCE motion (open loop in μUS).

    A summary of the tests carried out to assess the RCE’s ability to acquire μUS signals while in motion. Two configurations (A and B) were chosen to show the efficacy and repeatability of autonomous positioning for μUS acquisition in a defined environment. All the repetitions were performed on a horizontally oriented silicone phantom (fig. S1B).

    TestDescriptionControl strategyResults
    RCE approach configuration (A)From arbitrary start conditions,
    approach −2° RCE tilt and
    1-N coupling force.
    AutonomousConfiguration reached. 40% of repetitions acquired
    μUS images (n = 5).
    RCE approach configuration (B)From arbitrary start conditions,
    approach 3° tilt and 0.9-N coupling force.
    AutonomousConfiguration reached. 100% of repetitions acquired
    μUS images (n = 5).
    Linear trajectoryMove RCE forward in linear trajectory
    (8 cm) while maintaining 0° to
    1° tilt and 0.6-N coupling force.
    TeleoperatedTrajectory complete in 100% of repetitions. Weak
    acoustic coupling achieved in all trials. Intermittent
    regions of strong echoes observed (n = 5).
    AutonomousTrajectory complete in 100% of repetitions.
    Weak acoustic coupling achieved in all trials.
    Intermittent regions of strong echoes observed (n = 5).
  • Table 2 Echo detection (closed loop in μUS).

    A summary of the tests carried out to assess the efficacy of using μUS feedback. During echo detection, two substrate orientations were chosen to show repeatability and environmental adaptation. During echo maintenance, the external disturbances were in the form of pulling the tether and manually disturbing the pose of the EPM; these were included to simulate the presence of uncertainties that may be encountered in clinical practice. All repetitions were performed on a silicone phantom (fig. S1B).

    TestDescriptionSubstrate orientationControl strategyResults
    Echo detectionFrom arbitrary start pose, adjust force and
    tilt until an echo is detected
    (i.e., ESRthresh exceeded).
    Horizontal (tilt = 0°)TeleoperatedEchoes found in 100% of repetitions (n = 5).
    AutonomousEchoes found in 100% of repetitions (n = 5).
    Tilted*TeleoperatedEchoes found in 100% of repetitions (n = 5).
    AutonomousEchoes found in 100% of repetitions (n = 5).
    Echo maintenanceAdjust force and tilt to find echo and then
    react to external disturbances and
    attempt to maintain it.
    Horizontal (tilt = 0°)AutonomousEchoes found and reestablished
    after disturbances in 80% of
    repetitions (n = 5).

    *A fixed but unknown (to the system) arbitrary value.

    †An example B-scan from these tests is shown in Fig. 2B.

    ‡The unsuccessful repetition was attributed to a lack of US gel.

    • Table 3 Benchtop validation tests.

      A summary of the tests carried out to validate the ESR control algorithm and demonstrate the capability of using the system to create spatially relevant US images—performed on an agar-based phantom. To better mimic the uncertainty in in vivo conditions, the phantom was tilted by about 2°, unknown to the system.

      TestDescriptionControl strategyResults
      ESR validationFrom arbitrary start pose,
      adjust tilt until acoustic
      coupling is achieved
      (i.e., ESRthresh exceeded).
      Experiment duration: 90 s.
      Teleoperated (n = 10)Robust acoustic coupling achieved in
      70% of repetitions, small amplitude echoes
      observable in other 30% of repetitions (n = 10).
      Autonomous (n = 10)Acoustic coupling achieved in 100% of repetitions (n = 10).
      Spatial scanning
      (combined locomotion
      and echo detection)*
      Autonomous linear motion while
      maintaining acoustic coupling.
      When coupling is lost, RCE enters
      echo-search routine by adjusting tilt.
      Autonomous (n = 10, three
      features in each trajectory)
      100% of large (10 mm wide) features located.
      66.7% of small (3 mm wide) features located.
      Error in measuring length of gaps between features:
      0.98 ± 0.91 mm (n = 28 of 40 possible).
      Error in measuring all feature widths:
      1.08 ± 0.89 mm (n = 50 of 60 possible).

      *A representative image taken during one of these repetitions is shown in fig. S4.

      • Table 4 Concept validation tests (in vivo).

        A summary of the tests carried out to validate the concept, performed in a living porcine model. The autonomous echo detection tests were monitored by the clinician for safety reasons, but no assistance was given to the system.

        TestDescriptionResults
        Teleoperated echo
        detection
        Adjust force and tilt manually to find and maintain
        US coupling (i.e., ESRthresh exceeded).
        High cognitive burden. Only low-amplitude echoes seen and only in
        40% of repetitions (n = 5).
        Autonomous echo
        detection*
        Adjust force and tilt autonomously to find and maintain US
        coupling (i.e., ESRthresh exceeded).
        Low cognitive burden. Echoes seen in 100% of repetitions, with
        60% being very distinct (n = 5).

        *An in vivo B-scan taken during these tests is shown in Fig. 3B.

        Supplementary Materials

        • robotics.sciencemag.org/cgi/content/full/4/31/eaav7725/DC1

          A description of the experimental data submitted with the manuscript

          Fig. S1. RCE roll and tilt conventions and the benchtop phantoms.

          Fig. S2. Characterization of tilt and roll ranges using time series of filtered waveforms.

          Fig. S3. Series of filtered waveforms used to compute ESR during benchtop trials.

          Fig. S4. Spatially relevant B-scan generation.

          Fig. S5. Filtering of an in vivo μUS image.

          Fig. S6. Agar phantom manufacture.

          A description of the agar phantom and manufacture protocol

          Table S1. Agar phantom composition.

          A description of magnetic actuation methods

          Parylene C coating and RCE leak testing

          Data S1 (Zipped folder). All experimental data referenced in the manuscript.

          Movie S1 (.mp4 format). Concept overview.

          Movie S2 (.mp4 format). Benchtop testing.

          Movie S3 (.mp4 format). In vivo testing.

        • Supplementary Materials

          The PDF file includes:

          • A description of the experimental data submitted with the manuscript
          • Fig. S1. RCE roll and tilt conventions and the benchtop phantoms.
          • Fig. S2. Characterization of tilt and roll ranges using time series of filtered waveforms.
          • Fig. S3. Series of filtered waveforms used to compute ESR during benchtop trials.
          • Fig. S4. Spatially relevant B-scan generation.
          • Fig. S5. Filtering of an in vivo μUS image.
          • Fig. S6. Agar phantom manufacture.
          • A description of the agar phantom and manufacture protocol
          • Table S1. Agar phantom composition.
          • A description of magnetic actuation methods
          • Parylene C coating and RCE leak testing

          Download PDF

          Other Supplementary Material for this manuscript includes the following:

          • Data S1 (Zipped folder). All experimental data referenced in the manuscript.
          • Movie S1 (.mp4 format). Concept overview.
          • Movie S2 (.mp4 format). Benchtop testing.
          • Movie S3 (.mp4 format). In vivo testing.

          Files in this Data Supplement:

        Stay Connected to Science Robotics

        Navigate This Article