Research ArticleSENSORS

Wireless steerable vision for live insects and insect-scale robots

See allHide authors and affiliations

Science Robotics  15 Jul 2020:
Vol. 5, Issue 44, eabb0839
DOI: 10.1126/scirobotics.abb0839
  • Fig. 1 A mechanically steerable wireless camera mounted on a darkling beetle and a small robot.

    (A) Wirelessly steerable camera system attached to the abdomen of a live darkling beetle. (B) A wireless, power-autonomous terrestrial robot with a steerable vision system. The camera can stream video to a smartphone, which can also command the robot to move and pan its camera left or right. (C) Exploded view showing all of the components of the steerable camera system including the Bluetooth chip, camera and optics, robotic head, high-voltage electronics, and battery. (D) Diagram showing the components of the mechanism used to steer the camera. (E and F) Close-up diagrams showing hinge motion as the camera pans right and left.

  • Fig. 2 Evaluating the wireless camera and arm performance.

    (A) Labeled diagrams showing piezo actuator operation. (B) Low-voltage input current versus high-voltage output generated by the boost converter. (C) Sample 160 × 120 images showing the performance of the camera using a 1.5-mm-diameter 1-mm-focal-length lens (Edmund Optics 43394). (D) Sample 160 × 120 images showing the performance of the camera using a 3.8-mm-diameter 2.33-mm-focal-length lens (Panasonic EYLGUDM128). (E) Frame rate versus line-of-sight range from the insect. Error bars indicate mean ± 1 SD (n = 10 frames); the rate remains constant until the sensitivity limit of the wireless link, and then begins to degrade. (F) Battery life when continuously streaming 160 × 120 images at different rates, with and without robotic head motion, and different batteries. (G) Weight breakdown of the Bluetooth, camera, and robotic head, as well as each component’s percentage of total weight.

  • Fig. 3 Field evaluation of accelerometer-triggered camera on a death-feigning beetle.

    (A) Close-up image of the wireless camera without a microrobotic arm on the back of a death-feigning beetle. (B) Experiment site with a grove of trees, a dry stream bed, and gravel paths. The beetle walked freely in four different locations. (C) Images from the camera showing a person walking. (D) Beetle motions detected by the accelerometer per minute over the 363-min experiment. The inactivity explains the improvement in battery life over continuous streaming. (E) Top 25 intervals between accelerometer triggers when the system is in sleep mode.

  • Fig. 4 Field evaluation of microrobotic arm on darkling beetle capturing panorama images.

    (A) Wireless camera system with the microrobotic arm attached to a darkling beetle. The camera was set to capture five images as it rotated in 15° steps to capture a panorama. (B) Aerial view of the experiment site in a gravel parking lot showing buses and trucks in the southwest corner. (C) Panorama showing the trucks and buses composed of five images captured by the insect-mounted camera while rotating 60°. (D) Beetle motions detected by the accelerometer per minute. (E) Top 25 intervals of beetle inactivity.

  • Fig. 5 Evaluation of the insect scale wireless robot.

    (A and B) Close-up images of the robot with vibration motors, three legs, a battery, Bluetooth chip, camera, and robotic head. (C) Effect of carrying payload greater than the camera system and the battery on robot speed while running the vibration motors at a constant power. Error bars indicate mean ± 1 SD (n = 5 trials). (D) Robot power consumption for different motion types and speeds. (E) Energy consumption for different angles when using the motors versus head to turn the camera. (F) Battery life for different robot speeds without video streaming, with 1 fps video streaming, and 1 fps video streaming while panning the camera.

  • Fig. 6 Navigation and focusing on another moving robot from the insect-scale robot.

    (A) Overhead view of the robot’s path when a human operator uses the camera to navigate. (B) Sample images from the robot’s camera during navigation. (C) Side view showing that our wireless robot is stationary with the camera positioned to look at another wired robot marked with an arrow moving to the left. The background shows numbers incrementing from right to left to indicate camera motion. (D) A top view of the scene shows the moving robot going across the field of view of the camera. The camera is rotated in discrete steps to maintain focus on the moving robot. (E) Images captured by the robot at each angle.

  • Table 1 Comparison with previous power-autonomous robots.

    RF, radio frequency. “-X-” indicates “not specified.”

    Power-autonomous
    robots
    CameraLength
    (cm)
    Motion power
    consumption (mW)
    Weight (g)Wireless
    comm.
    Kilobot (40)3.3-X->3*Infrared
    HAMR (41)4.53952.8RF
    RoACH (39)3-X-2.4Infrared
    DASH (42)1027716.2RF
    This work1.6332.8RF

    *This design does not specify the weight but uses a CR2032 coin cell battery that itself weighs more than 3 g.

    • Movie 1. Overview of a mechanically steerable vision system for insects and small robots

    • Movie 2. Accelerometer-triggered camera

      A darkling beetle (E. nigrina) is held facing a metric ruler. The insect motion triggers the camera to turn and stream images to a smartphone.

    • Movie 3. Sample real-time video streams.

      The smartphone sets the resolution and starts streaming video. The inset video shows a reference view of the scene (a person walking).

    • Movie 4. Navigation of an insect-scale robot using wireless vision.

      An insect-scale robot uses its camera to navigate around obstacles. The camera streams video to a smartphone, allowing a human to steer the robot.

    • Movie 5. Focusing on moving objects

      A stationary robot streams video of another robot moving across its field of view. A human operator steers the camera to maintain focus on the moving robot.

    • robotics.sciencemag.org/cgi/content/full/5/44/eabb0839/DC1

      Text

      Fig. S1. Smartphone interface.

      Fig. S2. Boost converter circuit schematic and drive signal waveforms.

      Fig. S3. Full system block diagram.

      Fig. S4. Sample images comparing lenses.

      Fig. S5. Frame rate versus distance and resolution using Bluetooth.

      Fig. S6. Distortion during motion.

      Table S1. Comparison table for image sensors.

      Table S2. Frame rate versus range at 1 Mbps.

      Table S3. Frame rate versus range at 2 Mbps.

      Table S4. Robot speed versus increasing weight.

      Movie S1. Camera motion angle measurement.

      Movie S2. Automatic light level adjustment.

      Movie S3. Smartphone-controlled camera steering.

      Movie S4. Capturing a panorama.

      Movie S5. Video from beetle walking on flat surface.

      Movie S6. Free walking beetle in the field.

      Movie S7. Robot speed when carrying payloads.

    • Supplementary Materials

      The PDF file includes:

      • Text
      • Fig. S1. Smartphone interface.
      • Fig. S2. Boost converter circuit schematic and drive signal waveforms.
      • Fig. S3. Full system block diagram.
      • Fig. S4. Sample images comparing lenses.
      • Fig. S5. Frame rate versus distance and resolution using Bluetooth.
      • Fig. S6. Distortion during motion.
      • Table S1. Comparison table for image sensors.
      • Table S2. Frame rate versus range at 1 Mbps.
      • Table S3. Frame rate versus range at 2 Mbps.
      • Table S4. Robot speed versus increasing weight.

      Download PDF

      Other Supplementary Material for this manuscript includes the following:

      • Movie S1 (.mp4 format). Camera motion angle measurement.
      • Movie S2 (.mp4 format). Automatic light level adjustment.
      • Movie S3 (.mp4 format). Smartphone-controlled camera steering.
      • Movie S4 (.mp4 format). Capturing a panorama.
      • Movie S5 (.mp4 format). Video from beetle walking on flat surface.
      • Movie S6 (.mp4 format). Free walking beetle in the field.
      • Movie S7 (.mp4 format). Robot speed when carrying payloads.

      Files in this Data Supplement:

    Stay Connected to Science Robotics

    Navigate This Article