Research ArticleSENSORS

A neuro-inspired artificial peripheral nervous system for scalable electronic skins

See allHide authors and affiliations

Science Robotics  17 Jul 2019:
Vol. 4, Issue 32, eaax2198
DOI: 10.1126/scirobotics.aax2198
  • Fig. 1 The ACES architecture for large arrays with single-wire transmission capability.

    Illustration of ACES artificial receptors on e-skin (left) that independently and asynchronously transduce tactile events into pulse signatures, analogous to biological action potentials, or spikes (right). (A) ACES receptors generate tactile events with spatiotemporal structures (dashed lines) that encode the stimulation sequence. (B) Pulse signatures are combined and propagated via a single conductor. (C) Decoders match pulses in time. A strong match with correlation |T| exceeding a predefined decoder threshold (shaded box) indicates the presence of an event by the particular receptor. The decoded events preserve the spatiotemporal structure (dashed lines) of receptor activation with ultra-high temporal precision, resembling the spike patterns that enable rapid discrimination in the somatosensory system.

  • Fig. 2 Architecture and performance characterization of ACES.

    (A) Multiple ACES receptors (blue blocks) are connected to a decoder (orange block) via a single electrical conductor. The red variable resistor represents the resistive sensors used in the current prototype. (B) Temporal precision of decoded events versus number of overlapping pulse signatures for different numbers of pulses per signature (W). Shaded regions indicate SD. (C) The influence of decoder threshold on the detection error tradeoff graph (W = 10). Dashed lines represent simulation results. Error bars indicate 95% confidence bounds for simulated results. (D) Influence of pulse width on DSIR. (E) Effect of W on DSIR (pulse width = 60 ns). Simulations were used to derive (D) and (E).

  • Fig. 3 ACES receptor responses using integrated multimodal sensors.

    (A) Schematic of a multimodal ACES sensor array, where biomimetic models convert tactile stimuli to events that are concurrently propagated. (B) Top: Estimation of time-varying force intensities based on ACES-FA and ACES-SA receptor output. Center and bottom: Decoded response of ACES-FA and ACES-SA receptors to a force profile. The ACES-SA receptor modulates pressure intensity by event frequency, whereas the ACES-FA receptor transmits FA+ and FA− events whenever it increases and decreases in pressure are experienced, respectively. (C) Photograph of the flexible multimodal sensor, wrapped onto a finger of a robotic hand grasping a cup of hot coffee. Inset: Photograph of the sensor. (D) Raw events generated from pressure-sensitive (top) and temperature-sensitive (bottom) ACES receptors during the grasp. (E) Derived pressure and temperature profiles from (D). (F) Snapshots of pressure and temperature distribution before and after grasp onset.

  • Fig. 4 Spatiotemporal patterns in the ACES-FA receptor response enable rapid slip detection.

    (A) Schematic of the event-based algorithm for movement estimation. The input consists of spatiotemporal events from ACES-FA receptors. Each intermediate node computes local movement estimates for every event occurring within its sensitive region. The global movement estimate is computed by pooling responses from intermediate nodes. (B) Detection of slip as a disc is pulled out of grasp. A sudden loss of string tension indicates slip onset. Estimates of movement and direction are shown in the bottom and inset panels, respectively. Blue trace indicates speed derived from optical tracking (see Materials and Methods). (C) Detecting slippage of a needle.

  • Fig. 5 Feature computation and classification of grating pitch.

    (A) Features extracted from an example trial where an ACES-FA receptor array is moved over a grating pattern with 4-mm pitch. Red trace in top panel indicates estimated tangential speed using ACES, whereas the dotted blue trace is the ground truth speed derived from frequency components (see Materials and Methods). Bottom panel is a 2D histogram of event frequencies over time. Color codes correspond to bin counts C normalized by total counts ΣC. Magenta box highlights the 100 × 1 vector of bin counts for classification. (B) Selected trials illustrating outputs from five different grating sizes. (C) Classification accuracy of grating sizes obtained when sampled at different rates. Error bars denote SD. (D) Comparison of classification accuracy achievable (with speed estimates) based on data rate consumed. For event-based output, the vertical line in the box plot is the median data rate, the caps indicate 1st to 99th percentiles, and the sides of the box indicate 25th to 75th percentiles of distribution.

  • Fig. 6 Classification results from the indentation of objects with various geometry and hardness.

    (A) A photograph of the sensor array used in the experiment, where classification speed and accuracy were compared using outputs from ACES-FA and ACES-SA receptors, for rapid indentation of (B) three soft shapes, (C) three rigid shapes, (D) two sharp cones of different hardness, (E) two broad cones of different hardness, and (F) two domes of different hardness. Shapes are shown in insets at top left.

  • Fig. 7 Flexible and dynamic reconfiguration of ACES receptor placement.

    Photographs of multiple ACES receptors dynamically arranged into three different formations on a sheet of conductive fabric. The plots below each photograph indicate the corresponding FA events decoded when pressure is applied on each formation. Colored boxes highlight the receptors that responded.

  • Fig. 8 Damage resilience of ACES architecture compared with a conventional row-column multiplexed array.

    (A) Photographs show the array before (top left) and after damage (bottom left), where the stretchable polyurethane substrate embedded with ACES-SA receptors was cut at three different locations. The corresponding pressure distributions reconstructed from ACES receptor outputs are shown to the right of each image. (B) Schematic of signal propagation path on damaged substrate of ACES receptor array. (C) Photographs of the four-by-four multiplexed sensor array before and after being cut. The plots to the right of each photograph indicate the sensor readout corresponding to the physical state of the sensor. (D) Schematic representation of the multiplexed sensor array. Each sensor element is represented by a variable resistor. Dashed lines indicate traces affected by the damage.

Supplementary Materials

  • robotics.sciencemag.org/cgi/content/full/4/32/eaax2198/DC1

    Text S1. Pressure sensor fabrication and characterization

    Text S2. Temperature sensor fabrication and characterization

    Fig. S1. Additional characterization of ACES signaling.

    Fig. S2. Characterization of transducers.

    Fig. S3. Example prototypes of ACES sensor arrays.

    Fig. S4. SPICE circuit used for simulation.

    Fig. S5. Setup for local curvature and hardness classification.

    Movie S1. A typical 5 × 5 cross-bar sensor array subjected to physical damage.

    Movie S2. Robustness of an ACES sensor array to physical damage.

  • Supplementary Materials

    The PDF file includes:

    • Text S1. Pressure sensor fabrication and characterization
    • Text S2. Temperature sensor fabrication and characterization
    • Fig. S1. Additional characterization of ACES signaling.
    • Fig. S2. Characterization of transducers.
    • Fig. S3. Example prototypes of ACES sensor arrays.
    • Fig. S4. SPICE circuit used for simulation.
    • Fig. S5. Setup for local curvature and hardness classification.

    Download PDF

    Other Supplementary Material for this manuscript includes the following:

    • Movie S1 (.mp4 format). A typical 5 × 5 cross-bar sensor array subjected to physical damage.
    • Movie S2 (.mp4 format). Robustness of an ACES sensor array to physical damage.

    Files in this Data Supplement:

Stay Connected to Science Robotics

Navigate This Article