Research ArticlePROSTHETICS

Controlling sensation intensity for electrotactile stimulation in human-machine interfaces

See allHide authors and affiliations

Science Robotics  25 Apr 2018:
Vol. 3, Issue 17, eaap9770
DOI: 10.1126/scirobotics.aap9770

Abstract

A barrier to practical use of electrotactile stimulation for haptic feedback has been large variability in perceived sensation intensity because of changes in the impedance of the electrode-skin interface, such as when electrodes peel or users sweat. We show how to significantly reduce this variability by modulating stimulation parameters in response to measurements of impedance. Our method derives from three contributions. First, we created a model between stimulation parameters and impedance at constant perceived sensation intensity by looking at the peak pulse energy and phase charge. Our model fits experimental data better than previous models [mean correlation coefficient (r2) > 0.9] and holds over a larger set of conditions (participants, sessions, magnitudes of sensation, stimulation locations, and electrode sizes). Second, we implemented a controller that regulates perceived sensation intensity by using our model to derive a new current amplitude and pulse duration in response to changes in impedance. Our controller accurately predicts participant-chosen stimulation parameters at constant sensation intensity (mean r2 > 0.9). Third, we demonstrated as a proof of concept on two participants with below-elbow amputations—using a prosthesis with electrotactile touch feedback—that our controller can regulate sensation intensity in response to large impedance changes that occur in activities of daily living. These results make electrotactile stimulation for human-machine interfaces more reliable during activities of daily living.

INTRODUCTION

Haptic feedback is a critical part of many human-machine interfaces. One way to provide haptic feedback is through electrotactile stimulation, which involves the application of electrical current over the skin to stimulate sensory nerves. Compared with mechanotactile stimulation, such as from vibration motors or actuated pins, electrotactile stimulation has advantages in that it has low latency, it is energy-efficient, and it can be delivered in a small, unobtrusive form factor (1, 2).

By varying the waveform, frequency, location, or electrodes, various sensations can be elicited, including vibration, touch, tingling, itching, pinching, pressure, and pain (1). Because these sensations can be elicited with very small amounts of current (1 to 10 mA) for very short amounts of time (200 to 1000 μs), sensory neurons located more superficially can be stimulated without activating deeper motor neurons that would cause muscle contractions. Furthermore, whereas invasive methods of providing sensation to users have been successful (3, 4), the noninvasiveness of electrotactile stimulation makes it easily applicable and useful in a greater number of applications, such as conveying texture in a multi-touch display (5), touch information in a virtual reality setting (6), suture tension to a physician teleoperating a surgical robot (7), or touch and proprioception information to people with amputations who wear prostheses (8).

A barrier to practical use of electrotactile stimulation is the large variability in perceived sensation intensity that derives from changes in the impedance of the electrode-skin interface. The changes in impedance may be caused by mechanical disturbances, such as varying contact between the electrode and the skin, or by physiological disturbances, such as sweating at the electrode sites (1). Researchers typically estimate impedance by measuring the voltage across stimulation electrodes and computing a resistance by dividing by the applied current, detailed in note S1 (1, 911). Like the impedance, the resistance also changes in response to external disturbances such as peeling back electrodes (10, 11). For example, Fig. 1 shows current passing through the skin across various mechanoreceptors and nerve endings to produce sensation. As the electrode peels, the current becomes concentrated in a smaller area on the surface of the skin because of poor contact and can result in a shock.

Fig. 1 Variability in sensation intensity due to changes in impedance during electrotactile stimulation.

(A) Current flows across the mechanoreceptors in the skin to produce sensation. When the electrode is peeled back, current density increases, resulting in a stronger sensation. (B) Current stimulation waveform (top), with I = 2 mA, T = 200 μs at 20 Hz, delivered across the skin and the resulting measured voltage waveform (bottom). The mean and SD for an individual pulse are shown as the participant peeled back and reapplied an electrode. The recorded voltage, Vp, is the peak of the voltage waveform and is used as a measure of the steady-state voltage, Vss (note S1). (C) Changes in the peak resistance of the electrode-skin interface due to peeling back and reapplying the electrode. As more of the electrode is peeled back, the sensation gets stronger, leading to discomfort. The mean and SD over five trials are shown.

Efforts have been made to reduce variability in perceived sensation by modulating stimulation parameters, specifically current amplitude (I) and pulse duration (T), in response to measurements of impedance (1, 1012). To work well, these approaches require knowing precisely the relationship between stimulation parameters, impedance, and perceived sensation. In finding this relationship, researchers typically use psychophysical methods to quantify perceived sensation, in particular the method of adjustment (10, 11). Using the method of adjustment, researchers find stimulation parameters at varying impedance values that result in the same perceived sensation. Existing characterizations of this relationship are able to reduce variability in perceived sensation but rely on poor models relating sensation intensity to stimulation parameters. For example, seminal work by Tachi et al. (10) equated a constant sensation intensity with a constant pulse energy. This work was based on the assumption that impedance is independent of applied current, an assumption that is known to be false (1, 13). More recent work by Kajimoto (11) discarded this assumption but relied on a relationship between pulse duration and impedance with a low correlation coefficient (r2 = 0.359). We considered an r2 value greater than 0.7 to denote a strong correlation (14), suggesting that Kajimoto’s results may not have been consistent across different individuals, magnitudes of sensation, and locations of stimulation. Kantor et al. (15) observed that phase charge remains nearly constant for constant sensation intensity, but other studies, such as the one by Bowman and Baker (16), provide evidence that phase charge can vary even while perceived sensation intensity remains constant. In our previous work (17), we presented a preliminary model suggesting that, at a constant sensation intensity, both pulse energy and phase charge vary linearly with impedance. We observed these linear relationships for five participants across two sessions and two magnitudes of sensation. We also investigated linear relationships for two participants across three stimulation locations and two electrode sizes. Although our average r2 values were greater than 0.9 across all conditions, a one-tailed Wilcoxon signed-rank test with Bonferroni-Holm correction indicated that the r2 values were not statistically significantly greater than 0.7 across each condition (Z < 1.89, P > 0.05) because of the low sample sizes.

In this paper, we present a method of reducing variability in perceived sensation intensity during electrotactile stimulation due to changes in impedance. In our first experiment (exp. 1), we modeled the relationship between stimulation parameters and impedance at constant perceived sensation intensity across 10 participants. In our second experiment (exp. 2), based on the model from exp. 1, we implemented a controller that regulates perceived sensation intensity. We validated this controller in three subexperiments (exps. 2A to 2C), comparing controller-derived stimulation parameters with participant-derived values. In exp. 2A, we used electroconductive gel to vary impedance, and the same 10 participants matched sensation intensity to electrotactile stimulation provided by separate reference electrodes. In exp. 2B, we again used electroconductive gel to vary impedance; however, 10 new participants matched the sensation intensity to a vibrotactile reference. In exp. 2C, the same 10 participants from exp. 2B varied impedance by exercising and resting, and again matched the sensation intensity to a vibrotactile reference. In both exps. 1 and 2, a one-tailed Wilcoxon signed-rank test with Bonferroni-Holm correction was used to determine statistical significance. In our final experiment (exp. 3), we demonstrated as a proof of concept on two participants with below-elbow amputations that our controller can regulate sensation intensity in response to large impedance changes that occur from poor electrode contact (exp. 3A) and in activities of daily living (exp. 3B). These results make electrotactile stimulation for nerve stimulation and haptic feedback more reliable in human-machine interfaces.

RESULTS

Experiment 1: Ep and Q vary linearly with Rp at constant sensation intensity

To expand on our previous work (17) and further develop a model between stimulation parameters, impedance, and perceived sensation intensity, we recruited a total of 10 volunteers for experiment (exp.) 1 to test eight experimental conditions that included different sessions (A and B on separate days), magnitudes of sensation (weak and strong), stimulation locations (forearm, biceps, and back), and electrode sizes (20 mm by 15 mm and 28 mm by 20 mm). In all conditions, constant current electrotactile stimulation was delivered to the participant. Using the method of adjustment as in Tachi et al. (10), participants adjusted the current amplitudes of pulses with different pulse durations to match a reference sensation intensity. The pulse duration (T), participant-chosen current amplitude (I), and peak voltage (Vp) were recorded for 10 different values of T for each of the eight conditions. Our results from using this protocol to compare I with T are consistent with previous studies (1, 10, 18), as discussed further in note S2 and fig. S1.

All participants across all conditions show linear trends for both peak pulse energy (Ep) versus peak resistance (Rp) values and phase charge (Q) versus peak resistance (Rp) values that produced the same perceived sensation intensity. Furthermore, the best-fit lines for every condition tend to originate from a common point. On the basis of this trend, we determined values for the points of convergence for both Ep versus Rp and Q versus Rp. The data for 1 of the 10 participants across all conditions, along with their respective linear regressions constrained to the points of convergence, are shown in Fig. 2 (A to H). Data from all 10 participants constrained to the points of convergence are shown in fig. S2. The r2 values from the constrained linear regression are reported in Fig. 2I. Across the 10 participants × 8 conditions (n = 80), the average value for Ep versus Rp was 0.941 and for Q versus Rp was 0.925, where an r2 value of 1 denotes a perfect fit to the data, indicating very strong linear trends for both Ep versus Rp and Q versus Rp. The r2 values for each of the eight conditions across all 10 participants are statistically significantly greater than 0.7 for both Ep versus Rp (Z = 2.75, P < 0.01) and Q versus Rp (Z > 2.14, P < 0.05), denoting a strong correlation. The r2 distributions for each condition are shown in Fig. 2 (J and K). When comparing weak and strong magnitudes of sensation, we observed that strong sensations had higher linear slopes in every case.

Fig. 2 Results from exp. 1 modeling the relationship of peak resistance (Rp) to peak pulse energy (Ep) and phase charge (Q) at constant sensation intensity.

(A to D) Ep versus Rp and (E to H) Q versus Rp plots showing all trials from a single participant across sessions, magnitudes of sensation, stimulation locations, and electrode sizes. Each sensation felt at each of the 11 data points of the same condition was equivalent in subjective intensity. Best-fit lines were constrained to go through the origin in (A) to (D), whereas all the lines in (E) to (H) were constrained to go through an optimal point of convergence (−1.67 kilohms, 0.186 μC). Each best-fit line represents a line of constant sensation intensity. (I) r2 regression statistics constrained to the origin for Ep versus Rp and the point of convergence for Q versus Rp across 10 participants and 8 conditions (n = 80). The eight conditions, described in the figure key, cover two sessions, two magnitudes of sensation, three stimulation locations, and two electrode sizes. The r2 distributions for each of the eight conditions are shown in (J) for Ep versus Rp and in (K) for Q versus Rp. A signed-rank test indicated that the r2 values are statistically significantly greater than 0.7 (P < 0.05), denoting a strong correlation across all conditions. Blue crosses represent outlier data points beyond 1.5 times the interquartile range.

According to Eq. 7, because Ep = RpI2T, it holds that when Rp = 0 kilohm, Ep = 0 erg. Consequently, this dictates that all linear trends between the two variables should go through the origin. Furthermore, the slopes of these lines equal I2T, meaning that at constant sensation, the value of I2T is constant. A similar result was found by Tachi et al. (10), but an erroneous assumption led them to a different conclusion, as described in note S3. Because the value of Q is not defined by Rp according to Eq. 6, we solved for the optimal point of convergence over all conditions. Using constrained linear regression and a grid search, the optimal point of convergence was found to be Rp = −1.67 kilohms, Q = 0.186 μC, as described in note S4.

The model can be used to implement a controller

Using the linear relationships between Ep versus Rp and Q versus Rp, we can compute stimulation parameters I and T in response to changes in Rp. Figure 3 shows a block diagram of this process. Suppose a participant is initially stimulated with a monophasic square pulse at some initial values of I0 and T0. Given these values of I0 and T0, we can measure Rp0 after the initial pulse and compute the slopes of the lines of constant sensation intensity using the following two equations,Embedded Image(1)where mE is the slope of the constant sensation line from Ep versus Rp and is independent of Rp0, andEmbedded Image(2)where mQ is the slope of the constant sensation line for Q versus Rp and (x*, y*) is the point of convergence of all the lines of constant sensation for Q versus Rp.

Fig. 3 Block diagram for our controller that modulates stimulation parameters to keep perceived sensation intensity constant.

First, a desired sensation intensity is chosen, and the slopes of the lines of constant sensation intensity, mE and mQ, are calculated by a computer using the linear relationships determined by the modeling experiment. The value of mQ is used to compute Embedded Image, the desired value of Q from the line of constant sensation intensity relating Q to Rp, and is determined by the value of Rp at the previous time step. Using mE and Embedded Image, the current (I) and pulse duration (T) used for stimulation are computed, and the appropriate waveform is delivered to the participant. The time-varying voltage (V) is measured across the electrodes by the sensor, whose peak value (Vp) is divided by I to obtain the peak resistance (Rp) used in the next iteration.

Now that we know the slopes of the lines of constant sensation intensity we want to remain on, suppose Rp changes in response to a mechanical disturbance, such as an electrode peeling off. We can solve for the next values of I and T using the equationsEmbedded Image(3)where Embedded Image is the desired Q determined from the Q versus Rp line of constant sensation intensity. Rearranging these equations, we obtainEmbedded Image(4)These values of I and T produce values of Ep and Q that are on the original lines of constant sensation intensity for the measured value of Rp, and we would send these stimulation parameters to the participant. Consequently, holding mE (Eq. 1) and mQ (Eq. 2) constant would result in a constant perceived sensation intensity.

The linear relationship we have found for Q versus Rp is critical to computing stimulation parameters for constant sensation intensity in response to impedance changes, although it has not been considered in previous studies. As previously mentioned, the value of I2T should be constant to maintain a constant sensation intensity. However, simply holding this value constant is not sufficient because it does not account for changes in impedance. Recall the example in Fig. 1C. In this example, I and T were held fixed, thus making I2T constant. Nevertheless, changes in sensation intensity—corresponding to the changes in Rp shown—were still felt despite I2T being constant. Consequently, an additional relationship between the stimulation parameters I and T and Rp is necessary. Because by definition Q = IT (Eq. 6), the linear relationship for Q versus Rp satisfies this requirement.

Exp. 2A: The controller regulates sensation intensity when Rp varies

To validate the controller, 10 participants volunteered to test five experimental conditions including different magnitudes of sensation (weak and strong), stimulation locations (forearm, biceps, and back), and electrode sizes (20 mm by 15 mm and 28 mm by 20 mm). Participants wore a testing and reference pair of electrodes placed in the same location but on contralateral sides of the body. Electroconductive gel was applied to or removed from the testing pair of electrodes to change the impedance of the electrode-skin interface a total of three times for each condition. Upon changing the impedance, the controller would compute the new stimulation parameters I and T to keep sensation intensity constant. At the new Rp and controller-computed T, participants adjusted I on the testing pair of electrodes to match the perceived sensation intensity from the reference pair of electrodes.

Data showing the results from the controller experiment for both Ep versus Rp and Q versus Rp from a single participant across all five experimental conditions are plotted in Fig. 4 (A and B). The results from all 10 participants are shown in figs. S3 and S4. To evaluate the effectiveness of the controller, the r2 value was computed to determine how well the line of constant sensation intensity for a single condition fit the three participant-derived values for both Ep versus Rp and Q versus Rp. The r2 values from the linear regression are reported in Fig. 4E. Across the 10 participants and 5 conditions (n = 50), the average r2 for Ep versus Rp was 0.941 and for Q versus Rp was 0.917. As with the modeling results, these linear relationships held across 10 different participants, both magnitudes of sensation, both electrode sizes, and all three stimulation locations. The r2 values for each of the five conditions across all 10 participants are statistically significantly greater than 0.7 for both Ep versus Rp (Z > 2.65, P < 0.05) and Q versus Rp (Z = 2.75, P < 0.05). The r2 distributions for each condition are shown in Fig. 4 (F and G). These large values of r2 strongly suggest that these lines computed by the controller represent constant sensation intensities, even when there are up to 40-kilohm changes in the measured Rp of the electrode-skin interface.

Fig. 4 Results from exp. 2 validating the controller.

For every condition, participants were asked to match a reference sensation intensity at three differing values of Rp. (A) Peak pulse energy (Ep) and (B) phase charge (Q) for a participant across two magnitudes of sensation, three stimulation locations, and two electrode sizes using electroconductive gel to change Rp and an electrotactile reference. (C) Ep and (D) Q for a participant with a below-elbow amputation (participant TR1) at a weak magnitude of sensation over the right biceps using a vibrotactile reference and either electroconductive gel or exercise to change Rp. In (A) to (D), initial pre-gel/exercise values of Ep and Q versus Rp (blue) are used to compute lines of constant sensation intensity (dashed lines). When Rp changes, the controller computes new stimulation parameters to stay on the lines of constant sensation intensity (red). At the controller-computed pulse duration, participants adjusted the current amplitude to match a constant reference sensation intensity, and we derived Ep and Q (green). (E) r2 regression statistics from fitting the controller-computed lines of constant sensation intensity to the participant-derived values of Ep and Q across 10 participants without arm impairment in exp. 2A (10 participants × 5 conditions, n = 50) as well as 9 participants without arm impairment and participant TR1 in exps. 2B and 2C (10 participants × 1 condition, n = 10). The r2 distributions are shown in (F) for Ep versus Rp and in (G) for Q versus Rp. A signed-rank test indicated that the r2 values are statistically significantly greater than 0.7 (P < 0.05). Blue crosses represent outlier data points beyond 1.5 times the interquartile range.

Exps. 2B and 2C: The controller regulates sensation intensity after exercise

To simulate changes in electrode-skin interface impedance during activities of daily living, we had nine new participants without impairment and a participant with a right below-elbow amputation (participant TR1: male, age 39) perform an experiment similar to exp. 2A, replacing gel with sweat generated from exercise as the means to change the impedance. Participant TR1 was included to demonstrate the application of electrotactile stimulation in a prosthesis, described in exp. 3. In exps. 2B and 2C, an electrotactile reference could not be used because exercise would change the impedance of the reference electrodes, thus changing the perceived electrotactile sensation intensity. Instead, a vibrotactile reference was used on the left biceps, under the assumption that changes in skin impedance have little to no effect on perceived vibrotactile stimulation intensity.

To verify that vibrotactile stimulation could be used as a reference to compare with electrotactile stimulation, we first repeated the electroconductive gel experiment, similar to exp. 2A, for all 10 participants. However, users adjusted I to match a vibrotactile stimulation intensity instead for exp. 2B. After completing exp. 2B, the participants were asked to repeat the experiment while still using a vibrotactile reference and replacing gel with exercise as the means to change the impedance for exp. 2C. Participants were asked to ascend and descend stairs for 5 min to reduce their measured Rp and then rest for 10 min to increase their measured Rp.

Data showing the results from exps. 2B and 2C for participant TR1 for both Ep versus Rp and Q versus Rp are in Fig. 4 (C and D). The results from all 10 participants are shown in figs. S5 and S6 for exp. 2B and in figs. S7 and S8 for exp. 2C.

As in exp. 2A, the r2 value was computed to determine how well the line of constant sensation intensity for a single trial fit the three participant-derived values for both Ep versus Rp and Q versus Rp. The r2 values from the linear regression are reported in Fig. 4E. Across the 10 participants and 1 condition (n = 10), the average value for Ep versus Rp was 0.954 and for Q versus Rp was 0.900 for exp. 2B, and 0.950 and 0.918, respectively, for exp. 2C. In both experiments, the r2 values for all 10 participants are statistically significantly greater than 0.7 for both Ep versus Rp (Z = 2.75, P < 0.01) and Q versus Rp (Z = 2.75, P < 0.01). The r2 distributions for the two experiments are shown in Fig. 4 (F and G). These results suggest that vibrotactile stimulation can perform well as a reference for this experiment, which we discuss further in note S5. The results provide evidence that the controller will maintain constant sensation intensity even after changes in impedance due to exercise.

Exp. 3A: The controller can be applied when electrodes peel back

When performing activities of daily living with electrotactile stimulation, it is possible for electrodes to peel back or diminish contact over time, causing changes in perceived sensation intensity. To test the effectiveness of our controller in this situation, we asked two participants with right below-elbow amputations (participant TR1: male, age 39; participant TR2: female, age 49) to peel back and reapply one of the two electrodes during stimulation. The means and SDs of I, T, Rp, mE, and mQ over five trials with and without the controller in use are shown in Fig. 5. Real-time changes in these variables during peeling with and without the controller are shown in movie S1.

Fig. 5 Exp. 3A real-time results from two participants (TR1 and TR2) with below-elbow amputations peeling and reapplying electrodes during stimulation.

Upon stimulation with and without the controller, the participants were asked to peel back and reapply the electrodes within 10 s by 25, 50, and 75%, pausing for 5 s in between. (A and B) Rp increases as the electrodes were peeled back. (C and D) I and (E and F) T remain constant when the controller is not used but are modulated in response to changes in Rp when the controller is used. In both cases, (G and H) mE remains constant because it does not depend on Rp. In (I) and (J), mQ varies greatly when the controller is not use. When the controller is in use, I and T are modulated to keep mQ constant. The participants reported no change in sensation intensity when using the controller, but reported discomfort when peeling back the electrodes without the controller. The means and SDs for both participants over five trials are shown.

Rp values (Fig. 5, A and B) varied in response to the electrode being peeled back and reapplied. When the controller was not in use, I (Fig. 5, C and D) and T (Fig. 5, E and F) were held constant and resulted in a constant value of mE (Fig. 5, G and H), which is equal to I2T. However, because the controller was not in use, mQ (Fig. 5, I and J) varied, because it depends on Rp, which was changing throughout each trial. Like the example in Fig. 1C, the participants reported increasing discomfort each time they peeled back the electrode by an increasing amount.

When the controller was in use, I and T were modulated to keep mE and mQ constant. Because changes in I result in changes in Rp, fluctuations in Rp can be larger than when the controller is not in use (Fig. 5A). Differences in Rp between the participants could be due to varying skin conditions (e.g., hydration). Again, mE was always constant because its value does not depend on Rp. However, because mQ depends on Rp, there are small fluctuations in mQ that are modulated by the controller to hold it constant. By keeping mE and mQ constant, Ep and Q are driven to their lines of constant sensation intensity. The participants reported no discomfort throughout all five trials. In fact, the participants reported no changes in sensation intensity any time they peeled back the electrode by any amount. This is in sharp contrast to when the controller was not in use, where the participants reported discomfort proportional to the amount the electrode was peeled back.

Exp. 3B: The controller can be applied to a below-elbow prosthesis

We asked participants TR1 and TR2 to wear a prosthesis that gave electrotactile feedback on fingertip contact (19). While wearing the prosthesis, the participants were asked to perform three activities of daily living for 5 min each: stair ascent and descent, hammering nails into wood, and exercising on an elliptical trainer.

Figure 6 shows the measured Rp and the computed slopes of the lines of constant sensation intensity (mE and mQ) when the prosthesis came into contact with objects during stair ascent and descent (Fig. 6, A to C), hammering (Fig. 6, E to G), and using the elliptical trainer (Fig. 6, I and K). Peak resistance decreased over the 5 min of activity measured without the controller running. In response to changes in Rp, the controller was able to maintain constant values for mE and mQ, whereas the value of mQ deviates greatly from its initial value when the controller is not in use.

Fig. 6 Exp. 3B real-time results from two participants (TR1 and TR2) with below-elbow amputations using a prosthesis with electrotactile touch feedback during three different activities of daily living.

When the prosthesis came into contact with an object, participants would receive electrotactile feedback from electrodes placed on the biceps of the residual limb. (A) The participants ascended and descended stairs for 5 min. (B) Rp was recorded during contact when the controller was not in use. The decreases in Rp are consistent with the decreases that occurred when applying electroconductive gel. (C and D) When the controller was in use, stimulation parameters were modulated to keep mE and mQ constant; however, mQ varied when the controller was not in use. Similar results are shown for hammering a nail (E to H) and exercising on an elliptical trainer (I to L) for 5 min. Breaks in the plots correspond to times when the prosthesis was not in contact with an object. In each activity, the participants reported a lack of sensation by the end of the activity when the controller was not in use but reported the sensation to still be present when the controller was in use.

The purpose of the exercise and electrode peeling tests on participants with below-elbow amputations was to evaluate the effectiveness of the controller in examples of situations that decrease impedance (exercise) and increase impedance (electrode peeling). Movie S2 shows the response of the controller in real time during these examples. For each of the exercises (stairs, hammering, and elliptical), the measured Rp decreased consistently throughout the activity. The participants reported a lack of sensation by the end of each exercise across the electrodes whose stimulation intensity was not modulated by the controller due to the decrease in Rp. However, they reported the sensation to still be present across the electrodes whose stimulation intensity was modulated by the controller.

DISCUSSION

We presented a method of reducing variability in perceived sensation intensity during electrotactile stimulation because of changes in impedance. Our results make electrotactile stimulation more reliable for activities of daily living by reducing the risk of shocks or loss of sensation caused by changes in impedance.

Potential limitations of the controller

Changes in perceived sensation intensity could also be caused by nerve adaptation (20), for which our controller would not compensate since nerve adaptation is not reflected in measurements of impedance. However, by using short, intermittent stimulations on the order of seconds, adaptation can be delayed (21). In fact, in practical implementations of stimulation feedback, it has been shown that providing short stimulations in response to discrete events is more effective than providing continuous feedback (22).

If changes in Rp are large enough, the controller may compute values for I or T that can drop below sensation threshold. For example, sensation might disappear when I < 0.5 mA or when T < 100 μs. At these values, I or T can be fixed at its sensation threshold bound and the other stimulation parameter can be solved for to keep mE = I2T constant, although the sensation may feel diminished.

Controller compensation

Because changes in I induce changes in Rp, the controller must run for multiple iterations (i.e., multiple stimulation pulses) until the values for Q and Rp are back on the line of constant sensation (fig. S9, A and B)—that is, when the actual value of mQ approximates the original desired mQ computed using Eq. 2. Figure S9C shows an example of the change in mQ after gel is added and the subsequent nine iterations (180 ms) of the controller to drive the actual mQ toward the desired mQ until their difference was within a heuristically chosen threshold of 2.5 × 105 μC/kilohm. In preliminary tests, at this threshold, participants did not perceive a difference in sensation intensity between the converged stimulation parameters and the reference. Furthermore, the example in fig. S9 shows an extreme drop in Rp (>20 kilohms) because of electroconductive gel being applied, yet the controller still recovered within nine pulses. Smaller changes in Rp will recover more quickly.

Extensions and future work

The work presented in this paper can be extended in several ways. We used positive monophasic square pulses to study the relationship between stimulation parameters and sensation intensity. To control intensity for different qualities of sensation, the work would need to be extended to account for different waveforms, such as biphasic pulses (23). We also used adhesive-based gel electrodes to provide electrotactile stimulation because these electrodes are commonly used in previous studies (2426). In practical applications, dry electrode arrays that make contact with the skin could be used (27). Both the adhesive-based and dry electrodes would have issues with contact over time, causing changes in impedance for which our controller could compensate. State-of-the-art electrodes such as epidermally applied thin-film electrodes could help mitigate contact issues and provide more mechanically stable stimulation (28).

Another important opportunity for future work is to study the effectiveness of long-term electrotactile sensory substitution in improving sensorimotor control and embodiment of prostheses, as well as analyze long-term variations in impedance when performing activities of daily living. To enable this study, it would be necessary to develop an electrotactile stimulation system with an adjustable stimulation waveform and impedance monitoring that is also compact enough to fit in a prosthesis socket. Stimulators with these capabilities are under development by Kajimoto et al. (27) and Cornman et al. (29). However, further work is required to make these stimulators small enough to integrate seamlessly inside a socket while achieving compliance voltages greater than 100 V. These same stimulators (when used with our controller) could be integrated with other human-machine interfaces that require haptic feedback—future studies would be needed to verify the effectiveness of electrotactile sensory substitution in these other applications.

MATERIALS AND METHODS

Study design

All experimental protocols and equipment were approved by the Institutional Review Board of the University of Illinois, Urbana-Champaign (#13920). Ten human participants without arm impairment (five males and five females, ages 20 to 30) volunteered for exp. 1 (the modeling experiment) and exp. 2A (electroconductive gel experiment). Nine new human participants without arm impairment (four males and five females, ages 18 to 29) and a human participant with a right proximal below-elbow amputation (participant TR1: male, age 39) volunteered for exps. 2B and 2C (the vibrotactile reference experiments). Experiments 1 and 2 used the method of adjustment to evaluate equivalent perceived sensation intensities. Participant TR1 and an additional human participant with a right proximal below-elbow amputation (participant TR2: female, age 49) volunteered for the proof-of-concept demonstrations in exp. 3 (electrode peeling/placing and the activities of daily living).

Electrotactile stimulation

Monophasic positive square pulses were generated by an NI myDAQ (National Instruments, Austin, TX) data acquisition device for the modeling experiments, electroconductive gel experiments using an electrotactile reference, electroconductive gel experiments using a vibrotactile reference, and exercise experiments using a vibrotactile reference. A Teensy 3.2 microcontroller (PJRC, Sherwood, OR) generated square pulses for the electrode peeling/placing experiments and the activities of daily living experiments. The pulses were fed to a STMISOLA linear isolated stimulator (BIOPAC Systems, Goleta, CA) that provided a constant current stimulation to the participant. The voltage across the electrodes was also recorded by the NI myDAQ at 10 kHz or the Teensy 3.2 microcontroller at 100 Hz. All data were collected and processed using the MATLAB Data Acquisition Toolbox (MathWorks, Natick, MA).

To observe the effect of changing resistance with electrode contact, we applied a constant current positive monophasic square pulse across the skin superficial to the left flexor carpi radialis in a 21-year-old male participant. The participant peeled back and reapplied an electrode within 10 s by roughly 25, 50, and 75%, pausing for 5 s in between. We measured the peak voltage (Vp) across the electrodes of every pulse and show the average voltage response in Fig. 1B. We then obtained the peak resistance (Rp) by dividing Vp by the current amplitude, I. When the electrode was peeled back, the increase in current density was reflected by a sharp increase in Rp, and the participant experienced increased stimulation intensities to the point of discomfort as the electrode contact area decreased (Fig. 1C).

Equations for deriving peak resistance, peak pulse energy, and phase charge

In all experiments, we measured the peak voltage (Vp) across the electrodes after applying a monophasic square pulse with current amplitude I and pulse duration T. We then derived the peak resistance (Rp),Embedded Image(5)the phase charge (Q) for a monophasic square wave,Embedded Image(6)and the peak pulse energy (Ep) for a monophasic square wave,Embedded Image(7)

Exp. 1: Modeling experiments

Participants were asked to participate in two sessions held on different days with four conditions being tested, testing eight conditions total. In the first session (session A), the four conditions tested were (i) weak and (ii) strong stimulation using two large 28 mm–by–20 mm electrodes (Ambu Neuroline 710, Ballerup, Denmark) placed over an area of hairless skin on the proximal left forearm over the flexor carpi radialis muscle, and (iii) weak and (iv) strong stimulation using two small 20 mmm–by–15 mm electrodes (Ambu Neuroline 700, Ballerup Denmark) in the same location. In the second session (session B), the four conditions tested were again (v) weak and (vi) strong stimulation using the larger electrodes on the forearm, and weak stimulation using the larger electrodes on (vii) the skin lateral to the long head of the left biceps brachii muscle and (viii) the right lumbar paraspinal area of the back. In all conditions, the center-to-center distance between the electrodes was 3 cm.

In testing all conditions, the method of adjustment was used. Participants adjusted current amplitudes (I) at different fixed pulse durations (T) to match a particular magnitude of sensation (fig. S1A). T was varied between 200 and 700 μs in increments of 50 μs. Two magnitudes of sensation were used, weak and strong. The weak magnitude of sensation was chosen to be around the participant’s sensation threshold at 200 μs. The strong magnitude of sensation was chosen by increasing the current amplitude from the weak magnitude of sensation until it felt like a strong yet comfortable sensation. Pulses were delivered at a frequency of 50 Hz. For each condition, 11 data points were collected that had the same perceived magnitude of sensation, consisting of the value of I and the peak voltage (Vp) corresponding to each value of T. Vp was computed as the average of the peak voltages over 10 pulses. From this, we derived values for peak resistance (Rp), peak pulse energy (Ep), and phase charge (Q).

Testing each condition started by defining an initial reference sensation. The initial reference sensation was found by adjusting the current of a waveform with T = 200 μs until the sensation intensity for a specified magnitude of sensation (weak or strong) was reached. To ensure that each sensation intensity felt the same across all values of T, we compared each sensation felt above T > 200 μs with the initial reference sensation at T = 200 μs. The reference sensation would be presented to the participant for 2 s, followed by a 2-s period of rest before presenting the new stimulation for 2 s at a higher value of T. The participant would then adjust the current amplitude until the sensation intensity felt the same as the reference sensation. The participants were allowed to repeat the presentation of stimulations as many times as they needed.

Because a range of values of I may result in the same perceived sensation intensity as the reference sensation, participants were asked to increase the current amplitude to the upper bound of this range. The final values of I and Vp were recorded. To validate that the sensation intensity at each value of T felt the same, we set a new reference at the stimulation amplitude determined at T = 700 μs, and all values of I and Vp at shorter values of T were compared again and adjusted to match the new reference sensation intensity.

Because skin properties (e.g., hydration) may vary daily, the first two conditions tested in session A (weak and strong stimulation with larger electrodes on the forearm) were repeated in session B to observe trends despite different skin conditions. The other four conditions that were tested investigated the effect of changing the size of the electrodes as well as the stimulation location on sensation intensity. To minimize the time needed for participant testing, we did not repeat these additional conditions across sessions. For the smaller electrodes, stimulation took place on the forearm at weak and strong magnitudes of sensation. The forearm, biceps, and back locations were chosen because they are commonly used stimulation sites in haptic feedback studies (8, 10, 15, 3033).

Exp. 2A: Electroconductive gel experiments using an electrotactile reference

Participants were asked to participate in one session testing five conditions. The five conditions tested were (i) weak and (ii) strong stimulation using two large 28 mm–by–20 mm electrodes (Ambu Neuroline 710) placed over an area of hairless skin on the proximal left forearm over the flexor carpi radialis muscle, (iii) weak stimulation using two small 20 mm–by–15 mm electrodes (Ambu Neuroline 700) placed in the same location, and weak stimulation using the large electrodes on (iv) the skin lateral to the long head of the left biceps brachii muscle and (v) the right lumbar paraspinal area of the back. The hardware setup was the same as the modeling experiments.

For all conditions, two pairs of electrodes were used—a testing pair, whose impedance was manipulated to test the controller, and a reference pair to provide a constant reference sensation intensity for comparison. For the testing electrodes, electroconductive gel (Electro-Gel, Electro-Cap International, Eaton, OH) was either applied or removed between the electrodes and the skin to change the impedance. The reference electrodes were placed in the same location of the body on the contralateral side (e.g., right forearm corresponding to the location of the testing electrodes on the left forearm). Using the method of adjustment, participants were asked to adjust I on the testing electrodes to match the sensation intensity from the reference electrodes. Pulses were generated at a frequency of 50 Hz for each trial.

Testing each condition started by adjusting I at a pulse with T = 1000 μs sent across the reference electrodes until the sensation intensity reached one of two specified magnitudes of sensation, weak or strong. These two magnitudes of sensation were chosen in the same manner as in exp. 1. Next, the value of I sent across the testing pair of electrodes (again with T = 1000 μs) was adjusted by the participant until the sensation intensity matched that of the reference electrodes. To ensure that each sensation intensity felt the same throughout each trial, we used a process similar to exp. 1.

The values of I, T, and Rp for the initial sensation intensity from the testing electrodes that matched the reference were used to compute Ep and Q, corresponding to label 1 in fig. S9. Using the linear relationships and convergence points from exp. 1, we determined the slopes of the lines of constant sensation intensity for the controller to stay on (mE and mQ) in response to changes in Rp (label 2 in fig. S9). When gel was applied or removed, Rp changed and the controller computed new values of I and T that stay on the lines of constant sensation intensity using Eq. 4. I, T, and Rp are used to compute the converged values of Ep and Q (label 3 in fig. S9). To test how well the participant would match the controller-generated line of constant sensation intensity at the new value of Rp, we fixed T to the controller-computed value and asked the participant to adjust I until the sensation intensity from the testing electrodes matched that of the reference electrodes. We fixed T and adjusted I because, in general, when performing electrotactile stimulation, intensity changes are usually made by adjusting I (1, 10). This was also how sensation intensity was adjusted in exp. 1. The values of Ep and Q from the participant-chosen I and controller-computed T were derived, corresponding to label 4 in fig. S9A.

For each condition, this process was repeated three times. This resulted in seven data points per condition, each consisting of an Ep, Q, and Rp value for (point 1) an initial participant-chosen value of I at T = 1000 μs that matched the reference, (points 2 to 4) the controller-computed stimulation parameters after applying or removing gel three times, and (points 5 to 7) the participant-chosen values of I at the controller-computed values of T after applying or removing gel three times.

Exp. 2B: Electroconductive gel experiments using a vibrotactile reference

The same experiment as exp. 2A was performed by using a vibrotactile reference sensation instead. Two large 28 mm–by–20 mm electrodes (Ambu Neuroline 710) were placed over an area of hairless skin on the participant’s right biceps. Monophasic positive square pulses were sent across these electrodes at a frequency of 50 Hz. An electrotactile reference sensation could not be used because exercise would change the impedance of the reference electrodes as well as the testing electrodes. Instead, a vibrotactile motor (310-103, Precision Microdrives, London, UK) was used to provide a reference sensation intensity over the left biceps, under the assumption that changes in skin impedance have little to no effect on vibrotactile stimulation intensity.

Exp. 2C: Exercise experiments using a vibrotactile reference

Participants performed an experiment similar to exp. 2B, replacing gel with sweat generated from exercise as the means to change the impedance. The participants were asked to ascend and descend a flight of stairs for 5 min to reduce the electrode-skin impedance. At this point, as in exps. 2A and 2B, the user was asked to adjust the value of I at a controller-computed value of T to match the vibrotactile reference sensation intensity on the left biceps. The participant would then rest for 10 min to wait for his impedance to recover to a higher value before again trying to match the reference sensation intensity. Last, the participant was again asked to ascend and descend the flight of stairs for 5 min, after which we recorded the third participant-chosen sensation intensity.

Exp. 3A: Electrode peeling/placing experiments

Participants TR1 and TR2 peeled back and reapplied one of the two electrotactile stimulation electrodes placed on their right biceps during stimulation to manipulate the impedance. Upon stimulation, after 5 s, the participants were asked to peel back and reapply the electrodes within 10 s by 25, 50, and 75%, pausing for 5 s in between. This was repeated for five trials using the controller and five trials without using the controller. The initial stimulation parameters for participant TR1 were I = 0.7 mA and T = 1000 μs. For participant TR2, the initial stimulation parameters were I = 1 mA and T = 1000 μs. The different initial values for I across the participants were due to differences in stimulation intensity comfort levels. For both participants, their respective initial stimulation parameters were the same across all 10 trials. The values of I, T, Rp, mE, and mQ throughout each trial were recorded. The participants were asked to report any changes in sensation throughout each trial.

Exp. 3B: Activities of daily living with a prosthesis

Participants TR1 and TR2 felt a pulse with initial stimulation parameters of I = 1 mA and T = 1000 μs at 50 Hz when contacting an object with the index, middle, or thumb digits of the prosthesis. The electrotactile stimulation system was placed in a backpack that the participants wore throughout the experiment.

The participants ascended and descended stairs, hammered nails into wood, and exercised on an elliptical trainer for 5 min each. For stair ascent and descent, the participants made contact with the handrail on every step to receive sensory feedback. When hammering nails into wood, the participants hammered with their unimpaired left arm and received sensory feedback from the prosthesis when guiding the nail or holding the board in place. When using the elliptical trainer, the participants kept the prosthesis gripping the handle for most of the activity. Two sets of electrodes were placed on the right biceps. The values of I, T, Rp, mE, and mQ throughout each trial were recorded from each pair of electrodes. The controller was not in use on the first pair of electrodes, whereas the controller was used on the second pair of electrodes. To test the constancy of the sensation intensity, only a single sensation intensity was tested.

SUPPLEMENTARY MATERIALS

robotics.sciencemag.org/cgi/content/full/3/17/eaap9770/DC1

Note S1. The relationship between the electrode-skin interface impedance and resistance.

Note S2. The modeling experiment results are validated by previous studies.

Note S3. Only keeping I2T constant does not keep perceived sensation intensity constant.

Note S4. The point of convergence was computed using a grid search and gradient ascent.

Note S5. Use of a vibrotactile reference sensation is validated by consistent results.

Fig. S1. Validation of exp. 1 modeling results.

Fig. S2. Exp. 1 modeling results.

Fig. S3. Exp. 2A controller results (Ep versus Rp).

Fig. S4. Exp. 2A controller results (Q versus Rp).

Fig. S5. Exp. 2B controller results (Ep versus Rp, vibrotactile reference).

Fig. S6. Exp. 2B controller results (Q versus Rp, vibrotactile reference).

Fig. S7. Exp. 2C controller results (Ep versus Rp, stair ascent/descent).

Fig. S8. Exp. 2C controller results (Q versus Rp, stair ascent/descent).

Fig. S9. Methods for exp. 2 (controller experiments).

Movie S1. Exp. 3A response of controller in real time during electrode peeling/placing.

Movie S2. Exp. 3B response of controller in real time during exercise.

REFERENCES AND NOTES

Acknowledgments: We thank S. Goldfinger and J. Cornman for help in data collection, K. Y. Choi for assistance with the prosthesis and photographs, and D. Simons for useful discussion. Funding: This work was partially supported by NIH F30HD084201 to A.A. and NSF 0955088 and IIS-1320519 to T.B. Author contributions: All authors participated in the design of this study. A.A., J.S., and B.B. collected data from the participants and performed data analysis and interpretation. A.A., J.S., and T.B. prepared the manuscript. T.B. supervised the research. Competing interests: The authors declare that they have no competing financial interests. Data and materials availability: All data needed to evaluate the conclusions presented are in the paper. Data files may be requested from the authors.
View Abstract

Stay Connected to Science Robotics

Navigate This Article