Research ArticleMEDICAL ROBOTS

Instrument flight to the inner ear

+ See all authors and affiliations

Science Robotics  15 Mar 2017:
Vol. 2, Issue 4, eaal4916
DOI: 10.1126/scirobotics.aal4916


Surgical robot systems can work beyond the limits of human perception, dexterity, and scale, making them inherently suitable for use in microsurgical procedures. However, despite extensive research, image-guided robotics applications for microsurgery have seen limited introduction into clinical care to date. Among others, challenges are geometric scale and haptic resolution at which the surgeon cannot sufficiently control a device outside the range of human faculties. Mechanisms are required to ascertain redundant control on process variables that ensure safety of the device, much like instrument flight in avionics. Cochlear implantation surgery is a microsurgical procedure, in which specific tasks are at submillimetric scale and exceed reliable visuo-tactile feedback. Cochlear implantation is subject to intra- and interoperative variations, leading to potentially inconsistent clinical and audiological outcomes for patients. The concept of robotic cochlear implantation aims to increase consistency of surgical outcomes, such as preservation of residual hearing, and to reduce invasiveness of the procedure. We report successful image-guided, robotic cochlear implantation in human. The robotic treatment model encompasses computer-assisted surgery planning, precision stereotactic image guidance, in situ assessment of tissue properties, and multipolar neuromonitoring, all based on in vitro, in vivo, and pilot data. The model is expandable to integrate additional robotic functionalities such as cochlear access and electrode insertion. Our results demonstrate the feasibility and possibilities of using robotic technology for microsurgery on the lateral skull base. It has the potential for benefit in other microsurgical domains for which there is no task-oriented robotic technology available at present.


Current robotic surgical technology augments the surgeon’s toolkit and permits execution of human-like tasks with superior accuracy, consistency, or speed. Image-guided and model-based robotic systems are commissioned in surgical and interventional domains as diverse as orthopedics (1), neurosurgery (2), radio surgery (3), and interventional oncology (4). Surgical robots are used for telemanipulation of laparoscopic instruments in general (5), urological (6), gynecological (7), and ophthalmic surgery (8). Their application is often with compromised visual perception and diminished tactile information and therefore with reduced effectiveness. During the operation of robotic technology in surgery, a high-level control loop needs to be closed at all times via the human observer using direct visual, microscopic, or endoscopic inspection. Hence, performance of robot operation is limited to human sensing, processing, and execution capabilities (9), and, as a result, the “superhuman” sensing and actuation features inherent in robotic devices remain largely unharnessed in today’s operating rooms (ORs) (10). Next-generation robotic devices will have to execute procedures reliably at geometric scales, temporal resolutions, and safety levels beyond those possible for a human operator alone. If established, surgical robot technology that is capable of operation beyond the limitations of human sensing and tactile skills can contribute to disruptive surgical procedures addressing as yet unmet clinical needs.

Cochlear implantation is an otologic microsurgery procedure in which a silicone wire with platinum electrodes is inserted into the cochlea. The electrode array is part of a cochlear implant (CI), providing hearing sensation to severe to profoundly deaf patients, and some 65,000 CIs are implanted globally per year. Conventionally, surgeons are required to work at the limits of their visuo-tactile feedback and dexterity when accessing the middle ear through a 30-mm opening to place the electrode array (0.3 to 1 mm in diameter) within the cochlea of the inner ear while avoiding unnecessary disturbances and pressure transients at the same time. Although surgical techniques were developed to preserve residual hearing during implantation (“soft surgery”), about 30 to 55% of patients suffer significant hearing loss in the implanted ear (11). This can be ascribed at least in part to variations in surgeon-operator experience, practice, and method. It is our view that robotic technology has the potential to overcome these human operator limitations and allow reproducible, minimally invasive cochlear access and a controlled electrode insertion process. Thus, robotic cochlear implantation (RCI) could lead to deliberate and accurate placement of the electrode in the inner ear for higher consistency of residual hearing preservation and improved audiological outcomes. The increased consistency of a robotic procedure potentially widens CI patient eligibility in the future.

Research on RCI has so far focused on the individual elements of the procedure, such as image-based strategy planning (1214), guided keyhole trajectory drilling using surgical templates (15), industrial robotic manipulators (1618), and skull-mounted passive kinematic structures (1921). Options for the reproducible creation of cochlear access using robotic force feedback control (9) and the design of robotic electrode insertion systems (22, 23) have been addressed.

Zhang et al. (24) demonstrated the feasibility of deployment of steerable electrode arrays using robotic technology. In vivo sensing and subsequent adaptation of a surgical plan for inserting perimodiolar electrode arrays was demonstrated (25) together with the capability of failure prediction during the insertion process (26). Clark et al. (27) demonstrated efficacy of applying magnetically induced torque to the electrode array as a clinical strategy toward improved cochlear implant surgery. To date, Labadie et al. (28) have successfully combined several of the necessary elements that underpin an RCI technology suitable for clinical application and demonstrated the feasibility of geometrically constrained template drilling of a keyhole access in patients. Brett et al. (9) have put forward a robotic cochleostomy drill system, used in three patients, another necessary element in RCI.

Our work has focused on the development of a precise and safe approach for robotic middle ear access (RMA). In a stepwise process, we addressed all stages of middle ear access for RCI: a precise, stereotactically and image-controlled drill process using a task-specific robotic technology (29); secondary positional estimates using correlations of bone density and drill force (30); and task-specific neuromonitoring to detect whether the robotic drilling process passes at a sufficient distance from the facial nerve (31). Furthermore, specific parameters for drill forward thrust, pecking, and rotational speed have been experimentally validated to ensure a heat-minimized drill process. These stages are designed to permit middle ear access that intends to create reproducibly a keyhole drill trajectory (diameter from 1.8 to 2.5 mm) from the skull surface to a predefined target on the cochlea (Fig. 1B). Last, to enable an end-to-end RCI procedure, strategies for optimal electrode selection and sufficient placement of the cochlear implant electrode through the robotic keyhole have been developed and validated (32).

Fig. 1 Robotic cochlea implantation RCI.

(A) Elements of RCI: (i) computer-based patient-specific intervention planning, (ii) RMA, (iii) RIA, and (iii) robotic electrode array insertion. (B) Scale of RCI: A 1.8-mm trajectory to be planned and drilled starting from behind the ear (i), through the mastoid bone (ii) bypassing critical structures at <1-mm proximity and toward the inner ear. Trajectory viewed along its axis (iii) and from the side (iv).

Here, we describe how our robotic treatment model is applied to our technology and then used to carry out our first RCI successfully in a patient—consisting of computer-assisted planning, subsequent robotic access to the middle ear followed by manual inner ear access, and electrode insertion through the access created. Overall system setup, the clinical workflow, the results of the first intervention, and the operation of the safety mechanisms are set out. The case presented also includes analysis to show preclinical feasibility and functional outcome.


A general treatment model for RMA

We have developed and implemented a robotic treatment model that consists of several active and passive interlocking elements to minimize the risk of structural and heat damage to the relevant anatomy. Passive mode safety is contributed by precision image guidance and by heat-minimized robotic drilling. Active mode safety includes (i) a secondary position measurement based on correlation of bone density with applicable drilling force, (ii) neuromonitoring, and (iii) intraoperative imaging (Fig. 2 and Table 1). The geometric scale of the human ear and the precision required to operate in the proximity of critical anatomy necessitate the use of an image-guided surgical robot system. However, image guidance models are prone to errors and inaccuracies (imaging, image annotation, patient-to-image registration, instrument calibration, and tracking), and effective accuracy cannot be reliably verified using live cues from the robotic operating field as seen by the surgeon operator. Even more critical at this geometric scale, a surgeon operator is not equipped to monitor and judge the correct alignment of the robot’s drill axis according to the plan, solely using visual inspection. Safety mechanisms that allow cross-comparison of any safety relevant information and ultimately warrant safe robot performance must be in place. Unlike robotic guiding devices (19) or surgical templates (15), only robotic devices can collect and map sensor data spatially and temporally to drive a dynamically responsive safety system.

Fig. 2 Visual representation of the treatment model for RMA.

Procedural elements and risk mitigation activities of an RCI plan.

Table 1 RMA model.

The model’s elements consist of robotic activities (R), risk mitigation actions (A), and manual activities (M).

View this table:

The general model implemented here is the foundation for a patient-specific, image-based, precision drill plan and consists of the geometric definition of the trajectory within the mastoid, parameter sets for the various drilling phases, and risk mitigation actions such as intraoperative cone-beam computed tomography (CBCT) imaging, neuromonitoring, and bone density measurements, as well as relevant manual activities such as electrode insertion. The model is extendable to future requirements [i.e., additional robotic tasks such as robotic inner ear access (RIA) and robotic electrode insertion], is adaptable for other applications, and is individually parameterized for each patient using a specific software-based planning system, as described below.

Clinical investigation

With approval from the local institutional review board (Ethics Committee of the Canton of Bern, Switzerland, KEK-BE 156/13) and national medical device regulatory body (Swissmedic 2013-MD-0042, EUDAMED CIV-13-12-011779), a clinical trial on the feasibility of the robotic approach commenced with a first patient in summer 2016. A 51-year-old female patient with bilateral deafness was elected for unilateral cochlear implantation in the right ear. Before study inclusion, a minimum distance of 0.4 mm between the planned drill tunnel, the facial nerve, and the chorda tympani was confirmed using CT imagery acquired as part of the standard, preoperative CI clinical assessment protocol. The RCI plan included a direct tunnel drilling to the middle ear, manual access to the cochlea via an extended round window approach, and subsequent electrode placement. Microscopic visualization, electrode impedance, electrically evoked compound action potential measurements, and postoperative CT images were used to confirm effective electrode array placement. Postoperative CT imaging was used to verify geometric accuracy of the drilled tunnel. Pre- and postoperative levels of facial nerve activity and taste sensation were measured and compared to ascertain the effectiveness of the safety mechanisms used.

The robotic procedure

After implanting four surgical fiducial screws for registration (2.2 mm diameter × 5 mm length, M-5243.05, Medartis) into the mastoid, the patient’s lateral skull base was CT-imaged (Siemens SOMATOM Definition Edge). A robotic treatment plan was created using a validated, custom software tool (13). Upon automatic detection of the fiducials and semiautomatic annotation of critical anatomical structures, a tunnel trajectory of 23 mm with 0.5-mm clearance to critical anatomical structures was defined. The proximal end of the trajectory was defined as the center of the cochlea’s round window membrane. Risk mitigation actions (Table 1) as part of the robotic procedure were defined to be a secondary trajectory pose measurement using a correlation of bone density with drill force (A1); intraoperative imaging at a level of 3 mm before the facial nerve (A2); and, during drilling past the facial nerve, five attempts of facial nerve stimulation and subsequent response measurements were carried out in spatial increments of 0.5 mm (A3). General preoperative patient preparation involved placement of the patient’s head in a dedicated headrest specifically designed to provide stable, noninvasive head fixation during drilling (Fig. 3A).

Fig. 3 Surgical preparation, robotic drilling, and implant insertion.

(A) Noninvasive, steady placement of the patient’s head on pressure pads attached to a carbon fiber support structure. Electrodes for neuromonitoring of the facial nerve are attached to facial muscles. (B) The robotic drill accesses the situs through a 20-mm incision. (C) Using an insertion tube, the CI electrode is inserted through the 1.8-mm keyhole into the cochlea.

Facial nerve monitoring (FNM) electrodes were attached to the facial muscles, and ground and stimulation return current electrodes were attached to the upper chest (under the neck circumference, see Fig. 3A) to allow neuromonitoring of the facial nerve during the procedure. Hardware preparation included attachment of the robot system to the OR table, connection to the control system, and sterile draping. Initially, the patient’s lateral skull was registered using the bone-anchored fiducial screws to allow for a transfer of the drill trajectory’s entry point and the patient optical reference skull attachment point to the situs. The patient optical reference was mounted using a percutaneous 8-mm bone screw. A retractor was placed to keep the area around the trajectory entry point accessible for the robot drill. A secondary accuracy-optimized registration was performed immediately before drilling (fiducial registration error, 0.09 mm). The robotic drilling was carried out according to the plan created in the procedural work-up and by strictly avoiding any physical movement of the patient (Fig. 3B). Table 1 gives a summary and parameters of the drill process.

Implant management and electrode array insertion

On completion of the robotic drilling, the fiducial screws, the optical reference, and the robotic arm were removed from the situs. A tympanomeatal flap was created for additional visual and instrument access to the middle ear cavity (32, 33). A correct alignment of the tunnel with the round window was visually confirmed by endoscopic inspection. The tunnel was extensively rinsed to be cleaned from bone dust and blood. The round window membrane was manually opened through the transcanal access under careful irrigation and suction. A removable two-piece guide tube was placed in the tunnel to prevent contact of the electrode array with the drilled tunnel and to limit deviations or kinking during insertion (Fig. 3C). The electrode array (Flex24, MED-EL) was inserted through the guide tube and into the cochlea under microscopic visualization. Full insertion of the electrode array was visually confirmed. The array lead was fixed at the round window niche, the excess lead was embedded within the mastoid cortex, and wounds were closed.

Patient outcomes

Intraoperative implant telemetry demonstrated effective intracochlear placement of the electrode array: Functional impedances were measured at all electrode contacts (average, 4.1 kilohms; range, 2.6 to 6.6 kilohms), and the electrically evoked compound action potentials recorded indicated effective auditory nerve stimulation. Postoperative CT imaging confirmed safe distances of the robotically drilled tunnel to the facial nerve and the chorda tympani to be about 1.0 and 0.3 mm, respectively. An angular insertion depth of 360° of the electrode array was achieved (Fig. 4B). CT evaluation indicated 11 of 12 intracochlear electrodes within the cochlea. Postoperative neuromonitoring showed unchanged facial nerve function, confirming that the integrity of the facial nerve was maintained during the robotic procedure. The patient was discharged from hospital 1 day postoperatively. Implant fitting of the patient is currently ongoing, and audiological outcomes are reported for the 6 months post-activation session. As indicated by the postoperative CT, 11 electrodes were activated for stimulation. Evaluation of electrode impedance was within functional values, and the aided sound field hearing threshold was 22-dB hearing level, averaged for the frequencies between 0.25 and 4 kHz. The patient had 50% word recognition in quiet tested at 65-dB sound pressure level with the Freiburg monosyllabic test.

Fig. 4 Confirmation of safe passage and postoperative situation.

(A) Intraoperative CBCT imaging allows delineation of the trajectory and the facial nerve. A neuroradiologist manually confirms sufficient distance between the trajectory and the facial nerve. (B) Inserted electrode array, excess lead placement, and final implant position as measured in postoperative CT imaging.


The past decade has resulted in an extensive body of research aiming to innovate cochlear implantation. However, a true paradigm shift through comprehensive and consistent application of modern surgical technology that requires incorporation of robotics, image guidance, and sensor technology has not occurred. In essence, any technological innovation that seeks to supersede conventional cochlear implantation, such as RCI, must incorporate a set of fundamental elements that can deliver an improved CI procedure. These elements are (i) computer-based patient-specific intervention planning, (ii) RMA, (iii) RIA, and (iv) robotic electrode array insertion (Fig. 1A). Some or all of the above elements have to be combined methodically to result in predictable and reproducible surgical and audiological outcomes of nonmanual CI implantation technologies.

In this work, we have demonstrated a robotic implantation workflow model that combines parametric descriptions of all physical actions and activities during a robotic treatment with patient data to specify elements that ensure a reproducible, consistent, and safe procedure. This RCI workflow model may serve to facilitate successful development of radically new RCI iterations and confirm suitability of existing technological implementations.

We report our RCI procedure using a robotic, image-guided, and sensor-controlled drill process. Our intent is to enable innovations to cochlear access and implantation methods that improve surgical outcomes, maintain safety, and widen treatment inclusion. This can be achieved through the use of process control and multilayer safety management that can surpass the manually operating surgeon’s limitations in sensing, dexterity, and execution. Robotic technology delivers a submillimetrically guided, heat-minimized drill process, complemented by bone density to drill force mapping, intraoperative imaging, and facial nerve monitoring.

The overall objective of this work is to demonstrate the viability of a model-based approach to RCI through the integration of the latest findings in image-based planning, robotic image guidance, and auxiliary sensor technology to deliver a task-oriented surgical application that works beyond human proprioceptive skills. Our data confirm our hypothesis that a clinical application based on a systematic, comprehensive, and universal robotic treatment model would be able to resolve challenges of precision, accuracy, and safety. The resultant RCI platform solution, which is driven by a patient-specific planning and intervention algorithm, enabled the first procedure of RCI in a patient. We show here that a holistic RCI approach and a predetermined universal workflow model provide for a personalized treatment outcome with a high degree of standardization and reproducibility while navigating patient-specific anatomy with robotic precision at the microsurgical scale at the same time.

A central safety aspect of our RCI process is the dual approach to a position measurement system to ensure conformity with the image-based navigation model: first, correlation of bone density with drill forces to self-localize the robot created trajectory within the situs and, second, optical tracking, both implemented as mutually redundant features. In addition, automatic multipolar neuromonitoring tracks the structural integrity of the facial nerve during drilling independent of errors in the other safety mechanisms present. The clinical trial design underlying the procedure documented here also includes intraoperative CBCT imaging to verify the correctness of the trajectory. Nonetheless, as clinical data are generated to support the resilience of the existing safety modalities, the RCI procedure intended for clinical use will not require intraoperative imaging, avoiding the radiation burden and cost of this additional procedure.

In our hands, a stereotactically guided robotic solution is potentially more resilient for RMA compared with template-guided and manually drilled approaches (28) with respect to effective geometric accuracy (34) and active safety mechanisms [bone density/drill force correlation and electromyography (EMG)]. Such a tightly controlled drill process delivers the drill and insertion vector path assigned by the personalized treatment plan.

This study did not include an assessment of RIA, which has successfully been demonstrated in patients by Brett et al. (9). The single-function robotic technology described by Brett et al. is not extendable to perform RMA, because this requires fundamentally different, image-guided control mechanisms. By contrast, in (35), we have demonstrated our system’s capability to carry out RIA after RMA using the integrated force-torque sensing technology. Notwithstanding this and to comply with our approved trial protocol and experimental validation, we have omitted this functionality in the ongoing clinical trial. Similarly, we have not addressed the third stage of RCI (i.e., robotic electrode insertion) in this study, because we consider this future work.

Atraumatic insertion of the electrode array into the cochlea is a critical part of the minimal invasive concept. We aim to reduce insertion-related trauma by (i) providing an optimized alignment of the insertion trajectory using the preoperative planning software, (ii) deploying the guide tube to constrict the array to the planned trajectory and to minimize kinking or lateral movement during insertion, (iii) using soft surgery haptics for speed- and force-controlled electrode array insertion through the guide tube into the cochlea, and (iv) supervising the insertion through an auxiliary access. We are encouraged by our findings from this study that an end-to-end, nonmanual approach can be developed beyond cochlear access and are actively investigating a bias-minimized, nonmanual insertion methodology. Atraumatic electrode insertion is an area of new research for us and a vital component of any robotic CI solution in the future, including our own.

Considering the effective drilling time of about <10 min, the brevity of which is offset by a number of longer, additional but compulsory steps such as fiducial screw placement, computer-assisted planning, and CBCT imaging before passing the facial nerve, the overall procedure time of RCI has to be addressed. In the same way we used a robotic treatment model to achieve safety, we will apply a model-based approach to progress our goal of an equal or shorter procedure time for RCI compared with conventional CI. For example, subject to final validation of the efficacy of all other safety mechanisms, we consider CBCT imaging and associated radiation exposure (0.3 mSv) to be avoidable in the future. Because of the constrained size of the drill tunnel, the RCI approach is currently limited to lateral wall electrode arrays with diameters of less than 1.5 mm. Clinical application of perimodiolar electrode arrays with the robotic approach would require array leads with smaller dimensions and insertion tools for stylet retraction mechanisms that can be deployed within the dimensions of the current RCI tunnel. We believe that the reporting of our RCI approach will encourage comprehensive implant technology research and development that will produce such CI feature evolution.

Future work will address robotic solutions for additional phases in procedural workflow. The control afforded by robotic drilling of the cochlear access aims to minimize disturbances and trauma within the cochlea (9). During robotic drilling, the applicable force-torque data are monitored for specific transients that correlate with a breakthrough to the cochlear wall, which, in turn, forces the robot’s control system to stop the drill process much faster than a human could (9). Current work investigates the feasibility of our robot’s sensor systems to facilitate the integration of automatic cochlear wall drilling. Last, actuated and controlled insertion of the electrode array (22, 23) is currently under investigation and may provide for consistent insertion speed and reduced insertion forces to reduce trauma to intracochlear structures.

The workflow of the RCI robotic cochlear procedure still requires extensive teamwork at all stages of the procedure. Pre-, intra-, and postoperative stages are staffed in a multidisciplinary team structure: Bioengineering, audiology, neuroradiology, surgery, and allied health team members are present and active during the entire intervention, before and after. For clinical assessment and research purposes, this is mandatory and appropriate. In routine clinical care, this is prohibitive and makes any procedure unviable. The work required to enable clinical adoption will have to address this and suitably iterate RCI usability, which is a high-priority objective for us.

Beyond the scope of cochlear implantation, we consider our technology to be useful for local drug therapy in the inner ear, where the challenge of creating a reproducible, yet minimally invasive, access to structures within the cochlea persists. In other applications, precision drilling in the lateral skull base could permit surgical approaches via several instruments and camera keyholes (36), all joining near a specific target, effectively enabling minimally invasive neurosurgical approaches in the skull base.


Robotic surgical system

We regard that the development of task-specific surgical robotic manipulators optimized for a specific clinical use case has clear advantages over adapting ubiquitously available industrial robots (i.e., KUKA LWR). These encompass task specificity by design, clinical integration functionality, early-stage safety considerations, and flexibility to adapt the device and procedure to fit the specific requirements of clinical implementation. We aimed at developing a robotic solution that is safe for a human operator to carry (<10 kg), directly attachable to an OR table and without the need for an additional cart. A 5–degree of freedom serial manipulator and an associated control hardware were specifically developed for application during minimally invasive microsurgical procedures on the lateral skull base. A serial kinematic design was chosen over a parallel (head-mounted) approach because it can be removed from and replaced in the surgical situs quickly at any time and without requiring re-registration or re-fixation. An overall manipulator arm length of 700 mm (base to tool tip) was appointed to provide optimal working volumes of about 100 mm × 100 mm × 100 mm per configuration (left/right) and toward the side of the head being operated on at any given time.

Each of the manipulator joints consists of a motor/encoder/brake combination (axes 1 to 3, EC-max 30; axes 4 and 5, EC-max 22; Maxon) and integrated gearboxes (Harmonic Drive; axes 1 to 3 CPL-17; axes 4 to 5 CPL-14) connected through a timing belt, providing effective transmission ratios of 1:560 and 1:400 in axes 1 to 3 and axes 4 and 5, respectively. Inductive end switches are integrated in each joint to provide for a defined “zero position” per axis. Digital motor control units (EPOS2 Module 36/2, Maxon) are directly integrated into the robot assembly (robot base: unit 1, link 3: units 2 to 5). The setup provides for an effective Cartesian resolution of about 0.001 mm. A six-axis force-torque sensor (Mini-40, ATI) and a semiautomatic tool quick release are fitted at the wrist. The robot houses a battery-powered microcontroller (DSPIC33F, Microchip), as wells as an integrated voltage converter (24/5 V) located in the base, which is used to store all encoder values in nonvolatile RAM during the system’s intentional or emergency shutdown; these are then read back during repowering, eliminating the need for an initial calibration move at start-up in the absence of suitably sized absolute position encoders. Last, several temperature sensors and accelerometers are placed in each of the links, monitoring temperature and motion status of the robot.

The manipulator structure itself was designed as a semimonocoque structure (37) to effectively carry all tensile and compressive forces within its outer housing and without further internal frames that potentially contribute to additional weight. Subsequently, all links and joint components were computer numerical control–milled from aluminum-copper alloy (Certal, EN AW-7022) and high-strength aluminum (EN AW-7075), providing a superior manipulator stiffness of 0.01 mm N−1 at a resulting minimum weight of 1.6 and 5.5 kg for the monocoque structure and the complete robot system, respectively.

Communication between the robot and the higher-level control system is enabled via two independent controller area network (CAN) buses. One bus links all digital motion controllers to the robot’s real-time control system. The second bus is used to query encoder values at start-up as well as robot status and auxiliary sensor data. Electrical connection between the robot and navigation system is achieved via a combined power supply and bus cable (14 mm in diameter), effectively reducing clutter in the sterile area. Motor control is performed using a multilayered control system: Low level current control is provided by the built-in motion controllers, with higher-level position, velocity, and interpolated positioning control, as well as path planning, generated by proprietary, developed software in combination with the EPOS motion controllers. These higher-level control and path planning loops run and communicate with the motion controllers at 100 Hz, maximizing the available bandwidth of the used CAN bus. The developed higher-level control software operates in a modified real-time (RT-Preempt) Linux environment, on a 2.7-GHz i7-based mini-ITX single-board computer (MI980VF, iBase). In addition to kinematic, control, and path planning functions, the software interfaces with the higher-level navigation system, the integrated force-torque sensor, and other auxiliary sensors, as well as the optical tracking system through a transmission control protocol/internet protocol connection.

Auxiliary system components

A separate display and interface cart integrates computational hardware, power supply, signal conditioning/data acquisition hardware, medical-grade safety relay, and a touch screen user interface. Computational hardware consists of a Core i7 computer (quad-core, 2.3 GHz, 8 GB, Windows 10) and a proprietary high-level control software, guiding the user through all relevant steps of the workflow.

An optical tracking system (CamBar B1, Axios3D) is attached at the robot base (Fig. 5). The camera is capable of tracking custom light-emitting diode (LED)–illuminated tracking references across its entire workspace with an accuracy of 0.05 ± 0.025 mm (38) at a tracking frequency of 60 Hz. Incoming tracking data (at 60 Hz) is Kalman-filtered to reduce latency and synchronized with the frequency of the robot’s real-time control loop (100 Hz). Each of the tracking references contains five illuminated fiducials that are indirectly backlit each by four infrared LEDs (λ = 850 nm) to minimize orientation-dependent variation of lighting intensity. Both trackers are synchronously strobed at the camera’s exposure cycles with a duty cycle of 150 μs to avoid extensive heat buildup. The tracker fiducial geometry is optimized on the basis of the available workspace and specific task, maximizing interfiducial distances while minimizing tracker obstructions. One tracker is attached to the patient’s skull via an adjustable bone anchor, whereas the other rigidly mounted to the surgical drill hand piece.

Fig. 5 System overview.

Highlighting all relevant robotic, stereotactic, and surgical instrument components.

Because of a lack of appropriate, commercially available solutions, a proprietary drill system was designed and produced. Its stiffness (0.01 mm N−1) and runout (0.018 mm) values contribute significantly to meeting the accuracy requirements of the overall system. The brushless motor of the hand piece is also controlled via the robot control software through an EPOS motor control unit. The drill system is autoclaved for sterilization and is mechanically connected to the robot via the tool quick release. A custom drill bit with 1.8-mm-diameter (proximal, l = 10 mm) and 2.5-mm-diameter (distal, l = 27 mm) sections was designed and produced in medical-grade tungsten carbide for maximum tool stiffness. The overall geometric drilling accuracy and precision, excluding image-related errors, has been verified to be 0.1 ± 0.05 mm using a laser scanner–based measurement stage, whereas complete end-to-end system evaluation has revealed an accuracy of 0.15 ± 0.08 mm on n = 8 human cadaver specimens.

A clamp-like carbon fiber headrest with three inflatable pressure pads was designed to reduce the relative motion between robot and patient during drilling (stiffness, <0.1 mm N−1) without the need for invasive percutaneous head fixation (i.e., Mayfield clamps). The head rest (Fig. 3A) provides for noninvasive patient fixation while maintaining compatibility with intraoperative fluoroscopy–based imaging systems. Together with the robot itself, the head rest is mounted to the head side of the OR table via a rail mechanism.

Surgery planning

A software tool that assists in the identification and definition of the above-mentioned elements is derived from the preclinical image data and thereby produces a personalized patient-specific treatment plan that includes the robotic drill plan and electrode insertion parameters (Fig. 6) (13). Three-dimensional (3D) models of the relevant anatomy are created using semiautomated identification and annotation of the image data. Manual adjustments are performed by the user to ensure oversegmentation of structure boundaries, for example, facial nerve. Software functionality is provided for the geometric definition of a drilling trajectory, as well as the definition of the various drilling phases. The trajectory is defined from the mastoid surface to the cochlea, bypassing all surrounding critical anatomical structures, including the facial nerve, chorda tympani, ossicles, and the posterior portion of the external auditory canal.

Fig. 6 Planning the intervention.

The planning software tool allows for general image segmentation (i.e., identification of the fiducial screws, in green), segmentation of anatomy (i.e., facial nerve, in yellow), and parametrization of the general treatment plan.

During manual optimization of the trajectory orientation, the software automatically calculates geometric distances from the trajectory to the surrounding modeled anatomy as well as angles between the trajectory and to the course of the cochlear basal turn to allow optimal implant electrode insertion. In a linkage step, the software detects fiducial screws that are placed in the patient’s lateral skull base and are later used to intraoperatively register the plan to the patient. To accommodate accuracy requirements, we automated and verified detection of fiducial screws using fiducial volume accuracy and model-to-image matching variations (39). Comprehensive, general image quality tests are implemented to detect unacceptable image artifacts or insufficient image contrast levels. To conclude the planning procedure, we selected the most suitable implant electrode based on the patient’s cochlea parameters.

Instrument flight–like robot operation

The typical human facial recess has a width of about 2.54 ± 0.5 mm (40), with the tunnel diameter required for the insertion of the current generation of electrodes being 1.8 mm. Therefore, the accuracy required to consistently avoid the bounding nerves when drilling the minimally invasive access tunnel is well beyond that achievable using existing navigation systems. Any clinically feasible, minimally invasive robotic access to the cochlea is dependent on an image guidance model that has a maximum geometric inaccuracy of less than 0.3 to 0.4 mm to warrant safe operation and sparing of the facial nerve. Here, we set out the technical specifications that serve the purposes of geometric accuracy, redundant monitoring of the tool position independent of the image guidance model, and the electrophysiological stimulation and monitoring protocol. Patient-to-image registration involves physical digitization of topological coordinates of the patient’s mastoid bone via bone-anchored fiducials, followed by algorithmic localization of the corresponding fiducial positions within the 3D image data (i.e., in the image coordinate system). A least squares–based algorithm (41) is used to estimate a rigid transformation relating the physical and the image coordinate systems. Available geometric errors when measuring the fiducial position (fiducial localization error) are as low as 0.046 ± 0.029 mm and 0.153 ± 0.061 mm in the physical and image space, respectively (39).

Precision image guidance and visual servoing

During the drilling process, a visual servoing scheme is used for command of the manipulator position (42). The implemented vision-based controller uses multiple sources of sensor information to reduce latency and increase robustness. A Kalman filter fuses velocity information from the robot’s internal encoders (at 100 Hz) with the camera position data (60 Hz) (43). Active tracking is executed via optical references attached to the patient’s head and the robot’s end effector. This ensures critical compensation of minor patient movements (i.e., as a result of the finite stiffness of the head rest) and any remaining positional inaccuracies of the robot’s internal differential position encoders (up to 0.3 mm in a 200 mm × 200 mm × 200 mm workspace).

We have demonstrated a task-specific end-to-end geometric accuracy of our drilling process to be 0.15 ± 0.08 mm in human cadaveric specimen (29). The accuracy was measured as the geometric displacement of a trajectory preoperatively defined in CT image data versus the drilled trajectory as identified in a postoperative CT scan. Rigorous minimization of all errors contributing to inaccuracy in the guidance model, such as optical tracking instrument calibration, and fiducial localization both in the image and on the patient (39) were accomplished, together with a radical minimization of backlash and maximization of rigidity in all structural elements of the kinematic structure as well as the surgical drill system. Assuming a normal Gaussian distribution, the maximum geometric error of the robotic guidance system can be as low as 0.4 mm in 99.7% of all drilling attempts (3σ). This end-to-end geometric accuracy of an image-guided robotic system is the highest reported to our knowledge and is the base requirement to enable all other elements of the safety architecture.

Preliminary accuracy testing revealed deficiencies in standard surgical drilling hardware. Stainless steel drill bits deformed under lateral loads, with each traversal of an air cell in the mastoid bone. Factors such as tool deflection as a result of diameter and backlash in the drilling unit itself further reduce the effective drilling accuracy. Thus, a backlash-free drill spindle was developed and integrated in the robot’s end effector. In addition, high-rigidity tungsten carbide drill bits with one cutting edge were used to reduce tool bending (Louis Belet SA, Switzerland).

Heat minimization during the drilling process

The second element of ensuring safety through system design is the implementation of a heat-minimized drilling process. Drilling as close as 0.5 mm from the facial nerve likely creates exceptionally damaging amounts of heat energy. Labadie et al. (28) have reported the incidence of a heat-related facial palsy in a patient undergoing template-based cochlear implantation. We have confirmed the possible danger of thermal damage to the facial nerve in an in vivo sheep study. It was found that drilling of the dense bone in the region of the facial nerve can lead to a harmful temperature rise (43). Therefore, we investigated the use of a drill bit specifically designed with an optimized geometry to minimize the amount of heat generated during the drilling process. Further iteration of the drill process included interval drilling at increments of 2 mm (in segments R1 and R3) and 0.5 mm (in segment R2), which was used to limit the accumulation of heat (Table 2). In addition, at each increment, the drill bit’s flute is flushed (1 s, 2 ml) with saline solution. This prevents chip congestion, which potentially contributes to friction-induced heat buildup. Therefore, we demonstrate in the current intervention that the optimized drill bit and drilling process can maintain safe temperature levels while drilling at a 0.5-mm distance from the facial nerve (45). In addition, safe values for rotational and longitudinal drill speed have been established to be 1000 rpm and 0.5 mm s−1, respectively, to ensure the prerequisite temperature control. We assert that these features of the robotic device can maintain optimal compliance of the RCI procedure with the above parameters.

Table 2 Robotic access plan.

R, robotic actions; A, risk mitigation actions.

View this table:

Bone density–to–drilling force correlation

In situ confirmation of the robot’s effective positional accuracy inside the human skull and after image co-registration has not been achieved in a clinical setting to date. Despite its superior general accuracy, image guidance models are not fully reliable to be the exclusive informant for positional accuracy in a robotic surgical procedure. A secondary position measurement mechanism is essential to provide a physical cross-check for the validity of the image guidance model. We have developed a secondary localization method based on the correlation of drilling force and the variable bone density throughout the mastoid region. It allows the redundant prediction of the 6D pose of the drill trajectory independent of standard image guidance errors such as registration. It is assumed that any drill trajectory through the mastoid results in a unique bone density profile along this trajectory. It is further assumed that force/torque data acquired during trajectory drilling optimally correlates with the executed trajectories’ bone density profile. Similarity measures can be used to determine the best match from a given set of candidate trajectories that vary in position and orientation. These candidates are generated in an evenly spaced volume around the originally planned trajectory with variations in entry, target position, and angles (to 2.5°) relative to the original trajectory. The similarity between the force and density profiles acts as a weighting factor, which is then used to calculate the weighted mean position of the tool (Fig. 7). The initial implementation of this process demonstrated an accuracy of 0.29 ± 0.21 mm when using force data only (30). Subsequent improvements to the algorithm, including similarity metric, density extraction technique, and workspace configuration, have led to increases in both accuracy and speed of the algorithm, with a current accuracy of 0.19 ± 0.13 mm at a point 3 mm above the facial nerve observed. The specific layout of the alternative trajectories was initially defined on the basis of the existing accuracy of the robotic system during the development of the algorithm. Subsequent evaluation has revealed that changes to the configuration of the alternative trajectories do not alter the accuracy of the algorithm while allowing large errors [for example, due to registration failures, as described in (33)] to be detected accurately.

Fig. 7 Pose estimation using drill force and bone density.

Computation of the trajectory pose using a correlation of bone density (from CT) and drill force (recorded during drill process).

Robot-integrated neuromonitoring

The development of a suite of procedural safety components that would mitigate against the risk of structural damage to the facial nerve included a task-specific EMG facial nerve monitoring module. By measuring electromyogram signals in the facial muscles induced by electrical stimulation of the facial nerve through a dedicated stimulating probe, the distance of the trajectory to the facial nerve can be estimated to allow the drill path to remain within safe limits. In the worst-case scenario, the procedure can be aborted on the basis of the neuromonitoring data alone, before iatrogenic damage to the facial nerve would be sustained. Independent of all previously described safety mechanisms, this system is therefore to be considered the so-called last line of defense (31). The system is based on a commercial electrical nerve stimulation and monitoring system (both ISIS, inomed) and was fundamentally modified and customized with a multipolar stimulation probe/protocol (Fig. 8) and proprietary software control system. This resulted in a dedicated RCI EMG system with the following task-specific functionalities:

Fig. 8 Neuromonitoring of the facial nerve during robotic drilling.

(A) Optically tracked stimulation probe inserted in the drilled tunnel near the facial nerve before application of an automatic protocol through four channels of the probe. (B) Probe with cathode to anode distances to be di = 2, 4, 7 mm (Anodei), and monopolar stimulation enabled by a far-field needle electrode (superficial to the sternum). (C) After automatic stimulation between 0.2 and 2 mA, EMG responses only appeared at 2 mA and the monopolar configuration, suggesting a safe drilling passage at facial nerve distances above 0.7 mm. (D) Example of electrically elicited EMG signals during drilling, showing amplitude range and polyphasic nature of responses.

(1) Sensitivity monitoring of the nerve [i.e., positive control (46)] established through a pair of surface stimulation electrodes located on the superficial branch of the facial nerve. The stimulus threshold values are expected to be in the range of 20 to 50 mA (monophasic pulses, duration = 250 μs) and are dependent on the electrode-electrolyte and skin contact impedance.

(2) Functional nerve status is continuously monitored via nonelectrically triggered EMG (free-running EMG) responses to detect weak anesthesia conditions and potential nerve irritation due to, for example, excessive mechanical pressure or temperature rise (46).

(3) During drilling proximal to the facial nerve, the robot automatically evacuates from the drill tunnel, followed by manual insertion of the multielectrode stimulation probe. Each of the four electrode configurations in the stimulation probe (B1, B2, B3, and monopolar) is then subject to an automatic stimulus threshold search. Stimulus threshold values above 1 mA likely indicate that the remaining bone thickness between the drill trajectory and the facial nerve is sufficiently protective, whereas values below 0.3 mA may indicate facial nerve dehiscence or absence of nerve bone covering. During electrical stimulation of the nerve, the amplitude of an electromyogram-elicited signal is expected to be above 100 μV and is composed of a complex polyphasic muscle action potential response. The stimulus threshold is the minimum intensity of the stimulating pulses that provided an EMG signal above 100 μV. Provided that suitable electrode-tissue contact (impedance, <20 kohm) are displayed, a reference scale is consulted (Table 3) to estimate the proximity to the facial nerve and give the system a go/no go command by the surgeon. Given a positive margin, the next drilling segment is executed by Go, and the NM protocol is repeated with a final measurement as the facial nerve is cleared.

Table 3 EMG decision table.

Decisions are based on stimulus threshold values above (0) or below (1) 0.35 mA.

View this table:

(4) Manual stimulation parameters for spot measurements permit verification of inconclusive data points generated during the stimulus threshold search automated EMG measurement protocol.

Data acquired in a sheep animal model (31) have confirmed that this approach can reliably detect proximities of the drilled trajectory to the fallopian canal (i.e., the facial nerve) below 0.1 mm. We introduced the concept of discretization of the available drill–to–facial nerve distance range into safe and non-safe zones, where potentially hazardous collisions of the drill bit with the facial nerve can be identified with sufficient sensitivity and specificity (>95%). Histopathology of facial nerve sections from the in vivo study suggested no relevant axonal injury of the facial nerve upon penetration and immediate (i.e., triggered by neuromonitoring) ceasing of robotic operation in the fallopian canal.

Intraoperative CBCT

Although not part of the robotic system, intraoperative CBCT imaging aids in determining the drill trajectory pose in situ and contributes to procedural safety. The clinical trial protocol for the procedure reported here mandates the use of intraoperative CBCT imaging. We have established a protocol with a head scanner (xCAT, Xoran) acquiring images of the patient’s lateral skull base with a titanium rod inserted into the partially drilled access tunnel (action A1 of the robot plan). In the resulting scan, the automatically segmented titanium rod is extrapolated toward the target position (Fig. 4A). In addition, the preoperative plan and images are co-registered (using a normalized mutual information metric) to allow geometric distance calculation from the segmented rod to the preoperatively modeled facial nerve. To allow intraprocedural decision-making in line with the clinical trial protocol, a neuroradiologist confirms a safe distance upon visual inspection of the intraoperative images. Our aim is to forgo intraoperative imaging in the RCI procedure eventually to reduce patient irradiation, workflow complexity, and cost.



Funding: This work was supported by the Swiss National Science Foundation (project NCCR Co-Me), the Swiss Commission for Technology and Innovation (project MIRACI 17618.1), the European Commission (project HEAR-EU, 304857), the Swiss Nano-Tera initiative (project HearRestore), MED-EL GmbH (Innsbruck, Austria), and CAScination AG (Bern, Switzerland) Author contributions: S.W. and M.C. created the study design. B.B., T.W., K.G., N.G., W.W., M.S., A.F., J.A., C.R., D.S., and S.W. developed all hardware and software components, carried out accuracy experiments, analyzed the data, and performed in vivo animal testing. All authors had substantial clinical and technical responsibilities during the clinical trial. All authors reviewed the manuscript. Competing interests: B.B., T.W., and S.W. are inventors on the related patent EP2666248; A.F. and S.W. are inventors on the related patent EP16153033. S.W. is cofounder, shareholder, and advisor to the board and M.M. is chief technology officer and shareholder of CAScination AG (Bern, Switzerland), a company that is developing robotic cochlear implantation technology. M.C. is a member of the scientific advisory board of and M.K. received travel funding from MED-EL GmbH (Innsbruck, Austria), a cochlear implant company. Data and materials availability: There are no material transfer agreements or restrictive patents.
View Abstract

Navigate This Article