Brain–computer interface

Kennedy and Yang Dan

Phillip Kennedy (who later founded Neural Signals in 1987) and colleagues built the first intracortical brain–computer interface by implanting neurotrophic-cone electrodes into monkeys.

Yang Dan and colleagues' recordings of cat vision using a BCI implanted in the lateral geniculate nucleus (top row: original image; bottom row: recording)

In 1999, researchers led by Yang Dan at the University of California, Berkeley decoded neuronal firings to reproduce images seen by cats. The team used an array of electrodes embedded in the thalamus (which integrates all of the brain's sensory input) of sharp-eyed cats. Researchers targeted 177 brain cells in the thalamus lateral geniculate nucleus area, which decodes signals from the retina. The cats were shown eight short movies, and their neuron firings were recorded. Using mathematical filters, the researchers decoded the signals to generate movies of what the cats saw and were able to reconstruct recognizable scenes and moving objects.^[36] Similar results in humans have since been achieved by researchers in Japan (see below).

Nicolelis

Miguel Nicolelis, a professor at Duke University, in Durham, North Carolina, has been a prominent proponent of using multiple electrodes spread over a greater area of the brain to obtain neuronal signals to drive a BCI.

After conducting initial studies in rats during the 1990s, Nicolelis and his colleagues developed BCIs that decoded brain activity in owl monkeys and used the devices to reproduce monkey movements in robotic arms. Monkeys have advanced reaching and grasping abilities and good hand manipulation skills, making them ideal test subjects for this kind of work.

By 2000, the group succeeded in building a BCI that reproduced owl monkey movements while the monkey operated a joystick or reached for food.^[37] The BCI operated in real time and could also control a separate robot remotely over Internet protocol. But the monkeys could not see the arm moving and did not receive any feedback, a so-called open-loop BCI.

Diagram of the BCI developed by Miguel Nicolelis and colleagues for use on rhesus monkeys

Later experiments by Nicolelis using rhesus monkeys succeeded in closing the feedback loop and reproduced monkey reaching and grasping movements in a robot arm. With their deeply cleft and furrowed brains, rhesus monkeys are considered to be better models for human neurophysiology than owl monkeys. The monkeys were trained to reach and grasp objects on a computer screen by manipulating a joystick while corresponding movements by a robot arm were hidden.^[38][39] The monkeys were later shown the robot directly and learned to control it by viewing its movements. The BCI used velocity predictions to control reaching movements and simultaneously predicted handgripping force. In 2011 O'Doherty and colleagues showed a BCI with sensory feedback with rhesus monkeys. The monkey was brain controlling the position of an avatar arm while receiving sensory feedback through direct intracortical stimulation (ICMS) in the arm representation area of the sensory cortex.^[40]

Donoghue, Schwartz and Andersen

BCIs are a core focus of the Carney Institute for Brain Science at Brown University

Other laboratories which have developed BCIs and algorithms that decode neuron signals include the Carney Institute for Brain Science at Brown University and the labs of Andrew Schwartz at the University of Pittsburgh and Richard Andersen at Caltech. These researchers have been able to produce working BCIs, even using recorded signals from far fewer neurons than did Nicolelis (15–30 neurons versus 50–200 neurons).

John Donoghue's lab at the Carney Institute reported training rhesus monkeys to use a BCI to track visual targets on a computer screen (closed-loop BCI) with or without assistance of a joystick.^[41] Schwartz's group created a BCI for three-dimensional tracking in virtual reality and also reproduced BCI control in a robotic arm.^[42] The same group also created headlines when they demonstrated that a monkey could feed itself pieces of fruit and marshmallows using a robotic arm controlled by the animal's own brain signals.^[43][44][45]

Andersen's group used recordings of premovement activity from the posterior parietal cortex in their BCI, including signals created when experimental animals anticipated receiving a reward.^[46]

Other research

In addition to predicting kinematic and kinetic parameters of limb movements, BCIs that predict electromyographic or electrical activity of the muscles of primates are being developed.^[47] Such BCIs could be used to restore mobility in paralyzed limbs by electrically stimulating muscles.

Miguel Nicolelis and colleagues demonstrated that the activity of large neural ensembles can predict arm position. This work made possible creation of BCIs that read arm movement intentions and translate them into movements of artificial actuators. Carmena and colleagues^[38] programmed the neural coding in a BCI that allowed a monkey to control reaching and grasping movements by a robotic arm. Lebedev and colleagues^[39] argued that brain networks reorganize to create a new representation of the robotic appendage in addition to the representation of the animal's own limbs.

In 2019, researchers from UCSF published a study where they demonstrated a BCI that had the potential to help patients with speech impairment caused by neurological disorders. Their BCI used high-density electrocorticography to tap neural activity from a patient's brain and used deep learning methods to synthesize speech.^[48][49] In 2021, researchers from the same group published a study showing the potential of a BCI to decode words and sentences in an anarthric patient who had been unable to speak for over 15 years.^[50][51]

The biggest impediment to BCI technology at present is the lack of a sensor modality that provides safe, accurate and robust access to brain signals. It is conceivable or even likely, however, that such a sensor will be developed within the next twenty years. The use of such a sensor should greatly expand the range of communication functions that can be provided using a BCI.

Development and implementation of a BCI system is complex and time-consuming. In response to this problem, Gerwin Schalk has been developing a general-purpose system for BCI research, called BCI2000. BCI2000 has been in development since 2000 in a project led by the Brain–Computer Interface R&D Program at the Wadsworth Center of the New York State Department of Health in Albany, New York, United States.

A new 'wireless' approach uses light-gated ion channels such as Channelrhodopsin to control the activity of genetically defined subsets of neurons in vivo. In the context of a simple learning task, illumination of transfected cells in the somatosensory cortex influenced the decision-making process of freely moving mice.^[52]

The use of BMIs has also led to a deeper understanding of neural networks and the central nervous system. Research has shown that despite the inclination of neuroscientists to believe that neurons have the most effect when working together, single neurons can be conditioned through the use of BMIs to fire at a pattern that allows primates to control motor outputs. The use of BMIs has led to development of the single neuron insufficiency principle which states that even with a well tuned firing rate single neurons can only carry a narrow amount of information and therefore the highest level of accuracy is achieved by recording firings of the collective ensemble. Other principles discovered with the use of BMIs include the neuronal multitasking principle, the neuronal mass principle, the neural degeneracy principle, and the plasticity principle.^[53]

BCIs are also proposed to be applied by users without disabilities. A user-centered categorization of BCI approaches by Thorsten O. Zander and Christian Kothe introduces the term passive BCI.^[54] Next to active and reactive BCI that are used for directed control, passive BCIs allow for assessing and interpreting changes in the user state during Human-Computer Interaction (HCI). In a secondary, implicit control loop the computer system adapts to its user improving its usability in general.

Beyond BCI systems that decode neural activity to drive external effectors, BCI systems may be used to encode signals from the periphery. These sensory BCI devices enable real-time, behaviorally-relevant decisions based upon closed-loop neural stimulation.^[55]

The BCI Award

The Annual BCI Research Award is awarded in recognition of outstanding and innovative research in the field of Brain-Computer Interfaces. Each year, a renowned research laboratory is asked to judge the submitted projects. The jury consists of world-leading BCI experts recruited by the awarding laboratory. The jury selects twelve nominees, then chooses a first, second, and third-place winner, who receive awards of $3,000, $2,000, and $1,000, respectively.

Vision

Invasive BCI research has targeted repairing damaged sight and providing new functionality for people with paralysis. Invasive BCIs are implanted directly into the grey matter of the brain during neurosurgery. Because they lie in the grey matter, invasive devices produce the highest quality signals of BCI devices but are prone to scar-tissue build-up, causing the signal to become weaker, or even non-existent, as the body reacts to a foreign object in the brain.^[57]

In vision science, direct brain implants have been used to treat non-congenital (acquired) blindness. One of the first scientists to produce a working brain interface to restore sight was private researcher William Dobelle.

Dobelle's first prototype was implanted into "Jerry", a man blinded in adulthood, in 1978. A single-array BCI containing 68 electrodes was implanted onto Jerry's visual cortex and succeeded in producing phosphenes, the sensation of seeing light. The system included cameras mounted on glasses to send signals to the implant. Initially, the implant allowed Jerry to see shades of grey in a limited field of vision at a low frame-rate. This also required him to be hooked up to a mainframe computer, but shrinking electronics and faster computers made his artificial eye more portable and now enable him to perform simple tasks unassisted.^[58]

Dummy unit illustrating the design of a BrainGate interface

In 2002, Jens Naumann, also blinded in adulthood, became the first in a series of 16 paying patients to receive Dobelle's second generation implant, marking one of the earliest commercial uses of BCIs. The second generation device used a more sophisticated implant enabling better mapping of phosphenes into coherent vision. Phosphenes are spread out across the visual field in what researchers call "the starry-night effect". Immediately after his implant, Jens was able to use his imperfectly restored vision to drive an automobile slowly around the parking area of the research institute.^[59] Unfortunately, Dobelle died in 2004^[60] before his processes and developments were documented. Subsequently, when Mr. Naumann and the other patients in the program began having problems with their vision, there was no relief and they eventually lost their "sight" again. Naumann wrote about his experience with Dobelle's work in Search for Paradise: A Patient's Account of the Artificial Vision Experiment^[61] and has returned to his farm in Southeast Ontario, Canada, to resume his normal activities.^[62]

Movement

BCIs focusing on motor neuroprosthetics aim to either restore movement in individuals with paralysis or provide devices to assist them, such as interfaces with computers or robot arms.

Researchers at Emory University in Atlanta, led by Philip Kennedy and Roy Bakay, were first to install a brain implant in a human that produced signals of high enough quality to simulate movement. Their patient, Johnny Ray (1944–2002), suffered from 'locked-in syndrome' after suffering a brain-stem stroke in 1997. Ray's implant was installed in 1998 and he lived long enough to start working with the implant, eventually learning to control a computer cursor; he died in 2002 of a brain aneurysm.^[63]

Tetraplegic Matt Nagle became the first person to control an artificial hand using a BCI in 2005 as part of the first nine-month human trial of Cyberkinetics's BrainGate chip-implant. Implanted in Nagle's right precentral gyrus (area of the motor cortex for arm movement), the 96-electrode BrainGate implant allowed Nagle to control a robotic arm by thinking about moving his hand as well as a computer cursor, lights and TV.^[64] One year later, professor Jonathan Wolpaw received the prize of the Altran Foundation for Innovation to develop a Brain Computer Interface with electrodes located on the surface of the skull, instead of directly in the brain.

More recently, research teams led by the BrainGate group at Brown University^[65] and a group led by University of Pittsburgh Medical Center,^[66] both in collaborations with the United States Department of Veterans Affairs, have demonstrated further success in direct control of robotic prosthetic limbs with many degrees of freedom using direct connections to arrays of neurons in the motor cortex of patients with tetraplegia.

Communication

In May 2021, a Stanford University team reported a successful proof-of-concept test that enabled a quadraplegic participant to input English sentences at about 86 characters per minute and 18 words per minute. The participant imagined moving his hand to write letters, and the system performed handwriting recognition on electrical signals detected in the motor cortex, utilizing hidden Markov models and recurrent neural networks for decoding.^[67][68]

A report published in July 2021 reported a paralyzed patient was able to communicate 15 words per minute using a brain implant that analyzed motor neurons that previously controlled the vocal tract.^[69][50]

In a recent review article, researchers raised an open question of whether human information transfer rates can surpass that of language with BCIs. Given that recent language research has demonstrated that human information transfer rates are relatively constant across many languages, there may exist a limit at the level of information processing in the brain. On the contrary, this "upper limit" of information transfer rate may be intrinsic to language itself, as a modality for information transfer.^[70]

Technical challenges

There exist a number of technical challenges to recording brain activity with invasive BCIs. Advances in CMOS technology are pushing and enabling integrated, invasive BCI designs with smaller size, lower power requirements, and higher signal acquisition capabilities.^[71] Invasive BCIs involve electrodes that penetrate brain tissue in an attempt to record action potential signals (also known as spikes) from individual, or small groups of, neurons near the electrode. The interface between a recording electrode and the electrolytic solution surrounding neurons has been modelled using the Hodgkin-Huxley model.^[72][73]

Electronic limitations to invasive BCIs have been an active area of research in recent decades. While intracellular recordings of neurons reveal action potential voltages on the scale of hundreds of millivolts, chronic invasive BCIs rely on recording extracellular voltages which typically are three orders of magnitude smaller, existing at hundreds of microvolts.^[74] Further adding to the challenge of detecting signals on the scale of microvolts is the fact that the electrode-tissue interface has a high capacitance at small voltages. Due to the nature of these small signals, for BCI systems that incorporate functionality onto an integrated circuit, each electrode requires its own amplifier and ADC, which convert analog extracellular voltages into digital signals.^[74] Because a typical neuron action potential lasts for one millisecond, BCIs measuring spikes must have sampling rates ranging from 300 Hz to 5 kHz. Yet another concern is that invasive BCIs must be low-power, so as to dissipate less heat to surrounding tissue; at the most basic level more power is traditionally needed to optimize signal-to-noise ratio.^[73] Optimal battery design is an active area of research in BCIs.^[75]

Illustration of invasive and partially invasive BCIs: electrocorticography (ECoG), endovascular, and intracortical microelectrode.

Challenges existing in the area of material science are central to the design of invasive BCIs. Variations in signal quality over time have been commonly observed with implantable microelectrodes.^[76] Optimal material and mechanical characteristics for long term signal stability in invasive BCIs has been an active area of research.^[77] It has been proposed that the formation of glial scarring, secondary to damage at the electrode-tissue interface, is likely responsible for electrode failure and reduced recording performance.^[78] Research has suggested that blood-brain barrier leakage, either at the time of insertion or over time, may be responsible for the inflammatory and glial reaction to chronic microelectrodes implanted in the brain.^[78][79] As a result, flexible^[80][81][82] and tissue-like designs^[83][84] have been researched and developed to minimize foreign-body reaction by means of matching the Young's modulus of the electrode closer to that of brain tissue.^[83]

Endovascular

A systematic review published in 2020 detailed multiple studies, both clinical and non-clinical, dating back decades investigating the feasibility of endovascular BCIs.^[86]

In recent years, the biggest advance in partially invasive BCIs has emerged in the area of interventional neurology.^[2] In 2010, researchers affiliated with University of Melbourne had begun developing a BCI that could be inserted via the vascular system. The Australian neurologist Thomas Oxley (Mount Sinai Hospital) conceived the idea for this BCI, called Stentrode, which has received funding from DARPA. Preclinical studies evaluated the technology in sheep.

The Stentrode, a monolithic stent electrode array, is designed to be delivered via an intravenous catheter under image-guidance to the superior sagittal sinus, in the region which lies adjacent to motor cortex.^[87] This proximity to motor cortex underlies the Stentrode's ability to measure neural activity. The procedure is most similar to how venous sinus stents are placed for the treatment of idiopathic intracranial hypertension.^[88] The Stentrode communicates neural activity to a battery-less telemetry unit implanted in the chest, which communicates wirelessly with an external telemetry unit capable of power and data transfer. While an endovascular BCI benefits from avoiding craniotomy for insertion, risks such as clotting and venous thrombosis are possible. In pre-clinical animal studies implanted with Stentrode, twenty animals showed no evidence of thrombus formation after 190 days, possibly due to endothelial incorporation of the Stentrode into the vessel wall.^[87]

First-in-human trials with the Stentrode are underway.^[87] In November 2020, two participants that suffer from amyotrophic lateral sclerosis were able to wirelessly control an operating system to text, email, shop, and bank using direct thought through the Stentrode brain-computer interface,^[89] marking the first time a brain-computer interface was implanted via the patient's blood vessels, eliminating the need for open brain surgery.

ECoG

Electrocorticography (ECoG) measures the electrical activity of the brain taken from beneath the skull in a similar way to non-invasive electroencephalography, but the electrodes are embedded in a thin plastic pad that is placed above the cortex, beneath the dura mater.^[90] ECoG technologies were first trialled in humans in 2004 by Eric Leuthardt and Daniel Moran from Washington University in St Louis. In a later trial, the researchers enabled a teenage boy to play Space Invaders using his ECoG implant.^[91] This research indicates that control is rapid, requires minimal training, and may be an ideal tradeoff with regards to signal fidelity and level of invasiveness.^{[note 1]}

Signals can be either subdural or epidural, but are not taken from within the brain parenchyma itself. It has not been studied extensively until recently due to the limited access of subjects. Currently, the only manner to acquire the signal for study is through the use of patients requiring invasive monitoring for localization and resection of an epileptogenic focus.

ECoG is a very promising intermediate BCI modality because it has higher spatial resolution, better signal-to-noise ratio, wider frequency range, and less training requirements than scalp-recorded EEG, and at the same time has lower technical difficulty, lower clinical risk, and may have superior long-term stability than intracortical single-neuron recording.^[93] This feature profile and recent evidence of the high level of control with minimal training requirements shows potential for real world application for people with motor disabilities.^[94][95] Light reactive imaging BCI devices are still in the realm of theory.

Recent work published by Edward Chang and Joseph Makin from UCSF revealed that ECoG signals could be used to decode speech from epilepsy patients implanted with high-density ECoG arrays over the peri-Sylvian cortices.^[96][97] Their study achieved word error rates of 3% (a marked improvement from prior publications) utilizing an encoder-decoder neural network, which translated ECoG data into one of fifty sentences composed of 250 unique words.

Non-EEG-based human–computer interface

Functional near-infrared spectroscopy

In 2014 and 2017, a BCI using functional near-infrared spectroscopy for "locked-in" patients with amyotrophic lateral sclerosis (ALS) was able to restore some basic ability of the patients to communicate with other people.^[100][101]

Electroencephalography (EEG)-based brain-computer interfaces

Recordings of brainwaves produced by an electroencephalogram

After the BCI challenge was stated by Vidal in 1973, the initial reports on non-invasive approach included control of a cursor in 2D using VEP (Vidal 1977), control of a buzzer using CNV (Bozinovska et al. 1988, 1990), control of a physical object, a robot, using a brain rhythm (alpha) (Bozinovski et al. 1988), control of a text written on a screen using P300 (Farwell and Donchin, 1988).^[13]

In the early days of BCI research, another substantial barrier to using Electroencephalography (EEG) as a brain–computer interface was the extensive training required before users can work the technology. For example, in experiments beginning in the mid-1990s, Niels Birbaumer at the University of Tübingen in Germany trained severely paralysed people to self-regulate the slow cortical potentials in their EEG to such an extent that these signals could be used as a binary signal to control a computer cursor.^[102] (Birbaumer had earlier trained epileptics to prevent impending fits by controlling this low voltage wave.) The experiment saw ten patients trained to move a computer cursor by controlling their brainwaves. The process was slow, requiring more than an hour for patients to write 100 characters with the cursor, while training often took many months. However, the slow cortical potential approach to BCIs has not been used in several years, since other approaches require little or no training, are faster and more accurate, and work for a greater proportion of users.

Another research parameter is the type of oscillatory activity that is measured. Gert Pfurtscheller founded the BCI Lab 1991 and fed his research results on motor imagery in the first online BCI based on oscillatory features and classifiers. Together with Birbaumer and Jonathan Wolpaw at New York State University they focused on developing technology that would allow users to choose the brain signals they found easiest to operate a BCI, including mu and beta rhythms.

A further parameter is the method of feedback used and this is shown in studies of P300 signals. Patterns of P300 waves are generated involuntarily (stimulus-feedback) when people see something they recognize and may allow BCIs to decode categories of thoughts without training patients first. By contrast, the biofeedback methods described above require learning to control brainwaves so the resulting brain activity can be detected.

In 2005 it was reported research on EEG emulation of digital control circuits for BCI, with example of a CNV flip-flop.^[103] In 2009 it was reported noninvasive EEG control of a robotic arm using a CNV flip-flop.^[104] In 2011 it was reported control of two robotic arms solving Tower of Hanoi task with three disks using a CNV flip-flop.^[105] In 2015 it was described EEG-emulation of a Schmidt trigger, flip-flop, demultiplexer, and modem.^[106]

While an EEG based brain-computer interface has been pursued extensively by a number of research labs, recent advancements made by Bin He and his team at the University of Minnesota suggest the potential of an EEG based brain-computer interface to accomplish tasks close to invasive brain-computer interface. Using advanced functional neuroimaging including BOLD functional MRI and EEG source imaging, Bin He and co-workers identified the co-variation and co-localization of electrophysiological and hemodynamic signals induced by motor imagination.^[107] Refined by a neuroimaging approach and by a training protocol, Bin He and co-workers demonstrated the ability of a non-invasive EEG based brain-computer interface to control the flight of a virtual helicopter in 3-dimensional space, based upon motor imagination.^[108] In June 2013 it was announced that Bin He had developed the technique to enable a remote-control helicopter to be guided through an obstacle course.^[109]

In addition to a brain-computer interface based on brain waves, as recorded from scalp EEG electrodes, Bin He and co-workers explored a virtual EEG signal-based brain-computer interface by first solving the EEG inverse problem and then used the resulting virtual EEG for brain-computer interface tasks. Well-controlled studies suggested the merits of such a source analysis based brain-computer interface.^[110]

A 2014 study found that severely motor-impaired patients could communicate faster and more reliably with non-invasive EEG BCI, than with any muscle-based communication channel.^[111]

A 2016 study found that the Emotiv EPOC device may be more suitable for control tasks using the attention/meditation level or eye blinking than the Neurosky MindWave device.^[112]

A 2019 study found that the application of evolutionary algorithms could improve EEG mental state classification with a non-invasive Muse device, enabling high quality classification of data acquired by a cheap consumer-grade EEG sensing device.^[113]

In a 2021 systematic review of randomized controlled trials using BCI for upper-limb rehabilitation after stroke, EEG-based BCI was found to have significant efficacy in improving upper-limb motor function compared to control therapies. More specifically, BCI studies that utilized band power features, motor imagery, and functional electrical stimulation in their design were found to be more efficacious than alternatives.^[114] Another 2021 systematic review focused on robotic-assisted EEG-based BCI for hand rehabilitation after stroke. Improvement in motor assessment scores was observed in three of eleven studies included in the systematic review.^[115]

Dry active electrode arrays

In the early 1990s Babak Taheri, at University of California, Davis demonstrated the first single and also multichannel dry active electrode arrays using micro-machining. The single channel dry EEG electrode construction and results were published in 1994.^[116] The arrayed electrode was also demonstrated to perform well compared to silver/silver chloride electrodes. The device consisted of four sites of sensors with integrated electronics to reduce noise by impedance matching. The advantages of such electrodes are: (1) no electrolyte used, (2) no skin preparation, (3) significantly reduced sensor size, and (4) compatibility with EEG monitoring systems. The active electrode array is an integrated system made of an array of capacitive sensors with local integrated circuitry housed in a package with batteries to power the circuitry. This level of integration was required to achieve the functional performance obtained by the electrode.

The electrode was tested on an electrical test bench and on human subjects in four modalities of EEG activity, namely: (1) spontaneous EEG, (2) sensory event-related potentials, (3) brain stem potentials, and (4) cognitive event-related potentials. The performance of the dry electrode compared favorably with that of the standard wet electrodes in terms of skin preparation, no gel requirements (dry), and higher signal-to-noise ratio.^[117]

In 1999 researchers at Case Western Reserve University, in Cleveland, Ohio, led by Hunter Peckham, used 64-electrode EEG skullcap to return limited hand movements to quadriplegic Jim Jatich. As Jatich concentrated on simple but opposite concepts like up and down, his beta-rhythm EEG output was analysed using software to identify patterns in the noise. A basic pattern was identified and used to control a switch: Above average activity was set to on, below average off. As well as enabling Jatich to control a computer cursor the signals were also used to drive the nerve controllers embedded in his hands, restoring some movement.^[118]

SSVEP mobile EEG BCIs

In 2009, the NCTU Brain-Computer-Interface-headband was reported. The researchers who developed this BCI-headband also engineered silicon-based microelectro-mechanical system (MEMS) dry electrodes designed for application in non-hairy sites of the body. These electrodes were secured to the DAQ board in the headband with snap-on electrode holders. The signal processing module measured alpha activity and the Bluetooth enabled phone assessed the patients' alertness and capacity for cognitive performance. When the subject became drowsy, the phone sent arousing feedback to the operator to rouse them. This research was supported by the National Science Council, Taiwan, R.O.C., NSC, National Chiao-Tung University, Taiwan's Ministry of Education, and the U.S. Army Research Laboratory.^[119]

In 2011, researchers reported a cellular based BCI with the capability of taking EEG data and converting it into a command to cause the phone to ring. This research was supported in part by Abraxis Bioscience LLP, the U.S. Army Research Laboratory, and the Army Research Office. The developed technology was a wearable system composed of a four channel bio-signal acquisition/amplification module, a wireless transmission module, and a Bluetooth enabled cell phone.  The electrodes were placed so that they pick up steady state visual evoked potentials (SSVEPs).^[120] SSVEPs are electrical responses to flickering visual stimuli with repetition rates over 6 Hz^[120] that are best found in the parietal and occipital scalp regions of the visual cortex.^{[121][122][123]} It was reported that with this BCI setup, all study participants were able to initiate the phone call with minimal practice in natural environments.^[124]

The scientists claim that their studies using a single channel fast Fourier transform (FFT) and multiple channel system canonical correlation analysis (CCA) algorithm support the capacity of mobile BCIs.^[120][125] The CCA algorithm has been applied in other experiments investigating BCIs with claimed high performance in accuracy as well as speed.^[126] While the cellular based BCI technology was developed to initiate a phone call from SSVEPs, the researchers said that it can be translated for other applications, such as picking up sensorimotor mu/beta rhythms to function as a motor-imagery based BCI.^[120]

In 2013, comparative tests were performed on android cell phone, tablet, and computer based BCIs, analyzing the power spectrum density of resultant EEG SSVEPs. The stated goals of this study, which involved scientists supported in part by the U.S. Army Research Laboratory, were to "increase the practicability, portability, and ubiquity of an SSVEP-based BCI, for daily use". Citation It was reported that the stimulation frequency on all mediums was accurate, although the cell phone's signal demonstrated some instability. The amplitudes of the SSVEPs for the laptop and tablet were also reported to be larger than those of the cell phone. These two qualitative characterizations were suggested as indicators of the feasibility of using a mobile stimulus BCI.^[125]

Limitations

In 2011, researchers stated that continued work should address ease of use, performance robustness, reducing hardware and software costs.^[120]

One of the difficulties with EEG readings is the large susceptibility to motion artifacts.^[127] In most of the previously described research projects, the participants were asked to sit still, reducing head and eye movements as much as possible, and measurements were taken in a laboratory setting. However, since the emphasized application of these initiatives had been in creating a mobile device for daily use,^[125] the technology had to be tested in motion.

In 2013, researchers tested mobile EEG-based BCI technology, measuring SSVEPs from participants as they walked on a treadmill at varying speeds. This research was supported by the Office of Naval Research, Army Research Office, and the U.S. Army Research Laboratory. Stated results were that as speed increased the SSVEP detectability using CCA decreased. As independent component analysis (ICA) had been shown to be efficient in separating EEG signals from noise,^[128] the scientists applied ICA to CCA extracted EEG data. They stated that the CCA data with and without ICA processing were similar. Thus, they concluded that CCA independently demonstrated a robustness to motion artifacts that indicates it may be a beneficial algorithm to apply to BCIs used in real world conditions.^[122]

Prosthesis and environment control

Non-invasive BCIs have also been applied to enable brain-control of prosthetic upper and lower extremity devices in people with paralysis. For example, Gert Pfurtscheller of Graz University of Technology and colleagues demonstrated a BCI-controlled functional electrical stimulation system to restore upper extremity movements in a person with tetraplegia due to spinal cord injury.^[129] Between 2012 and 2013, researchers at the University of California, Irvine demonstrated for the first time that it is possible to use BCI technology to restore brain-controlled walking after spinal cord injury. In their spinal cord injury research study, a person with paraplegia was able to operate a BCI-robotic gait orthosis to regain basic brain-controlled ambulation.^[130][131] In 2009 Alex Blainey, an independent researcher based in the UK, successfully used the Emotiv EPOC to control a 5 axis robot arm.^[132] He then went on to make several demonstration mind controlled wheelchairs and home automation that could be operated by people with limited or no motor control such as those with paraplegia and cerebral palsy.

Research into military use of BCIs funded by DARPA has been ongoing since the 1970s.^[3][4] The current focus of research is user-to-user communication through analysis of neural signals.^[133]

DIY and open source BCI

In 2001, The OpenEEG Project^[134] was initiated by a group of DIY neuroscientists and engineers. The ModularEEG was the primary device created by the OpenEEG community; it was a 6-channel signal capture board that cost between $200 and $400 to make at home. The OpenEEG Project marked a significant moment in the emergence of DIY brain-computer interfacing.

In 2010, the Frontier Nerds of NYU's ITP program published a thorough tutorial titled How To Hack Toy EEGs.^[135] The tutorial, which stirred the minds of many budding DIY BCI enthusiasts, demonstrated how to create a single channel at-home EEG with an Arduino and a Mattel Mindflex at a very reasonable price. This tutorial amplified the DIY BCI movement.

In 2013, OpenBCI emerged from a DARPA solicitation and subsequent Kickstarter campaign. They created a high-quality, open-source 8-channel EEG acquisition board, known as the 32bit Board, that retailed for under $500. Two years later they created the first 3D-printed EEG Headset, known as the Ultracortex, as well as a 4-channel EEG acquisition board, known as the Ganglion Board, that retailed for under $100.

MEG and MRI

ATR Labs' reconstruction of human vision using fMRI (top row: original image; bottom row: reconstruction from mean of combined readings)

Magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI) have both been used successfully as non-invasive BCIs.^[136] In a widely reported experiment, fMRI allowed two users being scanned to play Pong in real-time by altering their haemodynamic response or brain blood flow through biofeedback techniques.^[137]

fMRI measurements of haemodynamic responses in real time have also been used to control robot arms with a seven-second delay between thought and movement.^[138]

In 2008 research developed in the Advanced Telecommunications Research (ATR) Computational Neuroscience Laboratories in Kyoto, Japan, allowed the scientists to reconstruct images directly from the brain and display them on a computer in black and white at a resolution of 10x10 pixels. The article announcing these achievements was the cover story of the journal Neuron of 10 December 2008.^[139]

In 2011 researchers from UC Berkeley published^[140] a study reporting second-by-second reconstruction of videos watched by the study's subjects, from fMRI data. This was achieved by creating a statistical model relating visual patterns in videos shown to the subjects, to the brain activity caused by watching the videos. This model was then used to look up the 100 one-second video segments, in a database of 18 million seconds of random YouTube videos, whose visual patterns most closely matched the brain activity recorded when subjects watched a new video. These 100 one-second video extracts were then combined into a mashed-up image that resembled the video being watched.^{[141][142][143]}

BCI control strategies in neurogaming