Soldiers able to communicate silently could change the face of warfare on the ground, and perhaps the mind reading technology could prove to be more dangerous than the nuclear bomb.
The brain typically works by sending a signal to peripheral nerves and muscles to induce movement of a limb or any such other action. A brain computer interface (BCI) is a bidirectional communication pathway between the brain and an external device, designed to acquire, analyse, and translate brain signals for a specific action. BCIs provide a new output channel for the brain to communicate with and ultimately control an external device, which could be an artificial limb, a simulated aircraft, or any other device interfaced with a computer.
The human brain cells’ messages are transmitted as electrical signals through nerves, and it is still not fully understood as to how the brain codes these electrical signals. Hence it is quite complex reading these signals completely and then transmitting or receiving them surreptitiously over a distance.
Though the ability to send thoughts directly to another person’s brain remains somewhat science fiction, technology is now available, in a limited way, to sense thoughts in one’s mind and transmit them into another brain.
Experiments have been carried out by planting electrodes inside the human skull to sense the electrical activity produced by the brain cells. Another technology used is non-invasive electro encephalography (EEG)—a monitoring method to record voltage fluctuations representing the activity of the brain.
Decoding received signals has been a problem, hence, respondents have been asked to imagine performing simple tasks (while not actually doing them) so that the electric activity created in the brain could be recorded as EEG patterns. Another challenge was beaming thoughts into someone else’s brain. Initially electrodes were used to pass current into the brain to stimulate the imagined thought. To avoid passing current into the brain, a transcranial magnetic stimulation (TMS) wand was used, which induced a current to activate the brain tissue.
In brain-to-brain communication, the ‘sender’ is asked to concentrate on a task, the electric activity of their brain is recorded using EEG and transmitted to a TMS wand placed on the head of the ‘receiver’ to stimulate their brain to reproduce the task.
Technology’s evolution
Brain-to-brain communication has been of great interest for past many years. Hans Berger, credited with the discovery of the electrical activity of the human brain, recorded it for the first time in 1924 using EEG. In 1969, Fetz showed that monkeys could control the deflection of a biofeedback arm with neural activity.
The research on BCIs began in the 1970s at the University of California while it was in 1977 that Professor Jacques Vidal demonstrated the visually evoked potential controlling a graphical object on a computer screen. In 1988, Bozinovski described controlling multiple start-stop-restart movements of a robot using EEG. In 2000, Nicolelis used BCIs to decode brain activity in monkeys and reproduce their movements in robotic arms. In 2008, photographs published in science journals showed a monkey operating a robotic arm by thinking.
By 2010s, the studies suggested that neural stimulation could restore functional connectively and associated behaviours through modulation of molecular mechanisms of synaptic efficacy. In 2011 O’Doherty demonstrated a BCI that enabled a monkey to brain control the position of an avatar arm by receiving sensory feedback through direct intracortical stimulation in the sensory cortex.
Recently, in 2019, a BCI showcased usage of deep learning to synthesize speech to help patients suffering from speech impairment due to neurological disorders. In 2020, Elon Musk’s Neuralink was successfully implanted in a pig and the University of California demonstrated a computing system to translate brainwaves into sentences with the word error rates as low as 3%. In 2021 Musk announced that a monkey could play video games using a BCI device.
Electromyography is the recording of the electrical activity of muscle tissue, or its representation as a visual display or audible signal, using electrodes attached to the skin or inserted into the muscle. BCIs that predict electromyographic activity are being developed to restore mobility in paralysed limbs.
The U.S. Army Research Laboratory has recently developed a mobile BCI device that measures steady state visually evoked potentials (SSVEPs), which are signals that are natural responses to visual stimulation at specific frequencies. Mind-controlled wheelchairs and home automation applications have been demonstrated being operated by people with limited or no motor control.
TYPES OF BCIs
Invasive BCIs
Invasive BCIs use electrodes implanted inside the skull to sense the electrical activity produced by the brain cells. As these lie in the grey matter, they produce the highest quality signals. However, they are prone to scar-tissue build-up due to body reaction to the presence of foreign objects causing them to become weaker or even non-existent with the passage of time. In some cases, the body may not accept the electrodes implanted into brain causing a medical condition.
Generally, the invasive BCIs are used for repairing damaged sight and providing new functionality for people with paralysis. For example, a person using a BrainGate chip-implant could control a robotic arm as well as a computer cursor, lights and TV just by thinking about the activity.
Partially invasive BCIs
Partially invasive BCI devices are implanted inside the skull but outside the grey matter. They produce signals with resolution higher than non-invasive BCIs and have a lower risk of forming scar-tissue in the brain. It has also been possible for these to be inserted via the vascular system, eliminating the need for open brain surgery. In another technology, called electrocorticography (ECoG), the electrodes are placed above the cortex, beneath the dura mater (outermost membrane enveloping the brain and spinal cord).
Partially invasive technology has lower complexity, lower clinical risk and superior long-term stability. They provide higher spatial resolution, better signal-to-noise ratio, wider frequency range, and have minimal training requirements.
Non-invasive BCIs
Non-invasive BCI sensors are placed on or very close to the head to measure the brain activity, thus avoiding any surgery. EEGs are the most commonly used ones. Another mechanism used is magnetoencephalography (MEG)—a non-invasive neurophysiological technique that measures the magnetic fields generated by neuronal activity of the brain. It is a direct measure of brain function providing a very high temporal resolution.
Non-invasive devices are easy to wear, have lower costs and greater portability. They have relatively poor spatial resolution and are unable to effectively use higher frequencies as the skull dampens the signals, dispersing and blurring the electromagnetic waves created by the neurons. Compared to invasive devices, these are susceptible to noise and have worse signal resolution due to the distance from the brain.
BCI TECHNOLOGIES
BCIs based on visual evoked potential
Visual evoked potentials (VEPs) are the brain activity modulations occurring in the visual cortex after encountering visual stimulus. They are easy to detect as the movement of stimulus closer to the central visual field greatly enhances the amplitude of the VEPs. In VEP based BCIs, the electrical potentials caused by visual stimuli such as an alternating checkerboard pattern or a flashing light are recorded on an EEG using electrodes placed at the back of head.
VEP based BCIs are becoming popular given the advantage of less user training that is required and high information transfer rate (ITR). VEP based BCIs include steady-state visually evoked potential (SSVEP) BCIs and P300 event related potential (ERP) based BCIs.
In SSVEP based BCI systems, when the eyes are excited by a visual stimulus signal, the brain generates response at the same frequency.The best responses are obtained for frequencies between 5 and 20Hz. The response signals of these stimuli are detected from the scalp and analysed. Stimuli used are a periodic visual stimulation such as alternating checkerboard patterns or flashing images.
The signals elicited have high signal-to-noise ratio, are robust, and are easy to record. However, these may be irritating and uncomfortable to the users, as they have to gaze for long at the flashing light or iterating symbols during the sessions. SSVEP based BCIs can provide hands-free human interaction with the environment.
Another type of VEP based BCI used is the P300VEP—a positive peak in the EEG that occurs at roughly 300ms after the appearance of a target stimulus, which the user is seeking or oddball stimuli. The P300 amplitude decreases as the target stimuli and the ignored stimuli become more similar.
In an experiment with P300 based BCI, the subject was asked to choose a letter from a grid of 6 by 6 letters and numbers, the EEG signals from the subject’s scalp were measured and processed in real time to detect the P300 potential, which reflected the subject’s imagined choice letter. The advantage of P300 is that the subject has to undertake shorter training to learn the use of the BCI paradigm
BCIs based on electrooculography
The technique of measuring cornea retinal potential associated with eye movement is called electrooculography (EOG). An eye is an electric dipole. Changes in the position of the eyeball produce variations in the electric charge registered by the electrodes.
EOG is used for measuring the corneo-retinal standing potential that exists between the front and the back of a human eye and the resulting signal, called electrooculogram, is used for recording eye movements to provide the sensitive visual stimuli.
In an experiment, a robot could be driven from start to a goal point by EOG based commands—forward, backward, left, right, and stop, using the visual fixation by the subject. A human-computer interface has also been developed, which does not require visual fixation or movement of the eyes and is based on directing attention to a chosen letter on a virtual keyboard by looking directly at the letter.
BCIs based on magnetoencephalography and functional magnetic resonance imaging (fMRI)
Magnetoencephalography (MEG) is a functional neuroimaging technique for mapping brain activity by recording magnetic fields produced by electrical currents occurring naturally in the brain. Functional magnetic resonance imaging (fMRI) measures brain activity by detecting changes associated with blood flow. This technique is based on the fact that cerebral blood flow and neuronal activation are coupled, such that when an area of the brain is in use, the blood flow to that region increases.
MEG and fMRI are used as non-invasive BCIs. In an experiment, fMRI based BCI was used by two respondents to play Pong (an electronic game that simulates table tennis by using in-game paddles to hit the ball back and forth) in real time by altering their brain blood flow through biofeedback techniques. fMRI has also been used to control robot arms with a seven-second delay between thought and movement.
Brain-to-brain communication system
In a brain-to-brain communication system two participants, Brain 1 and Brain 2, located about 2km apart communicate using EEG based non-invasive BCIs to play a video game. Brain 1 can see the game but has no controller. Brain 2 has a controller but cannot see the game. When wanting to make a move, Brain 1 is required to simply think about moving his hand and the brain signals produced are picked up by a cap of electrodes worn on his head and transmitted to a TMS device placed on the head of Brain 2. The TMS beams the signal into Brain 2. The stimulation automatically makes Brain 2’s hand to move on the touch pad to manipulate the controller as thought by Brain 1. The communication takes a split second and the success rate varies from 25 to 83%, depending on the pair of people participating.
BCI APPLICATIONS
Patients with disorders of consciousness
For patients suffering with disorders of consciousness, the end-of-life decisions are made by family members considering the patient’s best interest. For the patients who are able to perform basic cognitive tasks, the BCI technology could make available communication tools to enable them make these decisions as per their desire.
However, such patients may not be able to use BCIs that are based on vision, auditory or vibrotactile stimuli, as there are chances of the patients fading in and out of consciousness. In some cases, it may become difficult to determine when they would respond, or the response may last for a very little time only.
Motor imagery BCIs for neurorehabilitation
Motor imagery is a cognitive process in which a person imagines performing a movement without performing it, or even tensing his/her muscles. In motor imagery based BCIs the user generates the induced activity from motor cortex by imagining motor movements without any limb movement or external stimulus. Motor imagery BCIs serve as an interface to communicate the user’s intention without limb movement.
The most representative application is for neurorehabilitation. The University of Melbourne is developing a system to enable patients with paralysis to control external devices such as robotic limbs, computers, and exoskeletons by translating their brain activity.
Functional brain mapping
Functional brain mapping procedure is often required during neurosurgery for patients with tumors or epilepsy, for which electrodes are placed on the brain to precisely identify the locations of structures and functional areas. New ways are being explored for using BCI technology to improve the mapping using non-invasive means.
Neurogaming
Neurogaming involves the use of BCIs so that the users can interact with the game without the use of a traditional controller. BCIs detect and infer the gamer’s intent. Intensive training is required by the users to acquire the acceptable ability to master the control system.
BCIs for armed forces
Direct brain-to-brain communication offers the most transformative application for the armed forces for operational use. The US Army has initiated a research project to explore the possibility of creating a technology that will enable soldiers to convey messages by thought, for use in covert operations. The project involves recording one’s thoughts, transmitting the recording to a particular individual who decodes it as a voice or text.
The US Army Research Laboratory is developing helmets with EEG sensors to monitor the brain activity of the user soldiers. The US Air Force is developing helmets, incorporating BCI system to monitor pilot’s cognitive workload and stress. Défense Advanced Research Projects Agency (DARPA) has initiated research for development of BCI-enabled prosthetic hands and arms that provide feedback to the user’s brain and evoke sensations that the user perceives as coming from his/her own hand. DARPA has also initiated a proof-of-principle project, MOANA (magnetic, optical, and acoustic neural access), to create a dual-function, wireless headset capable of ‘reading’ and ‘writing’ brain activity to restore lost sensory function without the need for surgery.
Research is on to allow the paralysed soldiers to perform basic tasks like controlling a wheelchair using the brain signals. The US Air Force has initiated projects to provide the quadriplegics the ability to perform basic operations in an F-35 fighter jet simulator with their thoughts. However, the use of the technology involves operational and institutional risks such as creating new operational vulnerabilities and areas of ethical and legal risks.
BrainNet, a proof-of-principle network
A team of neuroscience and computer science researchers have created a network, called BrainNet, which allows three people to communicate using only their minds and an internet connection. The network uses the characteristic that the brain activity can easily synchronize or align with the external signals. For example, a light flashing at 15Hz causes the brain to emit an electrical signal at the same frequency, and if the frequency is changed to 17Hz the frequency of the brain signal also gets changed to 17Hz.
Architecture of BrainNet:
- Share this on Facebook (opens in new window)
- Share this on Twitter (opens in new window)
- SHAREAll sharing options
In an experiment, the BrainNet connected two ‘senders’ and one ‘receiver,’ located in separate rooms, to play a video game in which a falling block had to be rotated to fit into a line at the bottom of the screen. The senders were equipped with EEGs that recorded their brain’s electrical activity. The receiver, attached to a TMS, acted as per the received cues from the senders. The two senders selected the option by focussing on the lights flashing on either side of the screen—one at 15Hz and the other at 17Hz.
The brain responses of the senders, captured by the EEGs, were transmitted over the internet to the TMS connected to the receiver. The TMS delivered a magnetic pulse to the receiver’s brain causing a transcranial magnetic stimulation, which caused a flash of light (a phosphene) in the receiver’s visual field making him act as per the option selected by the sender. Once the receiver acted, the game concluded.
Areas of concern
The brain computer interfaces and the associated technologies raise ethical concerns such as managing patient expectations, the concept of personal identity, and the validity of informed consent. One significant ethical issue is getting an informed consent from the patients. It is necessary to be able to distinguish if the consent obtained is fully informed or is affected by the disability. Patients and their relatives are likely to have high expectations from BCI technologies that may not get met. Not working up to the expectations may cause significant distress to them, outweighing the possible benefits.
Use of implantable devices might pose risks for the patient’s health. The ability to read minds has a profound impact on the concept of personal identity and privacy. It is important to consider the ethical implications such as safety and security of the data transmitted and stored; breaching the ownership of personal information and data; and precluding other persons or the hackers accessing ‘thoughts.’
Areas of concern that need to be explored further include unknown long-term effect; effects of the technology on the quality of life of the subjects and their families; health-related side-effects such as the sleep quality, normal brain functioning and memory; and non-convertibility of the changes made to the brain.
There are also several legal and social issues that need to be settled such as the accountability of and the responsibility for the influence of BCIs; inaccurately translating the cognitive intentions; possible personality changes; inability to distinguish between human vs machine-controlled actions; misuse of the techniques during interrogation by the authorities; mind reading capability and privacy; and issues of mind control and emotion control. Fixation of legal responsibility in cases where accidents happen is an area of concern.
How would it be possible to distinguish a voluntary action from a malfunction of the system? Legal issues may also arise where the responses are obtained from seriously ill patients, as to whether these can be considered an informed consent or not given their status. The associated safeguards need to be in place before the technology is deployed and the ethical guidelines for research, development and dissemination need to be formed with global consensus.
Despite the exciting and dynamic future that the BCIs hold, one challenge is the lack of complete understanding of the brain. The human brain contains about 100 billion neurons with each having up to 10,000 connections to its surrounding neurons. The dynamics of the electrochemical interactions between the neurons and how these are translated into memories, behaviours, perceptions, and actions is still not fully understood.
The other challenge is the body’s response to invasive BCIs using an implanted array of micro-electrodes in direct contact with the specific neurons in the brain. The electrodes are recognised as a foreign body triggering the natural immune system and they get surrounded by fibrous capsules of tissue, reducing the signal recording ability of the electrodes, and ultimately causing the BCIs to become unusable.
Reducing power consumption to decrease the battery size and prolong lifespan is another challenge as there is a trade-off between the power consumption and efficient bio-security. To ensure bio-security, the signals need to be encrypted, and this increases the power consumption.
The biggest impediment to the BCI technology is the lack of a sensor modality that provides safe, accurate, and robust access to brain signals. Also developing sensors with additional channels to improve accuracy and minimising their power usage is a challenge. The ethical, legal, and social implications of BCIs may slow down, stop, or divert the technology into something completely different from what was aimed.
Conclusion and the future
All the information inside the brain cannot be put in a linguistic form, as it is difficult to verbalise it into words. For instance, an experienced surgeon cannot simply ‘tell’ an intern how to exactly position and move the fingers during a critical surgery. BCIs provide alternate methods to interact with the outside world as they do not depend on the brain’s normal output pathways such as peripheral nerves and muscles. Direct brain-to-brain communication has been driven by the motives as diverse as futurist enthusiasm and military exigency, to make it possible to replace language by directly linking the activity of human brains and ultimately providing a new means of communication.
However, despite all that the scientists and researchers have achieved in recent times, the mind remains a black box. To open a tiny crack in that black box is a significant achievement as there are many unknown areas such as interpreting the brain electric activity associated with deeper experiences like hope, love. or sadness.
In the last few years, BCI research has gathered significant momentum. Mark Zuckerberg recently revealed that they are looking at the brain as the next big communication platform and hope to produce a BCI that will allow people to type three times faster.
Elon Musk has created Neura-6 link to provide a direct cortical interface with the brain via neural lace, to help humanity better compete with AI. He is developing a robotically implantable BCI containing 3,000 electrodes to provide extensive interaction between computers and the brain nerve cells.
Kernal, headed by Bryan Johnson, is going big with a BCI device to enable people to learn faster, remember more, ‘co-evolve’ with artificial intelligence, and even connect into group minds. Charles Stark Draper Labs is developing wirelessly connected implants for prosthetics, a live remote-controlled dragonfly with a BCI, and a field cap to read neural activity noninvasively. DARPA is developing an implantable neural interface capable of engaging one million nerve cells simultaneously.
Brain-to-brain communication has potential for use in a future battlefield to enhance the speed of communication, improve situational awareness, and allow operators to control multiple technological platforms simultaneously. Neurotechnological advances have given quadriplegics the ability to perform in an F-35 simulator with their thoughts and the scientists the ability to decode the speech that the subjects are imagining, albeit imperfectly.
Nicolelis, a Brazilian soccer aficionado, built a robotic exoskeleton controlled by EEG impulses for the 2014 World Cup to enable a young paraplegic person to deliver the ceremonial first kick.
In the last decade, the genetic sequencing technology and modern tools for mapping have helped in increased understanding of the neuronal firing patterns and how they lead to different actions. Brain-interfacing devices are becoming more sensitive, smaller, smarter, and portable over time. Future technologies need to address issues related to ease of use, performance robustness, and cost reduction.
Dr Deepak Halan is currently Associate Professor with Jaipuria Institute of Management, NOIDA
