A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation
Volume 7 Issue 5
Sep.  2020

IEEE/CAA Journal of Automatica Sinica

  • JCR Impact Factor: 6.171, Top 11% (SCI Q1)
    CiteScore: 11.2, Top 5% (Q1)
    Google Scholar h5-index: 51, TOP 8
Turn off MathJax
Article Contents
Arnab Rakshit, Amit Konar and Atulya K. Nagar, "A Hybrid Brain-Computer Interface for Closed-Loop Position Control of a Robot Arm," IEEE/CAA J. Autom. Sinica, vol. 7, no. 5, pp. 1344-1360, Sept. 2020. doi: 10.1109/JAS.2020.1003336
Citation: Arnab Rakshit, Amit Konar and Atulya K. Nagar, "A Hybrid Brain-Computer Interface for Closed-Loop Position Control of a Robot Arm," IEEE/CAA J. Autom. Sinica, vol. 7, no. 5, pp. 1344-1360, Sept. 2020. doi: 10.1109/JAS.2020.1003336

A Hybrid Brain-Computer Interface for Closed-Loop Position Control of a Robot Arm

doi: 10.1109/JAS.2020.1003336
More Information
  • Brain-Computer interfacing (BCI) has currently added a new dimension in assistive robotics. Existing brain-computer interfaces designed for position control applications suffer from two fundamental limitations. First, most of the existing schemes employ open-loop control, and thus are unable to track positional errors, resulting in failures in taking necessary online corrective actions. There are examples of a few works dealing with closed-loop electroencephalography (EEG)-based position control. These existing closed-loop brain-induced position control schemes employ a fixed order link selection rule, which often creates a bottleneck preventing time-efficient control. Second, the existing brain-induced position controllers are designed to generate a position response like a traditional first-order system, resulting in a large steady-state error. This paper overcomes the above two limitations by keeping provisions for steady-state visual evoked potential (SSVEP) induced link-selection in an arbitrary order as required for efficient control and generating a second-order response of the position-control system with gradually diminishing overshoots/undershoots to reduce steady-state errors. Other than the above, the third innovation is to utilize motor imagery and P300 signals to design the hybrid brain-computer interfacing system for the said application with gradually diminishing error-margin using speed reversal at the zero-crossings of positional errors. Experiments undertaken reveal that the steady-state error is reduced to 0.2%. The paper also provides a thorough analysis of the stability of the closed-loop system performance using the Root Locus technique.


  • loading
  • [1]
    Z. T. Chen, Z. P. Wang, K. Wang, W. B. Yi, and H. Z. Qi, “Recognizing motor imagery between hand and forearm in the same limb in a hybrid brain computer interface paradigm: An online study,” IEEE Access, vol. 7, pp. 59631–59639, May 2019. doi: 10.1109/ACCESS.2019.2915614
    E. W. Yin, T. Zeyl, R. Saab, T. Chau, D. W. Hu, and Z. T. Zhou, “A hybrid brain-computer interface based on the fusion of P300 and SSVEP scores,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 23, no. 4, pp. 693–701, Jul. 2015. doi: 10.1109/TNSRE.2015.2403270
    J. R. Wolpaw and E. W. Wolpaw, Brain-Computer Interfaces: Principles and Practice. Oxford, UK: Oxford University Press, 2012.
    J. Zhao, W. Li, and M. F. Li, “Comparative study of SSVEP- and P300-based models for the telepresence control of humanoid robots,” PLoS One, vol. 10, no. 11, pp. e0142168, Nov. 2015. doi: 10.1371/journal.pone.0142168
    X. Q. Mao, W. Li, C. W. Lei, J. Jin, F. Duan, and S. Chen, “A brain-robot interaction system by fusing human and machine intelligence,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 27, no. 3, pp. 533–542, Mar. 2019. doi: 10.1109/TNSRE.2019.2897323
    T. Bastos-Filho, A. Floriano, E. Couto, and R. J. M. Godinez-Tello, “Towards a system to command a robotic wheelchair based on independent SSVEP-BCI, ” in Smart Wheelchairs and Brain-Computer Interfaces, P. Diez, Ed. London, UK: Academic Press, 2018, pp. 369–379.
    A. F. Salazar-Gomez, J. DelPreto, S. Gil, F. H. Guenther, and D. Rus, “Correcting robot mistakes in real time using EEG signals, ” in Proc. IEEE Int. Conf. Robotics and Automation, Singapore, 2017, pp. 6570–6577.
    Y. Yu, Z. T. Zhou, Y. D. Liu, J. Jiang, E. W. Yin, N. N. Zhang, Z. H. Wang, Y. R. Liu, X. J. Wu, and D. W. Hu, “Self-paced operation of a wheelchair based on a hybrid brain-computer interface combining motor imagery and P300 potential,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 25, no. 12, pp. 2516–2526, Dec. 2017. doi: 10.1109/TNSRE.2017.2766365
    A. Lopes, J. Rodrigues, J. Perdigao, G. Pires, and U. Nunes, “A new hybrid motion planner: Applied in a brain-actuated robotic wheelchair,” IEEE Robot. Autom. Mag., vol. 23, no. 4, pp. 82–93, Dec. 2016. doi: 10.1109/MRA.2016.2605403
    F. Arrichiello, P. Di Lillo, D. Di Vito, G. Antonelli, and S. Chiaverini, “Assistive robot operated via P300-based brain computer interface, ” in Proc. IEEE Int. Conf. Robotics and Automation, Singapore, 2017, pp. 6032–6037.
    S. L. Sheng, P. P. Song, L. Y. Xie, Z. D. Luo, W. N. Chang, S. R. Jiang, H. Y. Yu, C. Zhu, J. T. C. Tan, and F. Duan, “Design of an SSVEP-based BCI system with visual servo module for a service robot to execute multiple tasks, ” in Proc. IEEE Int. Conf. Robotics and Automation, Singapore, 2017, pp. 2267–2272.
    L. M. Alonso-Valerdi, R. A. Salido-Ruiz, and R. A. Ramirez-Mendoza, “Motor imagery based brain-computer interfaces: An emerging technology to rehabilitate motor deficits,” Neuropsychologia, vol. 79, pp. 354–363, Dec. 2015. doi: 10.1016/j.neuropsychologia.2015.09.012
    S. Bhattacharyya, S. Shimoda, and M. Hayashibe, “A synergetic brain-machine interfacing paradigm for multi-DOF robot control,” IEEE Trans. Syst. Man Cybern.:Syst., vol. 46, no. 7, pp. 957–968, Jul. 2016. doi: 10.1109/TSMC.2016.2560532
    S. Bhattacharyya, A. Konar, and D. N. Tibarewala, “Motor imagery and error related potential induced position control of a robotic arm,” IEEE/CAA J. Autom. Sinica, vol. 4, no. 4, pp. 639–650, Sept. 2017. doi: 10.1109/JAS.2017.7510616
    Y. S. Zhang, P. Xu, T. J. Liu, J. Hu, R. Zhang, and D. Z. Yao, “Multiple frequencies sequential coding for SSVEP-based brain-computer interface,” PLoS One, vol. 7, no. 3, pp. e29519, Mar. 2012. doi: 10.1371/journal.pone.0029519
    R. G. Muller-Putz, R. Scherer, C. Brauneis, and G. Pfurtscheller, “Steady-state visual evoked potential (SSVEP)-based communication: Impact of harmonic frequency components,” J. Neural Eng., vol. 2, no. 4, pp. 123–130, Dec. 2005. doi: 10.1088/1741-2560/2/4/008
    J. R. Wolpaw, N. Birbaumer, D. J. McFarland, G. Pfurtscheller, and T. M. Vaughan, “Brain-computer interfaces for communication and control,” Clin. Neurophysiol., vol. 113, no. 6, pp. 767–791, 2002.
    A. Khasnobish, A. Konar, D. N. Tibarewala, and A. K. Nagar, “Bypassing the natural visual-motor pathway to execute complex movement related tasks using interval type-2 fuzzy sets,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 25, no. 1, pp. 91–105, Jan. 2017. doi: 10.1109/TNSRE.2016.2580580
    E. Donchin, K. M. Spencer, and R. Wijesinghe, “The mental prosthesis: Assessing the speed of a P300-based brain-computer interface,” IEEE Trans. Rehabil. Eng., vol. 8, no. 2, pp. 174–179, Jun. 2000. doi: 10.1109/86.847808
    T. Bhattacharjee, R. Kar, A. Konar, A. Lekova, and A. K. Nagar, “A general type-2 fuzzy set induced single trial P300 detection, ” in Proc. IEEE Int. Conf. Fuzzy Systems, Naples, Italy, 2017, pp. 1–6.
    A. Riccio, L. Simione, F. Schettini, A. Pizzimenti, M. Inghilleri, M. O. Belardinelli, D. Mattia, and F. Cincotti, “Attention and P300-based BCI performance in people with amyotrophic lateral sclerosis,” Front. Hum. Neurosci., vol. 7, pp. 732, Nov. 2013.
    S. Bhattacharyya, A. Konar, and D. N. Tibarewala, “Motor imagery, P300 and error-related EEG-based robot arm movement control for rehabilitation purpose,” Med. Biol. Eng. Comput., vol. 52, no. 12, pp. 1007–1017, Sept. 2014. doi: 10.1007/s11517-014-1204-4
    S. K. Kim, E. A. Kirchner, A. Stefes, and F. Kirchner, “Intrinsic interactive reinforcement learning-using error-related potentials for real world human-robot interaction,” Sci. Rep., vol. 7, no. 1, pp. 17562, Dec. 2017. doi: 10.1038/s41598-017-17682-7
    C. Alain, H. E. McNeely, Y. He, B. K. Christensen, and R. West, “Neurophysiological evidence of error-monitoring deficits in patients with schizophrenia,” Cereb. Cortex, vol. 12, no. 8, pp. 840–846, Aug. 2002. doi: 10.1093/cercor/12.8.840
    J. G. Kerns, J. D. Cohen, A. W. McDonald Ⅲ, M. K. Johnson, V. A. Stenger, H. Aizenstein, and C. S. Carter, “Decreased conflict-and error-related activity in the anterior cingulate cortex in subjects with schizophrenia,” Am. J. Psychiatry, vol. 162, no. 10, pp. 1833–1839, Oct. 2005. doi: 10.1176/appi.ajp.162.10.1833
    F. L. Colino, H. Howse, A. Norton, R. Trska, A. Pluta, S. J. C. Luehr, T. C. Handy, and O. E. Krigolson, “Older adults display diminished error processing and response in a continuous tracking task,” Psychophysiology, vol. 54, no. 11, pp. 1706–1713, Nov. 2017. doi: 10.1111/psyp.12907
    P. E. Pailing and S. J. Segalowitz, “The error-related negativity as a state and trait measure: Motivation, personality, and ERPs in response to errors,” Psychophysiology, vol. 41, no. 1, pp. 84–95, Jan. 2004. doi: 10.1111/1469-8986.00124
    A. Kumar, L. Gao, E. Pirogova, and Q. Fang, “A review of error-related potential-based brain-computer interfaces for motor impaired people,” IEEE Access, vol. 7, pp. 142451–142466, Sept. 2019. doi: 10.1109/ACCESS.2019.2944067
    K. Ogata, Modern Control Engineering. 4th ed. New Jersey, USA: Prentice-Hall, 2002.
    A. C. Davison and D. V. Hinkley, Bootstrap Methods and their Application. Cambridge, UK: Cambridge University Press, 1997.
    F. Lotte, “A tutorial on EEG signal processing techniques for mental state recognition in brain-computer interfaces, ” in Guide to Brain-Computer Music Interfacing, E. R. Miranda and J. Castet, Eds. London, UK: Springer, 2014, pp. 133–161.
    D. J. McFarland, L. M. McCane, S. V. David, and J. R. Wolpaw, “Spatial filter selection for EEG-based communication,” Electroencephalography and Clinical Neurophysiology, vol. 103, no. 3, pp. 386–394, 1997.
    J. Kayser and C. E. Tenke, “Hemifield-dependent N1 and event-related theta/delta oscillations: An unbiased comparison of surface Laplacian and common EEG reference choices,” Int. J. Psychophysiol., vol. 97, no. 3, pp. 258–270, Sept. 2015. doi: 10.1016/j.ijpsycho.2014.12.011
    W. Witkowski, M. Cortese, M. Cempini, J. Mellinger, N. Vitiello, and S. R. Soekadar, “Enhancing brain-machine interface (BMI) control of a hand exoskeleton using electrooculography (EOG),” J. Neuroeng. Rehabil., vol. 11, no. 1, pp. 165, Dec. 2014. doi: 10.1186/1743-0003-11-165
    S. Theodoridis and K. Koutroumbas, Pattern Recognition. Amsterdam, Netherlands: Elsevier, 2003.
    S. Das, A. Abraham, and A. Konar, “Particle swarm optimization and differential evolution algorithms: Technical analysis, applications and hybridization perspectives, ” in Advances of Computational Intelligence in Industrial Systems, Y. Liu, A. Sun, H. T. Loh, W. F. Lu, and E. P. Lim, Eds. Berlin, Heidelberg, Germany: Springer, 2008, pp. 1–38.
    A. Khasnobish, S. Bhattacharyya, A. Konar, D. N. Tibarewala, and A. K. Nagar, “A two-fold classification for composite decision about localized arm movement from EEG by SVM and QDA techniques, ” in Proc. Int. Joint Conf. Neural Networks, San Jose, USA, 2011, pp. 1344–1351.
    B. Blankertz, R. Tomioka, S. Lemm, M. Kawanabe, and K. R. Muller, “Optimizing spatial filters for robust EEG single-trial analysis,” IEEE Signal Process. Mag., vol. 25, no. 1, pp. 41–56, Jan. 2008. doi: 10.1109/MSP.2008.4408441
    F. Lotte and C. T. Guan, “Regularizing common spatial patterns to improve BCI designs: Unified theory and new algorithms,” IEEE Trans. Biomed. Eng., vol. 58, no. 2, pp. 355–362, Feb. 2011. doi: 10.1109/TBME.2010.2082539
    Q. B. Zhao and L. Q. Zhang, “Temporal and spatial features of single-trial EEG for brain-computer interface, ” Comput. Intell. Neurosci., vol. 2007, Article ID: 37695, 2017.
    R. P. N. Rao and R. Scherer, “Statistical pattern recognition and machine learning in brain-computer interfaces, ” in Statistical Signal Processing for Neuroscience and Neurotechnology, K. G. Oweiss, Ed. London, UK: Academic Press, 2010, pp. 335–367.
    K. K. Ang, Z. Y. Chin, C. C. Wang, C. T. Guan, and H. H. Zhang, “Filter bank common spatial pattern algorithm on BCI competition IV datasets 2a and 2b,” Front. Neurosci., vol. 6, pp. 39, Mar. 2012.
    B. Chakraborty, L. Ghosh, and A. Konar, “Designing phase-sensitive common spatial pattern filter to improve brain-computer interfacing,” IEEE Trans. Biomed. Eng., vol. 67, no. 7, pp. 2064–2072, Jul. 2020.
    E. Haselsteiner and G. Pfurtscheller, “Using time-dependent neural networks for EEG classification,” IEEE Trans. Rehabil. Eng., vol. 8, no. 4, pp. 457–463, Dec. 2000. doi: 10.1109/86.895948
    A. Rakshit, S. Ghosh, A. Konar, and M. Pal, “A novel hybrid brain-computer interface for robot arm manipulation using visual evoked potential, ” in Proc. 9th Int. Conf. Advances in Pattern Recognition, Bangalore, India, 2017, pp. 1–6.
    Y. Shamash, “Model reduction using the Routh stability criterion and the Pade approximation technique,” Int. J. Control, vol. 21, no. 3, pp. 475–484, Mar. 1975. doi: 10.1080/00207177508922004
    World Medical Association, “World medical association declaration of Helsinki. Ethical principles for medical research involving human subjects,” Bull. World Health Organ., vol. 79, no. 4, pp. 373–374, 2001.
    L. H. He, D. Hu, M. Wan, Y. Wen, K. M. von Deneen, and M. C. Zhou, “Common Bayesian network for classification of EEG-based multiclass motor imagery BCI,” IEEE Trans. Syst. Man Cybern.:Syst., vol. 46, no. 6, pp. 843–854, Jun. 2016. doi: 10.1109/TSMC.2015.2450680
    G. Pfurtscheller and F. H. Lopes Da Silva, “Event-related EEG/MEG synchronization and desynchronization: Basic principles,” Clin. Neurophysiol., vol. 110, no. 11, pp. 1842–1857, Nov. 1999. doi: 10.1016/S1388-2457(99)00141-8
    A. Schlogl, F. Lee, H. Bischof, and G. Pfurtscheller, “Characterization of four-class motor imagery EEG data for the BCI-competition 2005,” J. Neural Eng., vol. 2, no. 4, pp. L14–L22, Aug. 2005. doi: 10.1088/1741-2560/2/4/L02
    J. T. Cleophas and A. H. Zwinderman, “One-sample continuous data (One-sample t-test, one-sample Wilcoxon signed rank test, 10 patients), ” in SPSS for Starters and 2nd Levelers. Cham, Germany: Springer, 2016, pp. 3–6
    R. H. Riffenburgh, “Tests on ranked data, ” in Statistics in Medicine, 3rd ed. R. H. Riffenburgh, Ed. San Diego, USA: Academic Press, 2012, pp. 221–248.
    X. Han, L. Ke, S. K. Gao, and X. R. Gao, “A novel system of SSVEP-based human-robot coordination,” J. Neural Eng., vol. 16, no. 1, pp. 016006, Feb. 2019. doi: 10.1088/1741-2552/aae1ba
    K. Lee, D. Liu, L. Perroud, R. Chavarriaga, and J. del R. Millan, “A brain-controlled exoskeleton with cascaded event-related desynch-ronization classifiers,” Robot. Auton. Syst., vol. 90, pp. 15–23, Apr. 2017. doi: 10.1016/j.robot.2016.10.005
    K. K. Ayten, M. N. Sahinkaya, and A. Dumlu, “Optimum trajectory generation for redundant/hyper-redundant manipulators,” IFAC-PapersOnLine, vol. 49, no. 21, pp. 493–500, Dec. 2016. doi: 10.1016/j.ifacol.2016.10.651
    Q. Gao, L. X. Dou, A. N. Belkacem, and C. Chen, “Noninvasive electroencephalogram based control of a robotic arm for writing task using hybrid BCI system, ” BioMed Res. Int., vol. 2017, Article ID: 8316485, Jun. 2017.
    H. Zeng, Y. X. Wang, C. C. Wu, A. G. Song, J. Liu, P. Ji, B. G. Xu, L. F. Zhu, H. J. Li, and P. C. Wen, “Closed-loop hybrid gaze brain-machine interface based robotic arm control with augmented reality feedback,” Front. Neurorobot., vol. 11, pp. 60, Oct. 2017. doi: 10.3389/fnbot.2017.00060
    C. Chen, P. Zhou, A. N. Belkacem, L. Lu, R. Xu, X. T. Wang, W. J. Tan, Z. F. Qiao, P. H. Li, Q. Gao, and D. Shin, “Quadcopter robot control based on hybrid brain-computer interface system,” Sens. Mater., vol. 32, no. 3, pp. 991–1004, Mar. 2020.
    C. W. Liu, Y. F. Fu, J. Yang, X. Xiong, H. W. Sun, and Z. T. Yu, “Discrimination of motor imagery patterns by electroencephalogram phase synchronization combined with frequency band energy,” IEEE/CAA J. Autom. Sinica, vol. 4, no. 3, pp. 551–557, Aug. 2017. doi: 10.1109/JAS.2016.7510121
    C. S. Kim, J. W. Sun, D. Liu, Q. S. Wang, and S. G. Paek, “Removal of ocular artifacts using ICA and adaptive filter for motor imagery-based BCI,” IEEE/CAA J. Autom. Sinica, 2017. doi: 10.1109/JAS.2017.7510370


    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(9)  / Tables(16)

    Article Metrics

    Article views (3298) PDF downloads(91) Cited by()


    • A novel scheme of Brain-comanded position-control of a robot arm in a 3-dimensional environment is proposed.
    • The proposed scheme uses SSVEP for random link selection, MI for movement initiation, and P300 to turn back the link on crossing the target positions.
    • Positional overshoot and steady-state error are reduced by an exponential decrease in speed and reversal of turning of the motors as the target position is crossed.
    • The fundamental contribution includes a non-traditional approach to computing transfer function of the position controllerand its parameter setting to attain stability by Root Locus analysis.


    DownLoad:  Full-Size Img  PowerPoint