ISSN 2071-8594

Russian academy of sciences

Editor-in-Chief

Gennady Osipov

V.I. Vasilyev, A.E. Sulavko, R.V. Borisov, S.S. Zhumazhanova Recognition of psychophysiological state of the user based on a hidden monitoring of computer systems

Abstract.

It is established that the signs of the voice, the keyboard handwriting and the nature of the work of the subject with a computer mouse contain the following information about psychophysiological state of the operator: normal, fatigue, intoxication, excited, relaxed (sleepy). Signs of voice are best in order to recognize fatigue or sleepy speaker. Keyboard handwriting in addition to these states has features that characterize the normal state of the operator. Some of the features of the work with a computer mouse contain information about the states of intoxication and sleepy. An experiment of the recognition of state was based on Bayes strategies and neural network approach, the best result: 5.9% errors of determination of the states for the monitoring of the subject is not more than 100 seconds.

Keywords:

keyboard handwriting, biometric characteristic, voice signs, identification of psychophysiological conditions, the option of working with a computer mouse.

PP. 21-37.

REFERENCES

1. Mashin V.A. Psihicheskaja nagruzka, psihicheskoe naprjazhenie i funkcional'noe sostojanie operatorov sistem upravlenija. Voprosy psihologii, 2007. № 6. P. 86-96.
2. Marcus J.H., Rosekind M.R. Fatigue in transportation: NTSB investigations and safety recommendations. Injury Prevention, 2016. doi: 10.1136/injuryprev-2015-041791.
3. Luzhnikov E.A. Klinicheskaja toksikologija. M.:Medicina, 1994. 256 p.
4. The Global State of Information Security® Survey 2016 [Elektronnyj resurs] // PricewaterhouseCoopers. URL: http://www.pwc.com/gx/en/issues/cyber-security/information-security-survey/download.html (data obrashhenija: 27.06.2016).
5. Bogomolov A.V., Gridin L.A., Kukushkin YU.A., Ushakov I.B. Diagnostika sostojanija cheloveka: matematicheskie podhody. M.: Medicina, 2003. 464 p.
6. Il'in E.P. Psihofiziologija sostoyanij cheloveka.SPb.: Piter, 2005.412 s.
7. Cacioppo J.T., Tassinary L.G., Berntson G. Handbook of Psychophysiology. 3rd Edition. Cambridge university press, 2007. P. 433-452.
8. Bayevsky R.M., Ivanov G.G., Chireykin L.V. HRV Analysis under the usage of different electrocardiography systems (Methodical recommendations) .Committee of Clinic Diagnostic Apparatus and the Committee of New Medical Techniques of Ministry of Health of Russia, 2002. Vol. 4. P. 2-67.
9. Malik M., Bigger J. T., Camm A. J. Heart rate variability: Standards of measurement, physiological interpretation, and clinical use // European Heart Journal, 1996. Vol. 17 (3). P. 354-381.
10. Mashin V.A., Mashina M.N. Klassifikacija funkcional'nyh sostojanij i diagnostika psihojemocional'noj ustojchivosti na osnove faktornoj struktury pokazatelej variabel'nosti serdechnogo ritma. Rossijskij fiziologicheskij zhurnal im. I.M. Sechenova, 2004. T. 90(12). P. 1508-1521.
11. Shi Ping, Vicente Azorin P., Echiadis A. Non-contact Reflection Photoplethysmography Towards Effective Human Physiological Monitoring. Journal of Medical and Biological Engineering, 2009. Vol. 30(3). P. 161-167.
12. Habib Tabatabai, David E. Oliver, John W. Rohrbaugh, Christopher Popadopoulos. Novel Applications of Laser Doppler Vibration Measurements to Medical Imaging. Sensing ant Imaging: An International Journal, 2013. Vol. 14(1-2). P. 13-28.
13. Jing B, Li H. A novel thermal measurement for heart rate. Journal of Computers, 2013. Vol. 8(9). P. 2163-2166.
14. Zhao F. Li M., Qian Y., Tsien J.Z. Remote Measurements of Heart and Respiration Rates for Telemedicine. PLoSONE, 2013. Vol. 8(10).
15. Sun Y., Yu X., Berilla J. An innovative non-invasive ECG sensor and comparison study with clinic system. Proceedings of 39th IEEE Annual Northeast Bioengineering Conference (NEBEC), 2013. P. 163-164.
16. Jain U., Tan B., Li Q. Concealed knowledge identification using facial thermal imaging. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2012. P. 1677-1680.
17. Carl B. Cross ; Julie A. Skipper, Petkie D. Thermal imaging to detect physiological indicators of stress in humans// Proceedings of SPIE. Vol. 8705. doi:10.1117/12.2018107.
18. Lozhnikov P.S., Sulavko A.E., Tolkacheva E.V., Zhumazhanova S.S. Raspoznavanie voditelej i ih funkcional'nyh sostojanij po obychnomu i teplovomu izobrazhenijam lica . Trudy nauchno-tekhnicheskoj konferencii klastera penzenskih predprijatij, obespechivajushhih bezopasnost' informacionnyh tekhnologij. Penza, 2016 T. 10. P. 63-65.
19. Epifancev B.N. Skrytaja identifikacija psihofiziologicheskogo sostojanija cheloveka-operatora v processe professional'noj dejatel'nosti: monografija. Omsk: Izd-vo SibADI, 2013. 198 p.
20. Vasil'ev V.I., Lozhnikov P.S., Sulavko A.E., Eremenko A.V. Tekhnologii skrytoj biometricheskoj identifikacii pol'zovatelej komp'yuternyh sistem (Obzor). Voprosy zashchity informacii. Moskva: Izd-vo: FGUP «VIMI», 2015. №3. S. 37-47.
21. Lozhnikov P.S., Sulavko A.E., Samotuga A.E. Personal Identification and the Assessment of the Psychophysiological State While Writing a Signature. Information, 2015. № 6. P. 454-466.
22. Epifantsev B. N., Lozhnikov P. S., Sulavko A. E., Zhumazhanova S. S.. Identification Potential of Online Handwritten Signature Verification. Optoelectronics, Instrumentation and Data Processing, 2016. № 3(52). P. 238-244. DOI: 10.3103/S8756699016030043.
23. Sulavko A.E., Eremenko A.V., Levitskaja E.A., Samotuga A.E. Identifikacija psihofiziologicheskih sostoyanij podpisantov po osobennostjam vosproizvedenija avtografa. Informacionno-izmeritel'nye i upravljajushhie sistemy, 2017. №1. P. 40-48.
24. Sulavko A.E., Eremenko A.V., Levitskaja E.A. Razgranichenie dostupa k informacii na osnove skrytogo monitoringa dejstvij pol'zovatelej v informacionnyh sistemah: portret nelojal'nogo sotrudnika. Izvestija Transiba. Omsk: Izd-vo OmGUPS, 2015. № 1(21). P. 80-89.
25. Ayadi M. El. Survey on speech emotion recognition: Features, classification schemes, and databases. Pattern Recognition, 2011. Vol. 44(3). P. 572-587.
26. Davydov A.G., Kiselyov V.V., Kochetkov D.S. Klassifikacija ehmocional'nogo sostojanija diktora po golosu: Problemy i reshenija // Trudy mezhdunarodnoj konferencii «Dialog 2011». M.: RGGU, 2011. P. 178-185.
27. Tkachenja A.V., Davydov A.G., Kiselyov V.V., Hitrov M.V. Klassifikacija ehmocional'nogo sostojanija diktora s ispol'zovaniem metoda opornyh vektorov i kriterija Dzhini. Izv. VUZov. Priborostroenie, 2013. T. 56(2). P. 61-66.
28. Orden K.F., Jung Tzyy-Ping, Makeig S. Combined eye activity measures accurately estimate changes in sustained visual task performance. Biological Psychology, 2000. № 52. P. 221–240.
29. Zaharchenko D.V. Dorohov V.B. Izmenenie otdel'nyh parametrov zritel'no-motornyh reakcij pod dejstviem alkogolja. Eksperimental'naja psihologija, 2012. № 2. P. 5-21.
30. Knjazev B.A., Gapanjuk YU.E. Raspoznavanie anomal'nogo povedenija cheloveka po ego ehmocional'nomu sostojaniju i urovnju naprjazhennosti s ispol'zovaniem ehkspertnyh pravil. Inzhenernyj vestnik, 2013. № 3. P. 509-523.
31. Mascord, D.J. & Heath, R.A. Behavioral and physiological indices of fatigue in a visual tracking task. Journal of Safety Research, 1992. Vol. 23. P. 19-25.
32. Borisov R.V., Zverev D.N., Sulavko A.E., Pisarenko V.Ju. Ocenka identifikacionnyh vozmozhnostej osobennostej raboty pol'zovatelja s komp'juternoj mysh'ju. Vestnik Sibirskoj gosudarstvennoj avtomobil'no-dorozhnoj akademii. Omsk: Izd-vo SibADi, 2015. № 5(45). P. 106-113.
33. Sulavko A.E., Eremenko A.V., Borisov R.V. Generacija kriptograficheskih kljuchej na osnove golosovyh soobshhenij. Prikladnaja informatika. Moskva: Izd-vo NOU VPO «MFPU «Sinergija», 2016. № 5. P. 76-89.
34. Raskin D. Interfejs: novye napravlenija v proektirovanii komp'juternyh sistem. SPb: Simvol-pljus, 2010. 272 p.
35. Vasil'ev V.I., Lozhnikov P.S., Sulavko A.E., Zhumazhanova S.S. Ocenka identifikacionnyh vozmozhnostej biometricheskih priznakov ot standartnogo periferijnogo oborudovanija. Voprosy zashchity informacii. Moskva: Izd-vo FGUP «VIMI», 2016. №1. S. 12-20.
36. Vasilyev V.I., Sulavko A.E., Eremenko A.V., Zhumazhanova S.S. Identification potential capacity of typical hardware for the purpose of hidden recognition of computer network users. Proceedings of X International IEEE Scientific and Technical Conference "Dynamics of Systems, Mechanisms and Machines" (Dynamics), 15-17 November 2016. P. 1-5. DOI: 10.1109/Dynamics.2016.7819106.
37. Lozhnikov P.S., Ivanov A.I., Kachajkin E.I., Sulavko A.E. Biometricheskaja identifikacija rukopisnyh obrazov s ispol'zovaniem korreljacionnogo analoga pravila Bajesa. Voprosy zashhity informacii. Moskva: Izd-vo FGUP «VIMI», 2015. №3. P. 48-54.
38. Ivanov A.I., Lozhnikov P.S., Serikova Yu.I. Reducing the Size of a Sample Sufficient for Learning Due to the Symmetrization of Correlation Relationships Between Biometric Data. Cybernetics and Systems Analysis, 2016. Vol. 52(3). P. 379-385.
39. Sulavko A.E., Eremenko A.V., Zhumazhanova S.S., Buraja E.V. Generacija klyuchevyh posledovatel'nostej i verifikacija sub"ektov na osnove dvumernogo izobrazhenija lica. Avtomatizacija processov upravlenija, 2017. № 1. P. 58-66.