SCIENTIA SINICA Informationis, Volume 48, Issue 4: 419-432(2018) https://doi.org/10.1360/N112017-00228

## Bayesian method for intent prediction in pervasive computing environments

Xin YI1,3,4, Chun YU1,2,3,4,*, Yuanchun SHI1,2,3,4
• AcceptedJan 29, 2018
• PublishedApr 10, 2018
Share
Rating

### Abstract

This paper describes the principle and examples of predicting users' intent using the Bayesian method. In natural interfaces, users use continuous, undetermined multi-modal data to express their interaction intention rather than using discrete, determined actions. In order to interpret their interaction intend, we can either use “black-box” machine-learning methods or use “white-box” user-behavior-modelling methods. The essence of the latter is to computationally model the users' interaction ability, which is crucial to understanding the users and exploring the computation principle of natural interaction. This paper summarizes the popular intelligent algorithms used in recent HCI researches, discusses the difference between these methods, and illustrates the methods of user modelling and the Bayesian method using some works from our laboratory.

### References

[1] Laput G, Zhang Y, Harrison C. Synthetic sensors: towards general-purpose sensing. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, 2017. 3986--3999. Google Scholar

[2] Yu C, Gu Y Z, Yang Z C, et al. Tap, dwell or gesture?: exploring head-based text entry techniques for HMDs. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, 2017. 4479--4488. Google Scholar

[3] Ilisescu C, Kanaci H A, Romagnoli M, et al. Responsive action-based video synthesis. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, 2017. 6569--6580. Google Scholar

[4] Wu C-J, Houben S, Marquardt N. EagleSense: tracking people and devices in interactive spaces using real-time top-view depth-sensing. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, 2017. 3929--3942. Google Scholar

[5] Vatavu R-D. Improving gesture recognition accuracy on touch screens for users with low vision. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, 2017. 4667--4679. Google Scholar

[6] Taranta II E M, Samiei A, Maghoumi M, et al. Jackknife: a reliable recognizer with few samples and many modalities. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, 2017. 5850--5861. Google Scholar

[7] Schneegass S, Oualil Y, Bulling A. SkullConduct: biometric user identification on eyewear computers using bone conduction through the skull. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, 2016. 1379--1384. Google Scholar

[8] Liu M Y, Nancel M, Vogel D. Gunslinger: subtle arms-down mid-air interaction. In: Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, Charlotte, 2015. 63--71. Google Scholar

[9] Sugano Y, Bulling A. Self-calibrating head-mounted eye trackers using egocentric visual saliency. In: Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, Charlotte, 2015. 363--372. Google Scholar

[10] Huang D, Zhang X Y, Saponas T S, et al. Leveraging dual-observable input for fine-grained thumb interaction using forearm EMG. In: Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, Charlotte, 2015. 523--528. Google Scholar

[11] Sun K, Wang Y T, Yu C, et al. Float: one-handed and touch-free target selection on smartwatches. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, 2017. 692--704. Google Scholar

[12] Gonz$\acute{\rm~~a}$lez R M, Appert C, Bailly G, et al. TouchTokens: guiding touch patterns with passive tokens. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, 2016. 4189--4202. Google Scholar

[13] Gil H, Lee D Y, Im S, et al. TriTap: identifying finger touches on smartwatches. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, 2017. 3879--3890. Google Scholar

[14] Hanafi M F, Abouzied A, Chiticariu L, et al. SEER: auto-generating information extraction rules from user-specified examples. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, 2017. 6672--6682. Google Scholar

[15] Sridhar S, Markussen A, Oulasvirta A, et al. WatchSense: on- and above-skin input sensing through a wearable depth sensor. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, 2017. 3891--3902. Google Scholar

[16] Noor M F M, Rogers S, Williamson J. Detecting swipe errors on touchscreens using grip modulation. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, 2016. 1909--1920. Google Scholar

[17] Chan L W, Chen Y-L, Hsieh C-H, et al. CyclopsRing: enabling whole-hand and context-aware interactions through a fisheye ring. In: Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, Charlotte, 2015. 549--556. Google Scholar

[18] Huang M X, Kwok T C K, Ngai G, et al. Building a personalized, auto-calibrating eye tracker from user interactions. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, 2016. 5169--5179. Google Scholar

[19] Krupka E, Karmon K, Bloom N, et al. Toward realistic hands gesture interface: keeping it simple for developers and machines. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, 2017. 1887--1898. Google Scholar

[20] Laput G, Xiao R, Harrison C. ViBand: high-fidelity bio-acoustic sensing using commodity smartwatch accelerometers. In: Proceedings of the 29th Annual Symposium on User Interface Software and Technology, Tokyo, 2016. 321--333. Google Scholar

[21] Chen X, Li Y. Bootstrapping user-defined body tapping recognition with offline-learned probabilistic representation. In: Proceedings of the 29th Annual Symposium on User Interface Software and Technology, Tokyo, 2016. 359--364. Google Scholar

[22] Zhou J H, Zhang Y, Laput G, et al. AuraSense: enabling expressive around-smartwatch interactions with electric field sensing. In: Proceedings of the 29th Annual Symposium on User Interface Software and Technology, Tokyo, 2016. 81--86. Google Scholar

[23] Zhang Y, Xiao R, Harrison C. Advancing hand gesture recognition with high resolution electrical impedance tomography. In: Proceedings of the 29th Annual Symposium on User Interface Software and Technology, Tokyo, 2016. 843--850. Google Scholar

[24] Zhang Y, Harrison C. Tomo: wearable, low-cost electrical impedance tomography for hand gesture recognition. In: Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, Charlotte, 2015. 167--173. Google Scholar

[25] Lin J-W, Wang C, Huang Y Y, et al. BackHand: sensing hand gestures via back of the hand. In: Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, Charlotte, 2015. 557--564. Google Scholar

[26] Li H C, Brockmeyer E, Carter E J, et al. PaperID: a technique for drawing functional battery-free wireless interfaces on paper. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, 2016. 5885--5896. Google Scholar

[27] Holz C, Knaust M. Biometric touch sensing: seamlessly augmenting each touch with continuous authentication. In: Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, Charlotte, 2015. 303--312. Google Scholar

[28] Li H C, Zhang P J, Moubayed S A, et al. ID-Match: a hybrid computer vision and RFID system for recognizing individuals in groups. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, 2016. 4933--4944. Google Scholar

[29] Yoon D, Hinckley K, Benko H, et al. Sensing tablet grasp + micro-mobility for active reading. In: Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, Charlotte, 2015. 477--487. Google Scholar

[30] Hagiya T, Horiuchi T, Yazaki T. Typing tutor: individualized tutoring in text entry for older adults based on input stumble detection. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, 2016. 733--744. Google Scholar

[31] Zhang Y, Zhou J H, Laput G, et al. SkinTrack: using the body as an electrical waveguide for continuous finger tracking on the skin. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, 2016. 1491--1503. Google Scholar

[32] Leong J, Parzer P, Perteneder F, et al. proCover: sensory augmentation of prosthetic limbs using smart textile covers. In: Proceedings of the 29th Annual Symposium on User Interface Software and Technology, Tokyo, 2016. 335--346. Google Scholar

[33] Alaoui S F, Françoise J, Schiphorst T, et al. Seeing, sensing and recognizing laban movement qualities. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, 2017. 4009--4020. Google Scholar

[34] Qian K, Wu C S, Zhou Z M, et al. Inferring motion direction using commodity wi-fi for interactive exergames. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, 2017. 1961--1972. Google Scholar

[35] Fridman L, Toyoda H, Seaman S, et al. What can be predicted from six seconds of driver glances? In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, 2017. 2805--2813. Google Scholar

[36] Buschek D, Alt F. ProbUI: generalising touch target representations to enable declarative gesture definition for probabilistic GUIs. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, 2017. 4640--4653. Google Scholar

[37] Banovic N, Buzali T, Chevalier F, et al. Modeling and understanding human routine behavior. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, 2016. 248--260. Google Scholar

[38] Huang M X, Li J J, Ngai G, et al. ScreenGlint: practical, in-situ gaze estimation on smartphones. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, 2017. 2546--2557. Google Scholar

[39] Evans A C, Davis K, Fogarty J, et al. Group touch: distinguishing tabletop users in group settings via statistical modeling of touch pairs. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, 2017. 35--47. Google Scholar

[40] McIntosh J, Marzo A, Fraser M, et al. EchoFlex: hand gesture recognition using ultrasound imaging. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, 2017. 1923--1934. Google Scholar

[41] Sugano Y, Zhang X C, Bulling A. AggreGaze: collective estimation of audience attention on public displays. In: Proceedings of the 29th Annual Symposium on User Interface Software and Technology, Tokyo, 2016. 821--831. Google Scholar

[42] Wang S W, Song J, Lien J, et al. Interacting with soli: exploring fine-grained dynamic gesture recognition in the radio-frequency spectrum. In: Proceedings of the 29th Annual Symposium on User Interface Software and Technology, Tokyo, 2016. 851--860. Google Scholar

[43] Vaccaro K, Shivakumar S, Ding Z Q, et al. The elements of fashion style. In: Proceedings of the 29th Annual Symposium on User Interface Software and Technology, Tokyo, 2016. 777--785. Google Scholar

[44] Liu C, Clark G D, Lindqvist J. Where usability and security go hand-in-hand: robust gesture-based authentication for mobile systems. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, 2017. 374--386. Google Scholar

[45] Kay M, Nelson G L, Hekler E B. Researcher-centered design of statistics: why Bayesian statistics better fit the culture and incentives of HCI. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, 2016. 4521--4532. Google Scholar

[46] Yi X, Yu C, Zhang M R, et al. ATK: enabling ten-finger freehand typing in air based on 3D hand tracking data. In: Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, Charlotte, 2015. 539--548. Google Scholar

[47] Yu C, Sun K, Zhong M Y, et al. One-dimensional handwriting: inputting letters and words on smart glasses. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, 2016. 71--82. Google Scholar

[48] Kangasrääsiö A, Athukorala K, Howes A, et al. Inferring cognitive models from data using approximate Bayesian computation. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, 2017. 1295--1306. Google Scholar

[49] Finnegan D J, O'Neill E, Proulx M J. Compensating for distance compression in audiovisual virtual environments using incongruence. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, 2016. 200--212. Google Scholar

• Figure 1

Motion interaction interface

• Figure 2

Structure of a basic motion

• Figure 3

(Color online) Illustration of a touch model

• Figure 4

(Color online) When tapping the middle finger, the index finger moves along it due to the correlation movement between fingers

• Figure 5

(Color online) The amplitude ratio (AR) of correlated finger movement

• Figure 6

(Color online) Augmented Bayes model

Citations

• #### 0

Altmetric

Copyright 2019 Science China Press Co., Ltd. 《中国科学》杂志社有限责任公司 版权所有