logo

SCIENTIA SINICA Informationis, Volume 48, Issue 4: 390-405(2018) https://doi.org/10.1360/N112017-00227

Tangible user interface: origins, development, and future trends

More info
  • ReceivedNov 6, 2017
  • AcceptedFeb 7, 2018
  • PublishedApr 9, 2018

Abstract

Tangible user interface (TUI) is a new paradigm in the field of human-computer interaction. By coupling with physical objects and digital information, a TUI allows a user to perform natural actions such as grasping, moving, and assembling to manipulate digital information. With the latest approaches by integrating actuators, a TUI can also move actively or change shape by itself, thus enabling physical feedback. A TUI can be used in several application domains including information visualization, daily work, edutainment, and supporting creativity. This paper introduces the origins and development of tangible user interfaces, and discusses the theoretical frameworks, implementation technologies, application domains, and future research trends for TUIs.


Funded by

国家重点研发计划(2016YFB1001402)

国家自然科学基金重点项目(61232013)

国家自然科学基金青年项目(61402250)


Acknowledgment

戴国忠研究员为促成本文成稿做了许多工作, 特此致谢. 姚远, 麦龙辉完成了本文诸多图片绘制工作. 路奇为本文排版做了大量工作. 在此一并致谢.


References

[1] Frazer J, Frazer P. Intelligent physical three-dimensional modelling systems. Comput Graph, 1980, 80: 359--370. Google Scholar

[2] Perlman R. Using computer technology to provide a creative learning environment for preschool children. MIT Logo Memo, 1976. Google Scholar

[3] Abrams R. Adventures in tangible computing: the work of interaction designer “Durrell bishop" in context. South Asia Res, 1999, 28: 23--48. Google Scholar

[4] Fitzmaurice G W, Ishii H, Buxton W A S. Bricks: laying the foundations for graspable user interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Denver, 1995. 442--449. Google Scholar

[5] Ishii H, Ullmer B. Tangible bits: towards seamless interfaces between people, bits and atoms. In: Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, Atlanta, 1997. 234--241. Google Scholar

[6] Ullmer B, Ishii H. The metaDESK: models and prototypes for tangible user interface. In: Proceedings of the 10th Annual ACM Symposium on User Interface Software and Technology, Banff, 1997. 223--232. Google Scholar

[7] Ishii H, Ratti C, Piper B. Bringing Clay and Sand into Digital Design - Continuous Tangible user Interfaces. BT Tech J, 2004, 22: 287-299 CrossRef Google Scholar

[8] Weiss M, Jennings R, Khoshabeh R, et al. SLAP widgets: bridging the gap between virtual and physical controls on tabletops. In: Proceedings of CHI'09 Extended Abstracts on Human Factors in Computing Systems, Boston, 2009. 481--490. Google Scholar

[9] Pangaro G, Maynes-Aminzade D, Ishii H. The actuated workbench: computer-controlled actuation in tabletop tangible interfaces. ACM Trans Graph, 2003, 22: 181--190. Google Scholar

[10] Pedersen E W, Hornbæ k K. Tangible bots: interaction with active tangibles in tabletop interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Vancouver, 2011. 2975--2984. Google Scholar

[11] Le Goc M, Kim L H, Parsaei A, et al. Zooids: building blocks for swarm user interfaces. In: Proceedings of the 29th Annual Symposium on User Interface Software and Technology, Tokyo, 2016. 97--109. Google Scholar

[12] Raffle H S, Parkes A J, Ishii H. Topobo: a constructive assembly system with kinetic memory. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Vienna, 2004. 647--654. Google Scholar

[13] Baudisch P, Becker T, Rudeck F. Lumino: tangible building blocks based on glass fiber bundles. In: Proceedings of ACM SIGGRAPH 2010 Emerging Technologies, Los Angeles, 2010. Google Scholar

[14] Wang M, Mi H, Xu Y Q. TwistBlocks: non-rigid interface for 3D-aware design and actuation. In: Proceedings of China Human Computer Interaction Conference, Shanghai, 2016. Google Scholar

[15] Holman D, Vertegaal R. Organic user interfaces: designing computers in any way, shape, or form. Commun ACM Org User Interface, 2008, 51: 48--55. Google Scholar

[16] Minuto A, Vyas D, Poelman W, et al. Smart material interfaces: a vision. In: Proceedings of International Conference on Intelligent Technologies for Interactive Entertainment, Genoa, 2011. 57--62. Google Scholar

[17] Yao L N, Niiyama R, Ou J F, et al. PneUI: pneumatically actuated soft composite materials for shape changing interfaces. In: Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, St. Andrews, 2013. 13--22. Google Scholar

[18] Follmer S, Leithinger D, Olwal A, et al. inFORM: dynamic physical affordances and constraints through shape and object actuation. In: Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, St. Andrews, 2013. 417--426. Google Scholar

[19] Lu Q Y, Mao C P, Wang L Y, et al. LIME: liquid metal interfaces for non-rigid interaction. In: Proceedings of the 29th Annual Symposium on User Interface Software and Technology, Tokyo, 2016. 449--452. Google Scholar

[20] Ullmer B, Ishii H. Emerging frameworks for tangible user interfaces. IBM Syst J, 2000, 39: 915-931 CrossRef Google Scholar

[21] Mi H. Actuated tangible user interface: an extensible method for tabletop interaction. Dissertation for Ph.D. Degree. Tokyo: University of Tokyo, 2011. Google Scholar

[22] Ishii H, Lakatos D, Bonanni L, et al. Radical atoms: beyond tangible bits, toward transformable materials. Interactions, 2012, 19: 38--51. Google Scholar

[23] Ullmer B, Ishii H, Jacob R J K. Token+constraint systems for tangible interaction with digital information. ACM Trans Comput-Hum Interact, 2005, 12: 81-118 CrossRef Google Scholar

[24] Fishkin K. A taxonomy for and analysis of tangible interfaces. Pers Ubiquit Comput, 2004, 8: 347--358. Google Scholar

[25] Jansen Y, Karrer T, Borchers J. MudPad: tactile feedback and haptic texture overlay for touch surfaces. In: Proceedings of ACM International Conference on Interactive Tabletops and Surfaces, Saarbrücken, 2010. 4351--4356. Google Scholar

[26] Yao L N, Ou J F, Cheng C Y, et al. BioLogic: natto cells as nanoactuators for shape changing interfaces. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, Seoul, 2015. 1--10. Google Scholar

[27] Probst K, Seifried T, Haller M, et al. Move-it: interactive sticky notes actuated by shape memory alloys. In: Proceedings of CHI'11 Extended Abstracts on Human Factors in Computing Systems, Vancouver, 2011. 1393--1398. Google Scholar

[28] Ogata M, Fukumoto M. FluxPaper: reinventing paper with dynamic actuation powered by magnetic flux. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, Seoul, 2015. 29--38. Google Scholar

[29] Ono M, Shizuki B, Tanaka J. Touch & activate: adding interactivity to existing objects using active acoustic sensing. In: Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, St. Andrews, 2013. 31--40. Google Scholar

[30] Li H, Ye C, Sample A P. IDSense: a human object interaction detection system based on passive UHF RFID. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, Seoul, 2015. 2555--2564. Google Scholar

[31] Ou J F, Yao L N, Tauber D, et al. jamSheets: thin interfaces with tunable stiffness enabled by layer jamming. In: Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction, Munich, 2014. 65--72. Google Scholar

[32] Jiao Y, Gong J, Xu Y Q. Graille: design research of graphical tactile display for the visually impaired (in Chinese). Art Des, 2016, 01: 94--96. Google Scholar

[33] O'Malley C, Fraser D S. Literature review in learning with tangible technologies. NESTA Futurelab Report 12, 2004. https://www.nfer.ac.uk/publications/FUTL69/FUTL69.pdf. Google Scholar

[34] Antle A N. The CTI framework: informing the design of tangible systems for children. In: Proceedings of the 1st International Conference on Tangible and Embedded Interaction, Baton Rouge, 2007. 195--202. Google Scholar

[35] Kaltenbrunner M, Bencina R. reacTIVision: a computer-vision framework for table-based tangible interaction. In: Proceedings of the 1st International Conference on Tangible and Embedded Interaction, Baton Rouge, 2007. 69--74. Google Scholar

[36] Wang M, Mi H, Liu Y, et al. PolyHinge: shape changing TUI on tabletop. In: Proceedings of the 8th Conference on Interfaces and Human Computer Interaction, Lisbon, 2017. Google Scholar

[37] Perelman G, Serrano M, Raynal M, et al. The roly-poly mouse: designing a rolling input device unifying 2D and 3D interaction. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, Seoul, 2015. 327--336. Google Scholar

[38] Nakagaki K, Follmer S, Ishii H. LineFORM: actuated curve interfaces for display, interaction, and constraint. In: Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, Charlotte, 2015. 333--339. Google Scholar

[39] Shaer O, Horn M S, Jacob R J K. Tangible user interface laboratory: Teaching tangible interaction design in practice. AIEDAM, 2009, 23: 251-261 CrossRef Google Scholar

  • Figure 1

    (a) A Roman abacus and (b) a Chinese abacus

  • Figure 2

    (a) Frazer's 3D manipulating system and (b) Bishop's marble answering machine

  • Figure 3

    MCRit model

  • Figure 4

    ATUI model

  • Figure 5

    From GUI to Radical atoms

  • Figure 6

    Classifications of passive TUIs

  • Figure 7

    Classifications of new TUIs

  • Table 1   Tangible user interface: origins and development
    特征 代表性工作 特点 发表年份
    早期先驱工作 Slot machine [1] 使用物理卡片进行编程 1976
    1970s–1997 早期萌芽 Marble answering machine [2] 使用物理实体指代信息 1995
    Graspable interface [3] 可抓握界面 1995
    metaDesk [4] 桌面上的实物控件操作 1997
    无源 SandScape [5] 柔性任意塑形的信息耦合呈现 2002
    SLAP widget 桌面上较完备的实物操作组件 2009
    Tangible bits Topobo [6] 拼插结构与动作记忆 2004
    组装 Lumino 光纤束结构实现无源多模块组装 2010
    1997– TwistBlock [7] 传感器网络结构实现复杂组装 2017
    Actuated workbench [8] 电磁阵列实现多物体动态反馈 2002
    自驱动 Tangbile bots [9] 小机器人实现复杂动态反馈 2011
    zooids [10] 实现动态反馈的微型机器人平台 2017
    PneUI [11] 气动控制形状改变 2013
    Radical atoms 物理属性 inForm [12] 垂直运动阵列实现复杂表面形态变化 2013
    bioLogic [13] 生物材料实现形状改变 2015
    2012– 形状改变 LIME [14] 液态材料实现物理特征及形状改变 2016
    Programmable droplets$^{1)}$ 水滴受控运动及变形 2018

    1) http://tangible.media.mit.edu/project/programmable-droplets/.

Copyright 2019 Science China Press Co., Ltd. 《中国科学》杂志社有限责任公司 版权所有

京ICP备18024590号-1