Synlab: Papers

RENATI is an acronym for recontextualizing narratives for tangible interfaces. It serves as an umbrella term for our art/research experiments within a hybrid environment that uses oral narratives, and non-generative and immersive art with sensing technologies to create tangible narratives. In this paper we introduce our first prototype, which uses a custom-built mannequin to allow viewers to engage with a multi-viewpoint story titled Flying Over Purgatory.

The International Conference Tangible and Embedded Interaction (TEI) is the first conference with "tangible" in its title, motivated by the field’s growth over the past decade. One of this research area's most fascinating features is the range of perspectives and disciplines, including HCI, the arts, design, technology, and architecture. TEI invites a multi- and interdisciplinary community to submit contributions from their specific viewpoints and approaches. It brings together artists who employ tangible media with computer engineers developing toolkits, physical appliance designers with researchers conducting user studies, and researchers exploring novel interface technologies with interaction designers who develop mixed-media systems.

WiiArts is an experimental video, audio and image processing art project that makes use of pre-existing sensing technologies provided by Nintendo WiiRemotes and a Sensor Bar. Currently, most WiiRemote-based physical interactions have been designed to mimic the gesture of body movement in sports and other action-based games. These Wii games are generally competitive in nature, and players interact by responding to predefined interaction rules in either a single-user or multi-user mode. Making use of the WiiRemote as a pre-existing tangible and embedded interface, we explore applications that can engage participants in active and expressive art creation in a collaborative manner. In this paper, we describe several prototype applications based on this concept: Illumination (draWiing), Beneath (Waldo), WiiBand, Time Ripples. In these applications, three interactors can work together to compose both images and sounds.

In this paper we describe the concept, design, and implementation of Murmur, an interactive kinetic display made of one hundred computer CPU fans. Murmur responds to sound input from its environment via embedded microphones to produce patterns on a reactive surface. The reactive surface consists of hinged paper pieces situated in front of each fan. When activated by sonic elements in the environment, including sounds intentionally generated by an interactor, Murmur responds by turning on and off its fans in a sequence. The wind pressure generated by the movement of the fans stimulates the surface, forcing the paper up and out to create a variety of dynamic patterns. Each pattern represents characteristics of the sonic environment. We also analyze the feedback received from the interactors and discuss the possible ways of making the interaction more immersive.

The Graphical User Interface (GUI) has created an efficient work environment for many applications. However, when users are confined by keyboards and mice, they lose the ability to interact with the virtual world using habits from the real world. Our research examine how emerging modes of authorship, such as wikis, can be used to generate new possibilities for bringing atoms and bits together for digital annotation. Our goal is to combine the everyday habits in reading books with emerging digital possibilities. In this paper, we present a prototype system called WikiTUI, which brings digital media to physical paper books. This system allows readers to access the digital world through fingertip interactions on books, and enables them to share information with other readers using wiki technology. WikiTUI not only bridges the gap between the digital and the physical worlds, but also facilitates multiple contributions to a reference base spanning across these worlds. We present user evaluations of the WikiTUI prototype and discuss research implications.

The workshop aims to bring together researchers and practitioners working on subjects related to digital games with tangible interaction. We would like to involve participants with backgrounds in academia as well as industry, from diverse fields such as HCI, computer science, edutainment, interaction design and game design. Some of the topics we plan to address during the workshop are: different tangible interaction styles, designing for specific game types, and the advantages and disadvantages of different sensing and object tracking technologies.

In this paper, we describe the design and implementation of Teaching Table, an interactive tabletop audio-visual device aimed at enhancing the learning experience for prekindergarten children by involving them in physical activities. Using electromagnetic sensing technology, the table can track tagged objects placed on its surface, accurately identifying their type and location while providing a coincident visual display and audio feedback. Teaching activities that are aimed at developing early math skills have been created for the table in alignment with standard curriculum guidelines for pre-K schools. Additionally, we include software based assessment tools for mentors/teachers to easily track an individual child's progress during the process of interacting with the table.

In screen-based experiences, the screen itself can become the physical device used for interaction. The "move-ability" of the screen affords interactivity between the screen artifact and the viewer, and between the virtual and physical spaces. We have created a movable screen interface, called the tilting table, which provides a display surface via overhead projection. This interface invites user interaction through the action of tilting the tabletop. The degree of tilt affects the displayed visuals and audio output. This simple interaction makes users feel they have a closer connection to the virtual imagery, and the screen thus blurs the boundary between them. In this paper, we introduce four applications that have been implemented for the tilting table. Two use an artistic approach to create expressive and entertaining media interactions, while two have been developed for navigating large information spaces. Each application is described and user feedback is discussed.

In screen-based experiences, the screen itself can become the physical device used for interaction. The "move-ability" of the screen affords interactivity between the screen artifact and the viewer and between the virtual and physical spaces. As a movable screen interface, we have created the spinning screen, a two-sided monitor on a revolving base. This interface invites user interaction through the action of spinning the screen to manipulate media content. By allowing viewers to grasp the interplay between visibility and invisibility, the spinning screen can enrich user experiences of virtual imagery through direct tangible interaction in both playful and practical contexts. We describe an art piece and an application for 3D visualization and modeling.

The TViews Table Role-Playing Game (TTRPG) project explores the creation of digital tabletop role-playing games on the TViews table, in order to bridge the separate worlds of traditional role-playing games with the growing area of massively multiplayer online role-playing games. The TViews table is an interactive tabletop media platform that can track the location of multiple tagged objects in realtime as they are moved around its surface, providing a simultaneous and coincident graphical display. In this paper we present the implementation of the first version of TTRPG, with a content set based on the traditional Dungeons & Dragons rule-set. We also discuss the results of a user study that used TTRPG to explore the possible social context of digital tabletop role-playing games.

Emergent game formats, such as machinima, that use game worlds as expressive 3D performance spaces have new expressive powers with an increase of the quality of their underlying graphic and animation systems. Nevertheless, they still lack intuitive control mechanisms. Set direction and acting are limited by tools that were designed to create and play video games rather than produce expressive performance pieces. These tools do a poor job of capturing the performative expression that characterizes more mature media such as film. Tangible interfaces can help open up the game systems for more intuitive character control needed for a greater level of expression in the digital real-time world. The TUI3D project (Tangible User Interfaces for Real-Time 3D) addresses production and performative challenges involved in creating machinima through the development of tangible interfaces for controlling 3D virtual actors and environments in real-time. In this paper, we present the design and implementation of a tangible puppet prototype for virtual character control in the Unreal game engine and discuss initial user feedback.

The past several years of computer interaction research have shown an increasing interest in tabletops for shared user interactions through touch or tangible objects. Digital media tables offer the potential to expand our digital interactions into casual social settings that are not appropriate for desktop platforms, such as home living rooms. We have developed a tangible media table called TViews, which provides an extensible architecture to enable multiuser interactions with a range of media applications and content via tagged tangible objects. The TViews object positioning utility functions on the surface of an embedded display and enables real-time tracking of a virtually unlimited set of uniquely identified wireless objects that can be used on the surface of any similar table. These objects can be physically customized in order to suit particular applications, and can provide additional functionality through external input and output elements on the objects themselves. In this paper, we present a first field trial of TViews to gain some initial insight into how such a device could be adopted in a real-world home.

With the rapid and continuous growth of digital media entertainment, we need new kinds of platforms and technologies that can better bridge our physical and digital worlds. Rather than confining our interactions to single-user portals into the digital space, emerging platforms such as digital tabletops can enable face-to-face social interaction with digital games. We present here a new sensing architecture for creating tangible media tables. The TViews architecture provides tangible interaction through real-time tracking of multiple objects on a sensing surface equipped with an embedded display. The design can generalize across diverse applications and physical contexts, ranging from home living rooms, to classrooms or public settings. We describe the technical design and our initial game development for the table.

Radio Frequency IDentification (RFID) technology has recently been moving into everyday use contexts. Previous work has shown wearable RFID systems to be a viable mechanism for collecting data about a user’s interactions with her environment. In this paper, we present wearable RFID systems as a promising new direction in tangible game interfaces. We provide an overview of the affordances of RFID for game-play, and present some existing and future wearable RFID-based games. We also describe the construction of a cheap, easy-to-build wearable RFID system and present a how-to resource for other researchers interested in building off our work.

We present a hybrid positioning and communication system for tracking interaction objects called 'pucks' on the surface of a large LCD or plasma display. Pucks are smart sensor packages consisting of a microcontroller as well as a contact-type acoustic receiving transducer and an infrared or radio data link. A puck may take the form of a standalone interaction object, or the puck circuitry may be integrated into an existing object, such as a digital camera, cellphone, PDA, or other device. In this work we take advantage of the glass surface atop an LCD or plasma display as a communication and sensing medium, and launch 200KHz Gaussian-shaped acoustic ranging pulses into that medium from transmitting transducers adhered to the corners of the glass. We present experimental results demonstrating millimeter-scale puck positioning accuracy over the entire surface of a 32-inch LCD, at an update rate of 100Hz. We also demonstrate the scalability of this approach to much larger displays. In our first implementation, power consumption of each puck is 3V at 12mA during data transmission, 5mA during positioning, and 50μA when idle, yielding 6-8 hours of continuous tracking from a 90mAH prismatic lithium polymer battery.

Tangible Comics is a computer vision based full-body interactive storytelling environment that also functions as a comics generator. Prevailing applications of full-body computer vision have not utilized the full storytelling or performance potential of these environments. Our aim is to produce an environment that can create a space for redefining the conventions of comics, performance, film, photography, and animation. In relation to that, we are exploring the design problems that can arise when computer vision technology is contextualized in an interactive story telling environment.

The Shadow Box is a tangible computing project that exploits visual association and auditory clues to teach children the representational relationship between words and their meanings. The Shadow Box contains three major components: the main box, picture blocks and word blocks. The Shadow Box activates when a block or a matching pair of blocks is placed inside. The box prompts children to find matching blocks and combine them together. When children successfully combine the right word and picture, the box rewards them with an animated video as if they had made the objects come alive. An informal study shows that children responded positively to the concept of the box. They played with it for a length of time and engaged in a collaborative learning process with other children.

WikiTUI is a tangible multimedia annotation system that allows users to manipulate multimedia information in common physical books. WikiTUI also facilitates the exchange of information with other readers through wiki services. Using tangible interface techniques, we seek to bring the collaborative annotation and authoring capabilities that are supported by the growing space of online wikimedia into the real world, thereby extending the use of paper books into new realms.

This article describes the design of WikiTUI, a tangible user interface system that allows readers to manipulate multimedia information tied to page-level locations in common books. This system will also facilitate the exchange of information with other readers using wiki technology. While other projects have coupled the dual worlds of atoms and bits, few have combined the idea of collaborative and distributed authoring with tangible interfaces. WikiTUI not only bridges the real and digital worlds, but facilitates multiple contributions to a reference base spanning across these worlds. The goal of our work is to apply the paradigm of coupling atoms and bits in order to extend the collaborative annotation and authoring possibilities of wiki media.

In this project, I explore how to keep the interaction and learning for the pre-K students in the real world using physical objects while harnessing the power of the virtual one using Tangible Computing. In that attempt, I present here the design and implementation of "Teaching Table" - a system of digital manipulatives and display surface embedded with sensing technology and linked to computational intelligence. The table can track tagged objects placed on its surface, accurately identifying their type and location while providing a coincident visual display and audio feedback, using which activities are created that impart math knowledge to the children playing on the table. I also propose how the table can be an integral addition to the present classroom environment as it can enhance learning experiences of children in an interactive and playful way by involving them in physical activities. Focusing on the age group of pre-kindergarten children, the Teaching Table activities will attempt to develop fundamental math skills by leveraging the already existing curriculum guidelines for pre-K schools set by the State of Georgia (or, any other state as appropriate).

We are creating unified programming tools that allow developers to easily build applications for many different tangible platforms, and that can accommodate the continued evolution of the underlying sensing technologies. The Synlab API extends across several media table object-tracking platforms, RFID tagged object interactions, manipulable displays and multi-touch surfaces.

TViews is an extensible method and acoustic-based sensing framework for creating digital media tables that overcome many limitations of singlepurpose systems. The system provides a means for real-time multiobject tracking on the tabletop and for the management of large numbers of objects and applications across multiple platform instances.

Traces is an ambient installation that uses the knowledge of the intersection of bodies with floors and walls in order to represent the absence of those bodies. By acting, users explore the traces of their past actions and their absence from those actions. The technique used is based on visual hull reconstruction from multiple cameras on the GPU.

Tangible Comics is an attempt to redefine comics’ conventions for a narrative performance in a digital environment. It seeks ways of taking digital comics out of the computer screen and merging comics’ conventions into three-dimensional real space. The installation will utilize the qualities of digital games, interactive narration, digital art, performance, and augmented reality.