Holography and settling technologies: Interdependency and ambivalent behaviour

After introducing the concept of touchable holography and its applied technologies hand tracking and airborne ultrasound tactile display, the following entry would like to regard this ensemble as a dispositiv and take a look at relation, behaviour and interaction between original holographic phenomenon and added technologies.

These two added technologies behave in an ambivalent manner, unselfish and parasitical to the same extend.

Airborne ultrasound tactile display and hand tracking put themself into service of holography, which is getting abilities, that without their support holography would not be able to accomplish. Only the positioning of additional technologies enhances the reality of a 3D virtual object. It is no longer just the eye as a witness of the object's existence. The eye's testimony gets confirmation by the sensation of touch onto the user's hand. In this process it doesn't matter if these two impressions are not received from one and the same device as a common source, as long as synchronization of their impulses is fine tuned.

The enhancement of reality is not limited just to proofing the existence of a virtual object. It also leads to new fields of applications, which require tactile feedback to make them work. Therewith settling technologies are not simply opening holography gates to new scopes, but are much more precursors of new phantasms. Regarding original holography technologies, it looks that their abilities are mostly exhausted. It can no longer keep up with all the promises, that it never made by itself. Not just to survive as a medium, but to evolve and not becoming forgotten again and as the basis for further hopes, dreams and wishes, holography is reliant to other technologies, gathering around and helping to embark to new visions.

Settling technologies do not just serve, but also behave parasitical. They (ab)use holography for their part to find application, to survive and to evolve. This remains valid for both technologies, hand tracking and airborne ultrasound tactile display. Even maybe for one a little more and for the other a little less.

In reverse also holography behaves in the same way. Part unselfish, part parasitical. It should not be about an estimation who's taking greatest advantage. It is much more about an interdependency, which this entry want to illustrate.

Airborne ultrasound tactile display

About one year prior Hiroyuki Shinoda adds the Wiimote hand tracking system to make up touchable holography preliminary, he presents an airborne ultrasound tactile display at SIGGRAPH 2008 (International Conference and Exhibition on Computer Graphics and Interactice Techniques), which ought to be introduced a little more detailed in this entry, as the second method of touchable holography - excluding the projection of a 3D floating image.

Airborne ultrasound tactile display radiates airborne ultrasound and produces high-fidelity pressure fields onto a user's hand. This method is based on a nonlinear phenomenon of ultrasound. Unlike air-jet, generated fields of static pressure can be localized and do not cause unnecessary vibrations. When this propagation of ultrasound is interrupted by an object, a pressure field is exerted onto its surface. This pressure is called acoustic radiation pressure. The spatial distribution of ultrasound is controlled by multiple ultrasound transducers, based on wave field synthesis techniques. Using these techniques, various patterns of pressure field can be produced in 3D free space. Airborne ultrasound can be applied directly onto the skin without the risk of penetration. Tactile feedback is provided to the users' hand without requiring gloves or any other mechanical attachments.

Airborne ultrasound tactile display

Tactile feedback is superimposed over 3D graphics projection. The generated pressure field corresponds with the 3D virtual image. The sensation of touch would be broken, for the synchronization between pressure field and image doesn't work.

The goal of this technique is to considerably improve the usability of systems, which enables users to manipulate 3D graphic objects with their hands. It can find application e. g. in video games or 3D modeling software. "It might also be useful to manipulate small particles or probingg the surface of an object to measure the viscoelastic properties of the object from a distant point."

Hand tracking

Giving a tactile sensation to holography, besides of course a projection of a floating three dimensional image, among others a tracking system is needed, which works as an interface between hologram and human. As it is shortly shown in the attached video in entry Touchable holography a Wii Remote can be used as a simple tracking system.

Johnny Chung Lee gives a little more detailed view on how the Wii Remote tracking system works. Even he applies it in another context than holography, that ought to be shown below, because the mechanics behind it is of the same kind.

Wii Remote tracking system

"While camera-based and marker-less hand tracking systems are demonstrated these days, we use Wiimote (Nintendo) which has an infrared (IR) camera for simplicity. A retroreflective marker is attached on the tip of user's middle finger. IR LEDs illuminate the marker and two Wiimotes sense the 3D position of the finger. Owing to this hand-tracking system, the user can handle the floating virtual image with their hands."

To give a more complete understanding of how touchable holography - as Shinoda Lab from University of Tokyo presents it at SIGGRAPH 2009 - works, another entry about airborne ultrasound tactile display is added.

The snippet attached below wants to illustrate what Johnny Chung Lee is talking above about Minority report.

Minority report (2002)

This movie forestalls the technology of hand tracking several years prior it finds application in combination with holography. So to say the technology catches the phantasm of user interface including hand tracking.

Touchable holography

"It's the sense of touch. In any real city, you walk, you know? You brush past people, people bump into you. In L.A., nobody touches you. We're always behind this metal and glass. I think we miss that touch so much, that we crash into each other, just so we can feel something."

[Graham Waters - Crash 2004]

In 2007 screening of Stanislaw Lem's Ijon Tichy: Raumpilot - Die Sterntagebücher airs on german TV. In the first episode world famous space pilot, galactic diplomat, discoverer and hero of the cosmos Ijon Tichy is on a space journey to far distant planet Eggman for quite an urgent meeting. Due to traveling night and day he falls asleep and his spaceship collides with a piece of space junk. He decides that he couldn't go like this any longer and constructs an electric colleague to help him on board so he hasn't got to do all the work by himself. He calls his invention "Analoge Halluzinelle", because she is an holographic projection. Her image isn't very stable. So Ijon Tichy allots her the task to take care for the fine adjustment. Therefore he passes an antenna to her. She tries to grab it but due to the fact that she consists of nothing but light, she can't hold. The same would happen for he tries to touch her. His hand would go right through without touching and feeling.

"Mid-air displays which project floating images in free space have been seen in SF movies for several decades [Rakkolainen 2007]. Recently, they are attracting a lot of attention as promising technologies in the field of digital signage and home TV, and many types of holographic displays are proposed and developed. You can see a virtual object as if it is really hovering in front of you. But that amazing experience is broken down the moment you reach for it because you feel no sensation on your hand.

Our objective is adding tactile feedback to the hovering image in 3D free space. One of the biggest issues is how to provide tactile sensation. Although tactile sensation needs contact with objects by nature, the existence of a stimulator in the work space depresses the appearance of holographic images. Therefore some kind of remote-controlled tactile sensation is needed. That is achieved by our original tactile display [Iwamoto et al. 2008]"

It seems that reconstructing a real three-dimensional image, holography simultaneously arises a phantasm and a yearning to touch and to feel as well. Just think of Robert Schinella's hologram "Hand in Jewels" exposed at Cartier at the 5th Avenue in New York City in 1972. And think of all the passers-by trying to grab it.

But today holography seems to reach a point where possible abilities catch up phantasms.

"Up until now, holography has been for the eyes only, and if you tried to touch it, your hand would go right through, but now we have a technology that adds the sensation of touch."

(Hiroyuki Shinoda, Tokyo University, 2009)

To make holograms touchable the japanese scientists add a tactile feedback to the hovering image in 3D free space.

Three things are needed to give the sensation of touch to the user's hand:

First of all of course a holographic display, that provides floating images from an LCD using a concave mirror.

Second to that an airborne ultrasound tactile display is used which provides a tactile sensation onto the user's hand. "It utilizes the nonlinear phenomenon of ultrasound; acoustic pressure. When an object interrupts the propagation of ultrasound, a pressure field is exerted on the surface of the object." (Shinoda, 2009)

Thirdly a Nintendo Wiimote provides the exact 3D position of the user('s hand). "Owing to this hand tracking system, the users can handle the floating virtual image with their hands." (Shinoda, 2009)

As great as the field of applications of holography have been prior touchable holography as great could even be the field of applications of touchable holography itself. It could be useful for video games, 3D CADs, redefining interfaces to move from actual physical switches to virtual holographic ones, which includes a hygienic consideration as well as an eco-friendly one. With buttons appearing when one needs them an disappearing when not.