We perceive everything about our reality through the combination of our senses.
It can therefore be hypothesized that by imparting artificially constructed information, our perception of reality would also change in response to it. The reality may not exist in real life, but based on the sensory stimulus, we may perceive it to be real. This is the essence of virtual reality.
In VR it is possible to interact in real time with different objects and experience a sense of presence, -the illusion of “being there”- 1 (Banos, Botella, & Perpina, 1999).Futurologists and science fiction writers of past and present have toyed with the idea of having fully immersive virtual environments. However, current virtual reality systems fall woefully short in terms of achieving a fully immersive experience, with most VR experiences catering to two senses – sight and hearing.
However, the future of true immersive experiences in mixed and virtual reality lies in the integration of all our senses – smell, taste and touch, – 2 ( Soo, D & Cheok,. 2016). Amongst all our senses, perhaps our sense of touch is so intrinsic to us that we almost take it for granted. It is one of the most underrated of all our senses yet we use it every day all the time be it sitting, sleeping, adjusting body postures, gripping or grasping objects, interacting with nature and interfacing with technology. Katherine Kuchenbecker, an associate professor of computer science at the University of Pennsylvania, likens the lack of touch in VR to something that most of us have experienced at some point of our life – a temporary loss of touch, perhaps while being anaesthetized for a dental procedure or our leg falling asleep. “That’s what current virtual reality is like: you can’t feel anything.” – 3 ( Gaudiosi, J. & Kuchenbecker, 2016 )As creatures with the perceived dominance of our sense of sight, we often do not stop to consider how incredible our sense of touch really is.
Using a combination of various different organs, we can determine various properties of an object such as hardness, temperature, geometry, texture and weight. It’s these advanced sensory abilities that allow us to pinpoint an object in our bags without looking.The sense of touch is so intuitive and effortless that it is not readily apparent to us when we lose it as compared to losing our sense of sight or hearing. The curious case of Mr. Waterman is a good example to illustrate the effects of the loss of sense of touch.
While recovering from a viral infection, Mr. waterman developed an auto immune reaction which destroyed most of his sensory nerves in his body. The immediate effects of this were staggering as Mr. Waterman couldn’t stand or walk properly.
He could move his limbs but could not control them in precise ways. To understand better the effects of loss of touch, studies were conducted where multiple people were anaesthetized in their hands and asked to do simple tasks. The studies revealed that immediately after local anaesthesia to the hand, it’s extremely difficult to grasp and manipulate small objects or perform skilled actions such as buttoning a shirt or lighting a match, despite having full visual feedback. – 4 ( Robles – De – L a – Torre.
2006 )The goal of creating technology which allows users to interact and immerse themselves in a virtual environment is alluring but certainly not a new idea. Designed in 1957 and patented in 1961, Morton Heilig created “Sensorama”, a system to simulate real world experiences and to stimulate the senses. It consisted of a 3D motion picture along with a display containing various artifacts such as images, sounds, smells and vibrations in order to trigger the senses. – 5 ( McLellan, H. 1996 )Since then, various ideas and experiments have been attempted for the purpose of improving immersive experiences in virtual environments.Researchers Dinh, Walker, Hodges, Song, and Kobayashi (1999) 6 7 conducted tests to explore the possibility of enhancing the VR experience by stimulating different senses. Using various different stimuli with different objects in the environment, they tested the effects of activating our senses like smell, sight, touch and hearing, towards the sense of presence and memory.
Their results showed that when the senses were stimulated, the sense of presence increased as well. This was particularly evident when the sense of touch and hearing were received.Other researchers like Hoffman, Hollander, Schroder, Rousseau, and Furness (1998) 6 8 also found convincing evidence of the role of tactile sensations. In their study, participants could interact with physical objects both in the virtual as well as the physical environment. While exploring the object in VR, participants could explore the same object in the physical environment simultaneously by means of tactile augmentation. The authors concluded that by being able to feel the objects and getting tactile feedback increased the sense of presence in the virtual environment. All these studies show that by stimulating the sense of touch, the experience and the efficacy of the virtual environment is increased.
Virtual and augmented reality technologies have become pretty good at tricking our brains into believing that virtual objects are really right in front of us. Sensor technologies like motion tracking and 3D mapping are constantly evolving and allow you to execute a range of motions in a virtual environment like pushing or pulling objects, grasping or throwing artefacts. Unfortunately, reaching out to touch a virtual object will not yield anything, all you will feel is thin air. Considering the ultimate goal of virtual reality is to create fully immersive experiences that mimic our real world experiences, it is clear that our tactile senses are essential to complete the illusion.
This is where Haptic Technologies become useful. The purpose of Haptic Technologies is to simulate the sense of touch using various different mechanisms. Touch can be used as a way to provide feedback to the users and also to give tactile sensation to virtual objects. The field of Haptics has steadily grown over the last few years and has slowly crept into most facets of our lives. Take our smartphones for instance; they all have capacitive touch screens which use vibrations as a form of feedback.
Since the touchscreens are just flat pieces of glass, vibrations are used to simulate the feel of pressing actual buttons. The “taptic engine” and “3D touch” features in the Iphone and the Apple Watch are examples of the haptic technology being used effectively to simulate touch and give information to the user.In an interview with Julia Franz, Katherine Kuchenbecker talks about the applications and future of Haptic Technology. “Haptic possibilities don’t end with our entertainment and communication tools — haptics are now being used to “smarten” everything from virtual reality to robotics technology. “Life is a lot more than just what you see and what you hear,” Kuchenbecker says. “You can’t do anything in the real world without reaching out and touching something.” Touch is an extremely valuable tool in fields like surgery, so it is not surprising that haptic technologies are being explored to augment medical tools training. – 9 ( Franz, J.
& Kuchenbecker, 2017)Credited with developing a technique called ‘Haptic Photography’ – A way to capture how surfaces feel to touch and to then recreate them, Kuchenbecker along with her lab have been working for years to create realistic haptic interfaces for virtual reality. Calling it haptography 2.0, she believes that adding touch is crucial to making realistic virtual environments. – 9 ( Franz, J. & Kuchenbecker, 2017)Textures are a component of touch that have so far been extremely difficult to replicate in virtual environment. Surprisingly, the Walt Disney Company has been working on this and has come up with textured touchscreens.
Instead of material that actually changes shape, it uses electrovibrations and algorithms to fool the brain into perceiving texture. – 10 ( Kim, S. C.
, Israr, A., & Poupyrev, I. 2013 )A major innovation that is taking place currently is also the development of sound wave projection based haptic experiences. Ultrahaptics 11, one of the forerunners in this domain, have devised a way to transmit ultra sound waves directly onto the hand making it seem as if you were feeling something in midair.
( see figure 1 )Individuals, companies and start-ups have also started investing in creating new technologies to try and simulate the sense of touch in VR. Kuchenbecker herself has partnered with tech entrepreneur Steven Domenikos to create a compact device that simulates the sense of touch in the digital world. This device is called the Tactai Touch 12. Senseglove 13 is a company based in the Netherlands who have designed a glove to simulate resistive touch.
The CyberGrasp 14 is a wearable hand exoskeleton that uses actuators and tendons to apply force resistance to each finger.Hiro III is a Haptic Interface Robot designed by scientists from Gifu University’s Kawasaki and Mouri Laboratory at Gifu in central Japan. – 15 ( Endo, T., Kawasaki, H.
, Mouri, T., Ishigure, Y., Shimomura, H., Matsumura, M., & Koketsu, K. 2011 )There are even companies developing full body suits with integrated haptic systems and motion tracking technology such as the Teslasuit 16.
This suit is washable and has a built in climate control system. ( See Figure 2 )By exploring the importance of touch, it has become clear that touch is an integral part of the human experience. Through its intuitive and effortless presence, it has become so intrinsic that we do not realize the importance of it until it is gone.The relevance of touch was made evident by introducing touch screen devices to the world; devices which have now become ubiquitous in almost all facets of life.
This emphasis on touch has had a profound change in how we perceive and interact with objects and technology in present day. It’s hard to believe that just a few decades ago; touchscreen technology could only be found in science fiction books and movies. Once hailed as the next big paradigm in personal computing, it is at risk of being overtaken by the immersive power and experience of virtual reality.Multiple studies have shown that by stimulating the senses, especially touch, through haptics, offers an extra dimension to a VR or 3D environment and is essential to the feeling of true immersion in those environments. It can have a profound effect on the VR experience as users felt that touch increased their sense of presence in the virtual environment. Dr. Adrian Cheok believes that the future of mixed reality—the integration of the virtual and physical world—belongs to smell, taste and touch. And by integrating them in the virtual world, will lead to experience communication – a way to share our experiences, not just information.
– 2 ( Soo, D & Cheok,. 2016). The body’s ability to physically interact with the environment forms a defining aspect of the VR experience.
Although, achieving true full touch simulation is not as simple and there is much that needs to be done. However, a lot of research and development is being done in the evolution of new and innovative haptic technologies, bringing us closer to the desired goal of creating fully immersive experiences.Apart from increasing the sense of presence in users and providing richer virtual experiences, the integration of touch in the VR environments can act as a valuable tool in creating new areas of focus and expanding the capabilities of what virtual reality technology can be used for.
A few examples to illustrate these new possible capabilities are mentioned below – Emotional aspect – An important component of touch is the emotional aspect and how it affects people. Haptic technology and VR can be leveraged as therapy to connect us to each other, as well. Amputees and the physically handicapped will greatly benefit from this form of therapy.Simulated Medical Procedures – Surgeons, dentists or other medical practitioners can practice on realistic VR simulators before trying out their procedures on actual real humans. – 17 ( Franz, J. and Kuchenbecker, 2017).
Touch feedback augmentation to VR simulators will help medical professionals hone their skills as well as get more experience.Treatment of Phobias – There is a growing number of studies and research being done as to how haptic technology along with VR can be used as distractive therapy for treatment of phobias and wounds such as burn pain reduction.Modulating Moods – By stimulating the senses, especially touch, in the virtual environment, it is possible to adjust peoples moods and can be a useful tool for behavioral therapy.
Enhanced gaming experiences – Adding realistic haptic feedback to games played in virtual environments will transform them from a visual experience to a multi sensorial dynamic experience.Sports training – Realistic haptic feedback will add another dimension to sports training. Individuals training for contact sports like Tennis, Baseball, Badminton will benefit from this added sense of realism where a realistic feel of the ball making contact can be felt.Artistic pursuits – Adding haptic feedback to VR programs like Tilt Brush will augment the creative process.
Simulating the effect of brush strokes on paper in the VR environment will add another dimension to the painting process and will allow the artist to get tactile feedback. Adding another layer of haptic resistance will enable the artworks to seem solid, where every element of the artwork can be touched and felt.To conclude, there is growing literature and evidence showing the value of adding tangible properties to virtual objects. Our sense of touch is a fundamental element in perceiving our environment and our reality.
It is only natural that in our quest to create completely immersive virtual experiences, augmenting the senses, especially our sense of touch, will play a crucial role on the road to achieving that goal. As an indicator to the rising consensus regarding the importance of touch, many individuals, researchers, institutions and companies have begun creating and testing new haptic technologies which simulate the sense of touch in VR environments. By integrating touch, we can expand the capabilities of the virtual environment in such ways that most aspects of life will be incorporated. Given the exponential rate at which haptic technology is progressing, in the coming years, we may actually get our first taste of full 100% immersive environments, experiences indistinguishable from physical ‘reality.
‘According to Alex Balladares, the future of virtual reality will not be limited to just gaming and movies, it will be integrated into the entire human experience where every possible aspect of life such as education, training, entertainment, treatment and casual experiences can be incorporated in the digital world. This will allow more and more people to experience things they had otherwise not experienced before. – 18 ( Balladares, A.