Virtual reality (VR)

Virtual reality technologies have appeared recently and the terminology is not yet settled (see the article “Augmented, virtual and other realities” for details). Wikipedia provides the following definitions.

Virtual reality (BP, virtual reality, VR, artificial reality) is a world created by technical means and transmitted to man through his senses: vision, hearing, touch and others. Virtual reality imitates both influence and reactions to influence. To create a convincing complex of sensations of reality, the computer synthesis of virtual reality properties and reactions is made in real time.

Virtual reality should not be confused with augmented reality. Their fundamental difference is that the virtual constructs a new artificial world, and the augmented reality only introduces separate artificial elements into the perception of the real world.

Virtual reality systems are devices that are more complete than conventional computer systems imitate interaction with the virtual environment by influencing all five human senses.

Such systems do not exist in full yet, but when creating a virtual reality, the developers try to make it happen:

– plausible – supported the user’s sense of reality

– interactive – ensured interaction with the environment

– available for study – provided an opportunity to explore a large, detailed world

– creating the effect of presence – involved in the process both the brain and the body of the user, affecting as much as possible the number of senses

Obviously, achieving these goals is possible only with the use of high-performance hardware and software.

Types of virtual reality

At this stage of development of VR technologies, the following types can be identified among them.

VR technologies with the effect of complete immersion, providing a plausible simulation of the virtual world with a high degree of detail. Their implementation requires a high-performance computer capable of recognizing user actions and reacting to them in real time, and special equipment that provides the effect of immersion.

VR technology without diving. These include simulations with images, sound and controllers that can be transmitted to the screen, preferably in a large format. These systems are considered to be virtual reality because they are far superior to other multimedia tools in terms of their impact on the viewer, although they do not fully meet the requirements of the VR.

VR technologies with a shared infrastructure. These include Second Life, a three-dimensional virtual world with elements of a social network that includes more than a million active users, the game Minecraft and others. Such worlds do not provide complete immersion (however, Minecraft already has a version for virtual reality that supports Oculus Rift and Gear VR helmets). But virtual worlds are well organized in interaction with other users, which is often lacking in “real” virtual reality products.

Virtual worlds are not only used in the gaming industry: platforms such as 3D Immersive Collaboration allow you to organize working and learning 3D spaces – this is called “presence-based collaboration”. Providing complete immersion and, at the same time, user interaction in virtuality is one of the most important areas of VR development.

VR based on Internet technology. These include, first of all, the language of Virtual Reality Markup Language, similar to HTML. Nowadays this technology is considered to be outdated, but it is possible that in the future virtual reality will be created also with the use of Internet technologies.

How VR technology works

The most common means of immersion in virtual reality are specialized helmets/glasses. A 3D video is displayed on the display in front of the user. A gyroscope and accelerometer attached to the body tracks head turns and transmits data to a computer system that changes the image on the display depending on the sensor readings. As a result, the user has the opportunity to “look around” inside the virtual reality and feel like in the real world.

For a more realistic immersion in the world of virtual reality, in addition to the sensors that track the position of the head, the VR devices can use tracking systems that track the movement of the pupils of the eyes and allow you to determine where a person is looking at each moment of time, as well as tracking the movements of the person to repeat them in the virtual world. Such tracking can be done with the help of special sensors or a video camera.

Traditional 2D controllers (mouse, joystick, etc.) are no longer sufficient to interact with virtual reality, so they are replaced by 3D controllers (manipulators that allow working in three-dimensional space).

Feedback devices are designed to enable the user to experience everything happening in the virtual world even more fully. Such devices can be vibrating joysticks, rotating armchairs.

VR devices and components

It’s believed that 80% of the information a person receives through their eyesight. Therefore, the developers of BP systems pay great attention to the devices that provide the formation of images. As a rule, they are supplemented with stereo sound devices, work on tactile effects and even imitation of smells. The impact on the taste buds is not yet reported.

A virtual reality helmet

Today’s virtual reality helmets (HMD-display, head-mounted display, video helmet) contain one or more displays that display images for the left and right eyes, a lens system to correct the geometry of the image, as well as a tracking system to track the orientation of the device in space. They now look like glasses, so they are increasingly called VR headsets or just virtual reality glasses. They can be divided into three groups:

Glasses in which the processing and output of images is provided by a smartphone (Android, iPhone, Windows Phone). A modern smartphone is a high-performance device capable of processing three-dimensional images independently. Smartphone displays have a high resolution. Almost every smartphone is equipped with sensors that allow you to determine the position of the device in space.

Glasses in which the image processing is provided by an external device (PC, Xbox, PlayStation, etc.). The external device should be high-performance and the glasses should be equipped with position sensors.

Autonomous virtual reality glasses (Lenovo Mirage Solo, together with Google, Oculus Quest from Facebook, Samsung Gear VR, etc.)

The helmets are a key component of a fully immersed VR because not only do they provide a three-dimensional image and stereo sound, but they also partially isolate the user from the surrounding reality.

MotionParallax3D displays

These displays use a human-like mechanism of perception of volume – parallax (motion parallax). For this purpose, at each moment of time, the viewer, based on its position relative to the screen, generates a corresponding projection of the three-dimensional object. Moving around the scene, the user can view it from all sides, and all objects of the scene will move relative to each other.

The parallax phenomenon multiplies the perception of volume. Unlike 3D cinema and 3D-TV, which use only binocular vision, MotionParallax3D technology allows the user to view the 3D scene from all angles, as if all its objects were real. The MotionParallax3D’s displacement of the viewer from the screen, which disrupts the volume effect of the 3D movie, only enhances the effect of the MotionParallax3D system.

A system that uses the parallax mechanism must capture the tiniest movements of the user’s head and track them with high speed and accuracy so that the brain does not detect geometric distortions of objects caused by delayed image changes.5 Delay should be no more than 20 ms, for interactive games – no more than 11 ms.6

These devices generally provide incomplete immersion because they are reproduced on displays and do not isolate the user from the environment. An exception is the CAVE (cave automatic virtual environment) virtual reality room. In such rooms, a stereoscopic image is projected onto each wall, designed for the specific location where the user is located. As a result, such an image surrounds a person from all sides, immersing him in himself. Some experts believe that VR-rooms are much better than VR-helmet: they provide higher resolution, there is no need to put on the head bulky device, some of which even slows down, and self-identification is easier because the user has the ability to see themselves all the time.

Sound

The multi-channel loudspeaker system allows for the localization of the sound source, so that the user can navigate the virtual world with the help of hearing.

Tactile and other sensations

Virtual reality gloves (information gloves, datagloves)

These gloves are equipped with sensors to track the movements of the hands and fingers. Technically, this can be done in a variety of ways: using fibre optic cables, strain gauges or piezoelectric sensors, as well as electromechanical devices (such as potentiometers).8 For example, scientists at EPFL and ETH Zurich have developed ultra-light gloves (weighing less than 8 grams per finger and only 2 mm thick). They provide “extremely realistic tactile feedback and can be powered by batteries, providing unprecedented freedom of movement.

A virtual reality costume

This suit should monitor the user’s entire body position and transmit tactile, thermal and vibration sensations, and in combination with the helmet – visual and auditory.

Smells and tastes

Odour synthesis has been going on for years,11 but the results are still far from being widely used. There are no significant achievements in the field of transmitting taste sensations.

Control devices

To interact with the virtual environment, special joysticks (gamepads, wands) are used, containing built-in position and motion sensors, as well as mouse-like buttons and scroll wheels. These joysticks are now increasingly becoming wireless.

The information gloves and virtual reality suits mentioned above can also be used as controls.

Unification issues

As is usually the case with the introduction of new technologies, each of the major suppliers who have entered a promising market, strives to promote their products, to disseminate their technical solutions. Accordingly, the leading companies, having released VR-headsets, develop or order content for them. The driving force behind the VR market at the moment are virtual games, primarily for gamers and were released headsets Oculus Rift, Samsung Gear VR, HTC Vive, PlayStation VR, etc.12

Games and other content designed for one headset do not play on the other. Players can’t wait for games to be ported between headsets of different developers. Manufacturers, advertisers and representatives of many other industries would implement VR more quickly, knowing that expensive equipment does not have to be changed because of the fact that the new, extremely attractive software was developed for other glasses, gloves and suits of virtual reality.

VR vendors are well aware that a well-established collaboration between them can take virtual reality to a whole new level. Therefore, as early as December 2016, the Global Virtual Reality Association (GVRA), a non-profit organization of virtual reality helmet (VR) manufacturers, was established to unite the efforts of companies in the development of this area. Acer Starbreeze, Google, HTC VIVE, Oculus, Samsung and Sony Interactive Entertainment took part in its creation.

According to the GVRA website,13 the main task of the association is to promote global growth and development of the VR industry. It is planned to create working groups to conduct research and develop recommendations on the most important topics for the industry. Ultimately, these groups will develop and share best practices openly.

However, as of October 2018, almost two years after the creation of GVRA, the only material that appeared on the association’s website was the report “Research on Virtual Reality and its Potential for Europe”, covering the period from 2016 to 2017.14 It seems that reaching global agreements between large companies is no less difficult than developing VR technologies themselves.

However, efforts to unify equipment continue

For example, on July 17th, 2017, NVIDIA, Oculus, Valve, AMD, and Microsoft unveiled the VirtualLink™ specification, an open industry standard that will allow next-generation VR headsets to connect to PCs and other devices using only one Type-C high-speed USB cable (instead of multiple cords and connectors currently in use). It is noted that VirtualLink is specifically designed for VR. It provides optimal latency and bandwidth, enabling helmet and PC manufacturers to create a new generation of virtual reality.15

Of course, unification challenges will still be met in one way or another, as has already been the case with other technologies, as long as this happens in the coming years.

Virtual reality in industry

Examples of the diverse applications of VR technologies in industry are given in the article “Virtual Reality (VR): Best Practices”.

Financial perspectives

Investors’ attitudes towards virtual reality are mixed. On the one hand, a VR helmet can be bought in any electronics store. Sony has sold more than 1.5 million PlayStation VR headsets for its console since the end of 2016. Thousands of companies create the appropriate content. However, with the introduction of VR technology to the commercial market, the developers of the first wave, apparently, rushed. As a result, users not only did not get the promised effect of complete immersion, but also faced with the imperfection of the technology, disappointed in it. Mass spread of VR/AR is restrained, firstly, by the low quality of VR content, secondly, by the fragmentation of platforms and the lack of uniform standards in its creation, and thirdly, by the lack of a clear distribution system, a single platform where the relevant products would be assembled.

Accordingly, the market behaves as well. In the first quarter of 2018, global shipments of virtual reality headsets increased by 16% year-on-year, according to experts from Canalys.16 But in the second calendar quarter of this year, according to IDC estimates, shipments decreased by 33.7%. However, analysts are confident that the situation is temporary. The emergence of new products, primarily Oculus Go and HTC Vive Pro, as well as new brands, should return the market to a positive direction.17

Gartner and IDC analysts say that VR/AR is approaching the stage of technological maturity. That is, very soon virtual reality will become part of everyday life. Technologically, everything is ready for its mass use.