Lakhasly

From a developer's standpoint, it would be ideal to program the VR system by providing high-level descriptions and having the software determine automatically all of the low-level details.Which components will become more like part of a VR "operating system" and which will become higher level "engine" components? Given the current situation, developers will likely be implementing much of the functionality of their VR systems from scratch. This may involve utilizing a software development kit (SDK) for particular headsets that handles the lowest level operations, such as device drivers, head tracking, and display output. Alternatively, they might find themselves using a game en- gine that has been recently adapted for VR, even though it was fundamentally designed for video games on a screen. This can avoid substantial effort at first, but then may be cumbersome when someone wants to implement ideas that are not part of standard video games. of th

Virt

comp

along

force

imag

the c

capt

and

the c

then

map

What software components are needed to produce a VR experience? Figure 2.13 presents a high-level view that highlights the central role of the Virtual World Generator (VWG). The VWG receives inputs from low-level systems that indicate what the user is doing in the real world. A head tracker provides timely estimates of the user's head position and orientation. Keyboard, mouse, and game controller events arrive in a queue that are ready to be processed. The key role of the VWG is to maintain enough of an internal "reality" so that renderers can extract the information they need to calculate outputs for their displays.For example, if you drop an object, then it should accelerate to the ground due to gravitational force acting on it. One important component is a collision detection algorithm, which determines whether two or more bodies are intersecting in the virtual world.Developer choices for VWGs To summarize, a developer could start with a basic Software Development Kit (SDK) from a VR headset vendor and then build her own VWG from scratch.In this case, the developer must build the physics of the virtual world from scratch, handling problems such as avatar movement, collision detection, lighting models, and audio.Physics The VWG handles the geometric aspects of motion by applying the appropriate mathematical transformations.If the developer follows patterns that many before her have implemented already, then many complicated details can be avoided by simply calling functions from a well-designed software library.In addition, the VWG usually imple ments some physics so that as time progresses, the virtual world behaves like the real world.In addition to handling the motions of moving objects, the physics must also take into account how potential stimuli for the displays are created and propagate through the virtual world.Within the virtual world, user interactions, includ ing collisions, must be managed by the VWG.As applications of VR broaden, specialized VR engines are also likely to emerge.For example, one might be targeted for immersive cinematog- raphy while another is geared toward engineering design.User Locomotion In many VR experiences, users want to move well outside of the matched zone.These correspond to rendering problems, which are covered in Chapters 7 and 11 for visual and audio cases, respectively.Networked experiences In the case of a networked VR experience, a shared virtual world is maintained by a server.If multiple users are interacting in a social setting, then the burdens of matched motions may increase.The SDK should provide the basic drivers and an interface to access tracking data and make calls to the graphical rendering libraries.In a perfect world, there would be a VR engine, which serves a purpost similar to the game engines available today for creating video games.Simulated physics can become quite challenging.C

C

C

Figu

coule.

From a developer's standpoint, it would be ideal to program the VR system by providing high-level descriptions and having the software determine automatically all of the low-level details. In a perfect world, there would be a VR engine, which serves a purpost similar to the game engines available today for creating video games. If the developer follows patterns that many before her have implemented already, then many complicated details can be avoided by simply calling functions from a well-designed software library. However, if the developer wants to try something original, then she would have to design the functions from scratch. This requires a deeper understanding of the VR fundamentals, while also being familiar with lower-level system operations.

C

C

C

Figu

coule. coule

the i

Unfortunately, we are currently a long way from having fully functional, general- purpose VR engines. As applications of VR broaden, specialized VR engines are also likely to emerge. For example, one might be targeted for immersive cinematog- raphy while another is geared toward engineering design. Which components will become more like part of a VR "operating system" and which will become higher level "engine" components? Given the current situation, developers will likely be implementing much of the functionality of their VR systems from scratch. This may involve utilizing a software development kit (SDK) for particular headsets that handles the lowest level operations, such as device drivers, head tracking, and display output. Alternatively, they might find themselves using a game en- gine that has been recently adapted for VR, even though it was fundamentally designed for video games on a screen. This can avoid substantial effort at first, but then may be cumbersome when someone wants to implement ideas that are not part of standard video games.

of th

Virt

comp

along

force

imag

the c

capt

and

the c

then

map

What software components are needed to produce a VR experience? Figure 2.13 presents a high-level view that highlights the central role of the Virtual World Generator (VWG). The VWG receives inputs from low-level systems that indicate what the user is doing in the real world. A head tracker provides timely estimates of the user's head position and orientation. Keyboard, mouse, and game controller events arrive in a queue that are ready to be processed. The key role of the VWG is to maintain enough of an internal "reality" so that renderers can extract the information they need to calculate outputs for their displays.
User Locomotion In many VR experiences, users want to move well outside of the matched zone. This motivates locomotion, which means moving oneself in the virtual world, while this motion is not matched in the real world. Imagine you want to explore a virtual city while remaining seated in the real world. How should this be achieved? You could pull up a map and point to where you want to go, with a quick teleportation operation sending you to the destination. A popular option is to move oneself in the virtual world by operating a game controller, mouse, or keyboard. By pressing buttons or moving knobs, your self in the virtual world could be walking, running, jumping, swimming, flying, and so on. You could also climb aboard a vehicle in the virtual world and operate its controls to move yourself. These operations are certainly convenient, but often lead to sickness because of a mismatch between your balance and visual senses. See Sections 2.3, 10.2, and 12.3.

Physics The VWG handles the geometric aspects of motion by applying the appropriate mathematical transformations. In addition, the VWG usually imple ments some physics so that as time progresses, the virtual world behaves like the real world. In most cases, the basic laws of mechanics should govern how objects move in the virtual world. For example, if you drop an object, then it should accelerate to the ground due to gravitational force acting on it. One important component is a collision detection algorithm, which determines whether two or more bodies are intersecting in the virtual world. If a new collision occurs, then an appropriate response is needed. For example, suppose the user pokes his head through a wall in the virtual world. Should the head in the virtual world be stopped, even though it continues to move in the real world? To make it more complet, what should happen if you unload a dump truck full of basketballs into abusy street in the virtual world? Simulated physics can become quite challenging. and is a discipline in itself. There is no limit to the complexity. See Section 8.3 for more about virtual-world physics.

In addition to handling the motions of moving objects, the physics must also take into account how potential stimuli for the displays are created and propagate through the virtual world. How does light propagate through the environment? How does light interact with the surfaces in the virtual world? What are the sources of light? How do sound and smells propagate? These correspond to rendering problems, which are covered in Chapters 7 and 11 for visual and audio cases, respectively.

Networked experiences In the case of a networked VR experience, a shared virtual world is maintained by a server. Each user has a distinct matched zone. Their matched zones might overlap in a real world, but one must then be careful so that they avoid unwanted collisions. Most often, these zones are disjoint and distributed around the Earth. Within the virtual world, user interactions, includ ing collisions, must be managed by the VWG. If multiple users are interacting in a social setting, then the burdens of matched motions may increase. As users meet each other, they could expect to see eye motions, facial expressions, and body language, see Section 10.4.

Developer choices for VWGs To summarize, a developer could start with a basic Software Development Kit (SDK) from a VR headset vendor and then build her own VWG from scratch. The SDK should provide the basic drivers and an interface to access tracking data and make calls to the graphical rendering libraries. In this case, the developer must build the physics of the virtual world from scratch, handling problems such as avatar movement, collision detection, lighting models, and audio. This gives the developer the greatest amount of control and ability to optimize performance, however, it may come in exchange for a difficult implementation burden. In some special cases, it might not be too difficult. For example, in the case of the Google Street viewer (recall Figure 1.10), the "physics" is simple: The viewing location needs to jump between panoramic images in a comfortable way while maintaining a sense of location on the Earth. In the case of telepresence using a robot, the VWG would have to take into account movements in the physical world. Failure to handle collision detection could result in a broken robot (or human!).

At the other extreme, a developer may use a ready-made VWG that is cus

tomized to make a particular VR experience by choosing menu options and writing

high-level scripts. Examples available today are OpenSimulator, Vizard by World-

Viz, Unity 3D, and Unreal Engine by Epic Games. The latter two are game engines

that were adapted to work for VR, and are by far the most popular among current

VR developers. The first one, OpenSimulator, was designed as an open-source

alternative to Second Life for building a virtual society of avatars. As already

stated, using such higher-level engines make it easy for developers to make a VRexperience in little time; however, the drawback is that it is harder to make highly original experiences that were not imagined by the engine builders.