📢Communication Technologies Unit 12 – Virtual and Augmented Reality

What's the Deal with VR and AR?

• Virtual Reality (VR) immerses users in a completely digital environment, blocking out the physical world
• Augmented Reality (AR) overlays digital information onto the real world, enhancing the user's perception of reality
• VR and AR exist on a spectrum called the Reality-Virtuality Continuum, with the real environment on one end and a fully virtual environment on the other
• Mixed Reality (MR) falls in the middle of the spectrum, blending real and virtual elements seamlessly
• VR and AR have gained popularity in recent years due to advancements in technology and increased accessibility
  • Smartphones have made AR more mainstream (Pokémon Go, Snapchat filters)
  • Standalone VR headsets have made VR more affordable and user-friendly (Oculus Quest)
• VR and AR have the potential to revolutionize various industries, from gaming and entertainment to education and healthcare

Key Concepts and Terminology

• Immersion refers to the degree to which a user feels present in a virtual environment
  • Factors influencing immersion include visual fidelity, audio quality, and haptic feedback
• Presence is the subjective feeling of being in a virtual environment, even when physically located in the real world
• Degrees of Freedom (DoF) describe the ways in which a user can move within a virtual space
  • 3DoF allows for rotational movement (pitch, yaw, roll)
  • 6DoF adds translational movement (forward/backward, up/down, left/right)
• Field of View (FOV) is the extent of the observable world seen at any given moment, usually measured in degrees
• Latency is the delay between a user's action and the system's response, which can break immersion if too high
• Haptics refer to the use of touch sensations to enhance user experience and interaction in VR/AR
• Inside-out tracking uses cameras and sensors on the VR/AR device to track the user's position and orientation in space
• Outside-in tracking relies on external sensors placed around the user's environment to track their movements

Evolution of VR and AR Tech

• Early VR systems in the 1960s were bulky, expensive, and limited in functionality (Sensorama, The Sword of Damocles)
• In the 1990s, VR gained popularity with the release of consumer-grade headsets (Nintendo Virtual Boy, Sega VR)
  • However, these early attempts faced issues with low resolution, high latency, and motion sickness
• AR technology emerged in the 1990s with the development of GPS and computer vision techniques
• In the 2000s, advancements in mobile technology and computer graphics paved the way for modern VR and AR
• The 2010s saw a resurgence of VR with the introduction of the Oculus Rift and HTC Vive, offering high-resolution displays and precise tracking
• Smartphone-based VR (Google Cardboard, Samsung Gear VR) made the technology more accessible to the masses
• The Microsoft HoloLens and Magic Leap One brought AR to the forefront, showcasing the potential for mixed reality applications
• Recent developments focus on improving resolution, reducing latency, and increasing comfort for extended use

Hardware: Headsets, Sensors, and More

• VR headsets display stereoscopic images to create a sense of depth and immersion
  • Tethered headsets (Oculus Rift, HTC Vive) connect to a powerful computer for rendering high-quality graphics
  • Standalone headsets (Oculus Quest, Pico Neo) have built-in processors and batteries for untethered experiences
  • Smartphone-based headsets (Google Cardboard, Samsung Gear VR) use a mobile device as the display and processor
• AR headsets (Microsoft HoloLens, Magic Leap One) use transparent displays to overlay digital content onto the real world
• VR controllers (Oculus Touch, Vive Controllers) allow users to interact with virtual objects using hand tracking and button inputs
• Sensors such as accelerometers, gyroscopes, and magnetometers track the user's head and body movements
• Cameras (RGB, depth, and infrared) enable inside-out tracking and environment mapping for AR applications
• Haptic devices (gloves, vests, suits) provide tactile feedback to enhance immersion and interaction
• Eye tracking sensors can improve rendering efficiency and enable more natural user interfaces

Software: Platforms and Development Tools

• Game engines like Unity and Unreal Engine have built-in support for VR and AR development
  • They provide tools for creating interactive 3D environments, physics simulations, and user interfaces
• WebVR and WebXR allow developers to create VR and AR experiences that run in web browsers
• AR SDKs (ARKit, ARCore, Vuforia) simplify the process of building AR applications for mobile devices
  • They offer features like plane detection, image tracking, and face recognition
• 3D modeling software (Blender, Maya, 3ds Max) is used to create assets for VR and AR experiences
• Visual scripting tools (Blueprints in Unreal, Playmaker in Unity) enable designers to create interactive logic without coding
• Spatial audio engines (Google Resonance, Steam Audio) simulate realistic sound propagation in virtual environments
• Collaboration platforms (Spatial, Mozilla Hubs) allow users to meet and work together in shared virtual spaces
• Cloud services (Amazon Sumerian, Google Poly) provide hosting and distribution for VR and AR content

Applications Across Industries

• Gaming and entertainment are the most well-known applications of VR and AR
  • VR games (Half-Life: Alyx, Beat Saber) offer immersive experiences that blur the line between reality and fiction
  • AR games (Pokémon Go, Harry Potter: Wizards Unite) encourage players to explore the real world and interact with digital content
• Education and training benefit from the ability to simulate real-world scenarios in a safe and controlled environment
  • Medical students can practice surgeries in VR without risk to patients
  • Mechanics can learn to repair complex machinery using AR overlays and step-by-step instructions
• Architecture and design firms use VR to visualize and iterate on building designs before construction begins
• Retail and e-commerce businesses employ AR to allow customers to try on clothes or see how furniture fits in their homes
• Healthcare professionals use VR for therapy, pain management, and rehabilitation
  • Exposure therapy in VR can help patients overcome phobias and PTSD
• Manufacturing and engineering teams collaborate in VR to design and prototype new products
• Marketing and advertising campaigns create memorable experiences using AR filters and interactive installations

Creating VR/AR Experiences

• Defining the target audience and platform is crucial for determining the scope and requirements of a VR/AR project
• Conceptualizing the user experience involves storyboarding, sketching, and prototyping to explore interaction design and user flow
• 3D assets such as models, textures, and animations are created using specialized software and optimized for real-time rendering
• Implementing interactivity requires programming skills in languages like C# (Unity) or C++ (Unreal)
  • Scripting is used to define object behaviors, user interactions, and game logic
• User interface design in VR/AR must consider the unique challenges of 3D space and motion controls
  • Diegetic interfaces are embedded within the virtual environment and interact with the user's gaze or gestures
  • Non-diegetic interfaces float in front of the user and are accessed through menus or buttons
• Testing and iteration are essential for ensuring a comfortable and engaging user experience
  • User testing helps identify issues with motion sickness, usability, and performance
• Optimization techniques (occlusion culling, level of detail) are employed to maintain high frame rates and reduce latency
• Publishing and distribution involve packaging the application, setting up servers, and submitting to app stores or online platforms

Challenges and Future Trends

• Technical challenges include improving display resolution, reducing latency, and increasing battery life for untethered devices
• Designing intuitive and accessible user interfaces is crucial for mainstream adoption of VR and AR
• Ethical considerations arise when blurring the lines between real and virtual worlds
  • Privacy concerns surrounding data collection and use in AR applications
  • Potential for addiction and escapism in immersive VR experiences
• Social and collaborative experiences are a growing trend in VR and AR
  • Platforms like AltspaceVR and VRChat allow users to interact in shared virtual environments
• Advancements in haptic technology will enable more realistic and immersive tactile sensations
• 5G networks will support streaming high-quality VR and AR content with low latency
• Edge computing will enable faster processing and rendering of VR and AR experiences on standalone devices
• Photorealistic graphics and real-time ray tracing will blur the line between virtual and real environments
• Brain-computer interfaces (BCIs) could enable direct control of virtual objects and characters using neural signals