👓VR/AR Art and Immersive Experiences Unit 1 – Evolution of Virtual & Immersive Art

Key Concepts and Definitions

• Virtual reality (VR) immerses users in a completely digital environment, typically using a headset that blocks out the physical world
• Augmented reality (AR) overlays digital information onto the real world, often through the use of smartphones or specialized glasses
  • Mixed reality (MR) blends elements of both VR and AR, allowing digital objects to interact with the physical environment
• Immersive experiences engage multiple senses (sight, sound, touch) to create a sense of presence and immersion in a virtual or augmented space
• Haptic feedback provides tactile sensations to enhance immersion, such as vibrations or resistance
• Stereoscopic 3D creates depth perception by presenting slightly different images to each eye
• Degrees of freedom (DOF) refer to the number of ways a user can move within a virtual space (3DOF allows rotational movement, while 6DOF adds translational movement)
• Latency is the delay between a user's actions and the system's response, which can break immersion if too high

Historical Context and Pioneers

• Early examples of VR and AR date back to the 1960s, with Morton Heilig's Sensorama (1962) and Ivan Sutherland's "The Ultimate Display" (1965)
• In the 1980s and 1990s, VR gained popularity in arcades with games like Virtuality's Dactyl Nightmare (1991) and Nintendo's Virtual Boy (1995)
• Jaron Lanier, founder of VPL Research, coined the term "virtual reality" in 1987 and developed early VR hardware and software
• In 1968, Ivan Sutherland and his student Bob Sproull created the first VR/AR head-mounted display (HMD), known as "The Sword of Damocles"
  • The device was primitive and bulky, but it demonstrated the basic principles of VR/AR
• Myron Krueger, a computer artist, developed early interactive environments in the 1970s, such as "Videoplace" (1975), which used computer vision to create interactive experiences
• In 1990, Tom Caudell, a Boeing researcher, coined the term "augmented reality" to describe a system that overlaid digital graphics onto the real world to aid in aircraft wiring

Technological Advancements

• The development of more powerful and affordable computers in the 1990s and 2000s made VR and AR more accessible to consumers
• Smartphones, with their built-in cameras, sensors, and displays, have become a common platform for AR experiences (Pokémon Go, Snapchat filters)
• High-resolution displays, such as 4K and 8K, provide clearer and more detailed visuals in VR and AR
• Wireless technology, like Wi-Fi and 5G, allows for untethered VR and AR experiences
  • This has led to the development of standalone VR headsets, such as the Oculus Quest, which don't require a separate computer or external sensors
• Advancements in computer vision and machine learning enable more accurate and responsive tracking and recognition of real-world objects and environments
• Haptic technology, such as gloves and suits, provides tactile feedback to enhance immersion
• Eye tracking allows for more natural interaction and can be used for foveated rendering, which reduces the computational load by only rendering high-resolution graphics where the user is looking

Artistic Movements and Influences

• VR and AR have been influenced by various artistic movements, such as Surrealism, which explored the blurring of reality and dreams
• The Futurists, an Italian art movement in the early 20th century, embraced technology and sought to capture the dynamism of modern life
• Conceptual art, which prioritizes ideas over traditional aesthetics, has informed the development of VR and AR as new mediums for artistic expression
• Interactive art, such as the works of Myron Krueger and Jeffrey Shaw, has explored the relationship between the viewer and the artwork, a key aspect of VR and AR experiences
• The Demoscene, a computer art subculture that emerged in the 1980s, pushed the boundaries of real-time graphics and influenced the aesthetics of early VR and AR
• Game art, particularly from immersive video games, has shaped the visual language and design principles of VR and AR experiences
  • Artists like Liam Young and Lawrence Lek have used game engines to create speculative virtual worlds that blur the line between art and gaming

Notable Works and Artists

• Jeffrey Shaw's "Legible City" (1989) is an early example of an immersive virtual environment, where users navigate a 3D city made of text by pedaling a stationary bicycle
• Char Davies' "Osmose" (1995) and "Ephémère" (1998) are pioneering VR artworks that explore themes of nature, embodiment, and spirituality
  • These works use breath and balance to control navigation, creating a more intuitive and meditative experience
• Brenda Laurel and Rachel Strickland's "Placeholder" (1993) is a multi-user VR installation that allows participants to inhabit the bodies of animals in a virtual forest
• Nonny de la Peña, known as the "Godmother of VR," has created immersive journalism pieces like "Hunger in Los Angeles" (2012) and "Project Syria" (2014), which use VR to foster empathy and understanding of social issues
• Chris Milk's "The Wilderness Downtown" (2010) is an interactive music video that incorporates Google Street View and satellite imagery to create a personalized AR experience
• Mel Chin's "Unmoored" (2018) is a site-specific AR installation in Times Square that visualizes the potential effects of climate change on New York City
• teamLab, a Japanese art collective, creates large-scale immersive installations that blend digital technology with traditional aesthetics, such as "Crystal Universe" (2015) and "Borderless" (2018)

Techniques and Tools

• Game engines, such as Unity and Unreal Engine, are commonly used to create VR and AR experiences due to their powerful graphics capabilities and built-in tools
• 3D modeling software, like Autodesk Maya and Blender, is used to create the virtual assets and environments for VR and AR
• Photogrammetry and 3D scanning techniques allow for the creation of realistic 3D models from real-world objects and environments
• Motion capture (mocap) systems, such as Vicon and OptiTrack, are used to record the movements of actors and translate them into virtual characters
• Spatial audio tools, like Google Resonance Audio and Facebook 360 Spatial Workstation, enable the creation of immersive soundscapes that respond to the user's position and orientation
• Visual scripting tools, such as Unreal Engine's Blueprints and Unity's Playmaker, allow artists and designers to create interactive logic without extensive programming knowledge
  • These tools use a node-based interface to define behaviors and interactions in a more visual and intuitive way
• AR development kits, like Apple's ARKit and Google's ARCore, provide APIs and tools for creating AR experiences on mobile devices

Current Trends and Future Directions

• Social VR platforms, such as VRChat and AltspaceVR, are becoming increasingly popular as spaces for virtual gatherings, events, and performances
• WebXR, a set of standards for delivering VR and AR content through web browsers, is making immersive experiences more accessible and shareable
• Volumetric video capture, which records 3D video of real people and objects, is being used to create more lifelike and interactive virtual characters
  • Companies like 8i and Microsoft are developing volumetric capture studios and tools
• Haptic feedback is becoming more advanced, with the development of full-body suits and gloves that can simulate a wider range of sensations
• Brain-computer interfaces (BCIs) are being explored as a way to control VR and AR experiences using neural signals, potentially making them more intuitive and accessible
• Augmented reality is being integrated into everyday devices, such as smart glasses and contact lenses, blurring the line between the digital and physical worlds
• Virtual production techniques, which use VR and AR to visualize and plan film and television shoots, are becoming more common in the entertainment industry
  • Examples include the use of LED volumes in "The Mandalorian" and the virtual production pipeline used in "The Lion King" (2019)

Impact on Society and Culture

• VR and AR are being used in a wide range of industries, from education and training to healthcare and therapy
  • Medical students can practice surgical procedures in VR, while patients can use AR to visualize their treatment plans
• Immersive journalism and documentaries are using VR and AR to create more engaging and empathetic stories, such as "The Displaced" (2015) and "Clouds Over Sidra" (2015)
• Virtual and augmented reality are changing the way we work, with remote collaboration tools like Spatial and Microsoft Mesh enabling more immersive and intuitive communication
• VR and AR are being used to preserve and provide access to cultural heritage sites, such as the recreation of the ancient city of Palmyra in "The Eternal City" (2019)
• The rise of virtual influencers and avatars, like Lil Miquela and Shudu, is raising questions about identity, authenticity, and representation in the digital age
• Concerns about privacy, addiction, and the psychological effects of prolonged VR and AR use are being studied and debated as the technologies become more widespread
  • Researchers are investigating the potential for VR and AR to create false memories, blur the line between reality and fiction, and exacerbate existing mental health issues
• The accessibility and inclusivity of VR and AR are being addressed through initiatives like SocialVR, which aims to create more diverse and welcoming virtual communities