7.4 Photogrammetry and 3D scanning for realistic assets

Photogrammetry and 3D scanning are game-changers for creating realistic assets in AR/VR. These techniques capture real-world objects and environments, turning them into detailed 3D models. They're essential for bringing authenticity to virtual worlds.

From photogrammetry software to LIDAR and structured light scanning, there are various ways to digitize the physical world. Each method has its strengths, helping developers choose the right tool for their project's needs and budget.

Photogrammetry Techniques

Fundamentals of Photogrammetry

Top images from around the web for Fundamentals of Photogrammetry

• 

  MLab in the Humanities » University of Victoria » Testing Photogrammetry for the Humanities View original

  Is this image relevant?

• 

  Frontiers | A Review of Photogrammetry and Photorealistic 3D Models in Education From a ... View original

  Is this image relevant?

• 

  Frontiers | A Review of Photogrammetry and Photorealistic 3D Models in Education From a ... View original

  Is this image relevant?

• 

  MLab in the Humanities » University of Victoria » Testing Photogrammetry for the Humanities View original

  Is this image relevant?

• 

  Frontiers | A Review of Photogrammetry and Photorealistic 3D Models in Education From a ... View original

  Is this image relevant?

1 of 3

Top images from around the web for Fundamentals of Photogrammetry

• 

  MLab in the Humanities » University of Victoria » Testing Photogrammetry for the Humanities View original

  Is this image relevant?

• 

  Frontiers | A Review of Photogrammetry and Photorealistic 3D Models in Education From a ... View original

  Is this image relevant?

• 

  Frontiers | A Review of Photogrammetry and Photorealistic 3D Models in Education From a ... View original

  Is this image relevant?

• 

  MLab in the Humanities » University of Victoria » Testing Photogrammetry for the Humanities View original

  Is this image relevant?

• 

  Frontiers | A Review of Photogrammetry and Photorealistic 3D Models in Education From a ... View original

  Is this image relevant?

1 of 3

• Photogrammetry involves capturing multiple overlapping photographs of an object or environment from different angles and positions
• Uses computer vision algorithms to analyze the photographs and extract 3D information
• Relies on the principle of triangulation to determine the 3D coordinates of points in the photographs
• Requires a high degree of overlap between photographs (typically 60-80%) to ensure accurate 3D reconstruction

Structure from Motion (SfM) Process

• Structure from Motion (SfM) is a specific photogrammetry technique that automatically extracts 3D structure from a series of 2D images
• SfM algorithms identify and match feature points across multiple images to estimate camera positions and orientations
• Creates a sparse point cloud representing the 3D structure of the scene or object
• Dense point cloud is generated by interpolating additional points between the sparse points to capture more detailed geometry

Point Cloud to Mesh Conversion

• Point cloud is a set of 3D points in space representing the surface of an object or environment
• Mesh reconstruction algorithms connect the points in the point cloud to create a polygonal mesh surface
• Delaunay triangulation is a common method for mesh reconstruction, creating a network of triangles that closely approximates the object's surface
• Mesh optimization techniques, such as mesh decimation and smoothing, can be applied to reduce the mesh complexity and improve its quality

Texture Mapping and Baking

• Texture baking is the process of projecting and combining the color information from the photographs onto the reconstructed 3D mesh
• UV mapping is used to define how the 2D texture coordinates correspond to the 3D mesh surface
• Texture atlas is created by unwrapping the 3D mesh and packing the texture information into a single 2D image
• High-resolution textures can be baked to capture fine details and realistic appearance of the object or environment (4K or 8K textures)

3D Scanning Methods

LIDAR Scanning

• LIDAR (Light Detection and Ranging) is an active 3D scanning technology that uses laser light to measure distances
• Emits laser pulses and measures the time it takes for the light to bounce back from the object's surface
• Creates a dense point cloud by scanning the object or environment from multiple viewpoints
• Provides high accuracy and can capture fine details, making it suitable for industrial and engineering applications (reverse engineering, quality control)

Structured Light Scanning

• Structured light scanning projects a pattern of light (stripes, dots, or grids) onto the object's surface
• Cameras capture the deformation of the projected pattern caused by the object's geometry
• Triangulation is used to calculate the 3D coordinates of points on the object's surface based on the deformation of the pattern
• Provides high-resolution and accurate 3D scans, commonly used for small to medium-sized objects (dental impressions, artifacts)

Comparison of 3D Scanning Methods

• 3D scanning methods differ in terms of accuracy, resolution, speed, and cost
• LIDAR scanning offers high accuracy and long-range capabilities but can be expensive and requires specialized equipment
• Structured light scanning provides high-resolution scans but is limited to smaller objects and requires controlled lighting conditions
• Photogrammetry is more accessible and cost-effective but may require more manual processing and have lower accuracy compared to active scanning methods

Photogrammetry Tools

Photogrammetry Software Workflow

• Photogrammetry software automates the process of generating 3D models from photographs
• Typical workflow includes importing photographs, aligning cameras, generating point clouds, creating meshes, and texturing
• Popular photogrammetry software includes Agisoft Metashape, RealityCapture, and AliceVision Meshroom
• Cloud-based photogrammetry services, such as Autodesk ReCap Photo and Pix4D, offer web-based processing and storage solutions

Key Features of Photogrammetry Software

• Camera alignment and bundle adjustment to estimate camera positions and orientations
• Dense point cloud generation using multi-view stereo algorithms
• Mesh reconstruction and optimization tools to create a polygonal mesh from the point cloud
• Texture mapping and baking functionality to project photographic details onto the mesh
• Editing tools for cleaning up and refining the generated 3D models (hole filling, noise reduction)
• Export options to common 3D file formats (OBJ, FBX, PLY) for use in other 3D software

Considerations for Choosing Photogrammetry Software

• Ease of use and learning curve, especially for beginners
• Compatibility with different camera types and file formats (DSLR, drone, smartphone)
• Processing speed and hardware requirements for handling large datasets
• Quality and accuracy of the generated 3D models
• Integration with other 3D software and pipelines (CAD, game engines, VFX)
• Cost and licensing options (one-time purchase, subscription, educational discounts)