11.5 Backup and archiving strategies

In cinematography, safeguarding your work is crucial. Backup and archiving strategies protect against data loss and preserve completed projects for future use. These practices ensure that valuable footage and files can be recovered, maintaining the integrity of your digital assets.

A robust strategy includes redundancy across storage media, geographical separation of backups, and automated processes. On-set and post-production workflows differ, with various storage options like HDDs, SSDs, and LTO tapes available. The 3-2-1 backup rule provides a solid foundation for protecting your work.

Importance of backup and archiving

• Backups protect against data loss due to hardware failure, human error, or other disasters, ensuring that valuable footage and project files can be recovered
• Archiving preserves completed projects for future reference, repurposing, or monetization opportunities
• Implementing a robust backup and archiving strategy is crucial for maintaining the integrity and longevity of digital assets in cinematography

Key components of effective strategy

Redundancy across storage media

Top images from around the web for Redundancy across storage media

• 

  Standard RAID levels - Wikipedia View original

  Is this image relevant?

• 

  business continuity — Define The Cloud View original

  Is this image relevant?

• 

  File Level Redundancy Solution Architecture - Lustre Wiki View original

  Is this image relevant?

• 

  Standard RAID levels - Wikipedia View original

  Is this image relevant?

• 

  business continuity — Define The Cloud View original

  Is this image relevant?

1 of 3

Top images from around the web for Redundancy across storage media

• 

  Standard RAID levels - Wikipedia View original

  Is this image relevant?

• 

  business continuity — Define The Cloud View original

  Is this image relevant?

• 

  File Level Redundancy Solution Architecture - Lustre Wiki View original

  Is this image relevant?

• 

  Standard RAID levels - Wikipedia View original

  Is this image relevant?

• 

  business continuity — Define The Cloud View original

  Is this image relevant?

1 of 3

• Utilizing multiple storage devices and media types (HDDs, SSDs, LTO tapes) to create redundant copies of data
• Redundancy ensures that if one storage device fails, the data can still be accessed from another copy
• Diversifying storage media helps mitigate the risk of a single point of failure

Geographical separation of backups

• Storing backup copies in different physical locations to protect against localized disasters (fires, floods, theft)
• Offsite backups can be kept in a separate building, city, or even a different region
• Cloud storage can be leveraged for geographical separation, as data is stored on remote servers

Automated backup processes

• Implementing software or scripts to automate the backup process, reducing the risk of human error and ensuring consistent backups
• Automated backups can be scheduled to run at regular intervals (daily, weekly, or after each shoot)
• Automation simplifies the backup workflow and allows for more frequent backups without manual intervention

On-set vs post-production workflows

• On-set backups focus on creating immediate copies of camera media to prevent data loss due to card failure or accidental deletion
• Post-production backups involve regularly backing up project files, assets, and work-in-progress to protect against software crashes, hardware failures, or user errors
• Archiving typically occurs at the end of post-production, creating long-term storage copies of the final project files and deliverables

Types of storage media

Hard disk drives (HDDs)

• Traditional spinning disk drives that offer large storage capacities at relatively low costs
• HDDs are suitable for both on-set and post-production backups, as well as intermediate-term archiving
• Considerations include spindle speed (5400 RPM, 7200 RPM), interface (USB, Thunderbolt, SATA), and form factor (2.5", 3.5")

Solid state drives (SSDs)

• Flash-based storage devices that provide faster read/write speeds and better durability compared to HDDs
• SSDs are ideal for on-set backups and high-performance post-production workflows
• Factors to consider include capacity, interface (USB, Thunderbolt, NVMe), and form factor (2.5", M.2)

LTO tapes for long-term archiving

• Linear Tape-Open (LTO) is a magnetic tape storage format designed for long-term data archiving
• LTO tapes offer high storage capacities, low cost per gigabyte, and a 30+ year lifespan when stored properly
• LTO drives and tapes are available in different generations (LTO-5, LTO-6, LTO-7, LTO-8), with each new generation doubling the storage capacity

Cloud storage considerations

• Cloud storage involves storing data on remote servers accessed via the internet, providing off-site backup and easy accessibility
• Services like Amazon S3, Backblaze B2, and Google Cloud Storage offer scalable and cost-effective solutions for backup and archiving
• Factors to consider include data transfer speeds, retrieval costs, and security measures (encryption, access control)

3-2-1 backup rule

3 total copies

• Maintain at least three copies of your data to ensure redundancy
• This includes the original data and two backup copies

2 different media types

• Store the backup copies on at least two different storage media types (HDDs, SSDs, LTO tapes, cloud)
• Using different media types reduces the risk of a single point of failure

1 offsite backup

• Keep one backup copy in a different physical location (offsite) to protect against local disasters
• Offsite backups can be stored in a separate building, a safety deposit box, or the cloud

Backup frequency and scheduling

• Determine the appropriate backup frequency based on the amount of data generated and the criticality of the project
• On-set backups should be performed immediately after each camera media is filled, ensuring a safe copy exists
• Post-production backups can be scheduled daily, weekly, or after significant milestones, depending on the project's pace and scale
• Automated backup software can be configured to run backups at the desired intervals, minimizing manual intervention

Archiving finished projects

Choosing file formats for archiving

• Select file formats that are widely supported, non-proprietary, and suitable for long-term preservation
• For video, consider using formats like QuickTime ProRes, DNxHD/HR, or DPX image sequences
• For project files, use standard formats like XML, AAF, or EDL to ensure compatibility with future software versions

Metadata and documentation

• Include relevant metadata and documentation with the archived project files to provide context and facilitate future use
• Metadata can include project title, date, crew information, camera settings, and software versions used
• Documentation may consist of shot lists, scripts, storyboards, and project notes

Retrieving from archives

• Establish a clear process for retrieving archived projects when needed
• Maintain an organized catalog or database of archived projects, including metadata and storage locations
• Regularly test the retrieval process to ensure that archived data can be accessed and restored successfully

Backup software and tools

• Utilize dedicated backup software or tools to streamline and automate the backup process
• Options include Retrospect, Archiware P5, Hedge, and Shotput Pro for on-set backups
• Post-production tools like CatDV, Axle AI, and Imagine Products PreRoll Post can manage backups and archives as part of an asset management workflow
• Many NLE systems (Avid Media Composer, Adobe Premiere Pro) offer built-in project backup and archiving features

Cost considerations for storage

• Factor in the cost of storage media, devices, and cloud services when planning a backup and archiving strategy
• HDDs and LTO tapes offer lower cost per gigabyte compared to SSDs and cloud storage
• Consider the long-term costs of maintaining and refreshing storage media over time
• Implement a tiered storage approach, using faster and more expensive media for active projects and cheaper, higher-capacity media for long-term archiving

Ensuring data integrity

Checksums and data verification

• Use checksums (MD5, SHA-1) to verify the integrity of backed up and archived data
• Checksums create a unique fingerprint of the data, allowing for comparison to detect any changes or corruption
• Perform periodic checks to ensure that the stored data matches the original checksums

Monitoring hardware health

• Regularly monitor the health of storage devices to detect potential issues before they lead to data loss
• Tools like S.M.A.R.T. (Self-Monitoring, Analysis, and Reporting Technology) can provide insights into hard drive health
• Replace or retire storage devices that show signs of deterioration or increased error rates

Refreshing media periodically

• Migrate data to new storage media every few years to prevent data loss due to media degradation
• HDDs and SSDs have limited lifespans, and magnetic media like LTO tapes can degrade over time
• Establish a media refresh cycle based on the manufacturer's recommendations and industry best practices

Security and access control

Encryption for sensitive data

• Implement encryption for sensitive or confidential data to protect it from unauthorized access
• Full disk encryption can be applied to storage devices, while file-level encryption can be used for specific assets
• Use strong encryption algorithms (AES-256) and manage encryption keys securely

User permissions and authentication

• Set up user permissions and access controls to limit access to backup and archived data
• Assign roles and permissions based on job responsibilities and the principle of least privilege
• Implement strong authentication methods (strong passwords, two-factor authentication) to prevent unauthorized access

Disaster recovery planning

Defining recovery objectives

• Establish recovery objectives that align with the project's requirements and stakeholder expectations
• Define Recovery Point Objective (RPO): the maximum acceptable data loss in the event of a disaster
• Determine Recovery Time Objective (RTO): the target time within which data and systems must be restored after a disaster

Testing restore processes regularly

• Regularly test the restore process to ensure that backups and archives can be successfully retrieved
• Perform test restores in a controlled environment to validate the integrity of backup data and the effectiveness of the recovery process
• Document and refine the restore procedures based on the results of the tests
• Involve relevant team members in the testing process to familiarize them with the recovery procedures