🖲️Operating Systems Unit 4 – File Systems

What Are File Systems?

• File systems provide a structured way to store, organize, and manage data on storage devices (hard drives, SSDs)
• Act as an interface between the operating system and the underlying storage hardware
• Enable users and applications to create, read, update, and delete files and directories
• Maintain metadata about files and directories (file name, size, creation date, permissions)
• Ensure data integrity and consistency through various mechanisms (journaling, error checking)
• Provide a hierarchical structure for organizing files and directories (tree-like structure with a root directory)
• Abstract the complexities of the storage device, presenting a unified view of the stored data

File System Architecture

• Layered architecture consisting of multiple components working together
  • User-level APIs and libraries for interacting with the file system
  • System call interface for communication between user-level applications and the kernel
  • Virtual File System (VFS) layer for supporting multiple file system types
  • File system-specific driver for handling the particular file system implementation
  • Device driver for communicating with the storage device hardware
• VFS layer provides a common interface for different file system implementations
  • Allows the operating system to support multiple file systems seamlessly
  • Enables file system independence and flexibility
• File system-specific driver implements the actual file system operations and data structures
• Device driver interacts with the storage device controller to perform low-level I/O operations

Types of File Systems

• Disk-based file systems: Designed for storage devices with fixed-size blocks (hard drives, SSDs)
  • Examples: FAT (File Allocation Table), NTFS (New Technology File System), ext4 (Fourth Extended File System)
• Network file systems: Enable access to files stored on remote servers over a network
  • Examples: NFS (Network File System), SMB/CIFS (Server Message Block/Common Internet File System)
• In-memory file systems: Reside entirely in the computer's main memory (RAM) for fast access
  • Example: tmpfs (temporary file system) in Linux
• Journaling file systems: Maintain a journal of file system transactions to ensure data integrity
  • Examples: ext3, ext4, NTFS, HFS+ (Hierarchical File System Plus)
• Distributed file systems: Span multiple storage devices or servers to provide scalability and fault tolerance
  • Examples: HDFS (Hadoop Distributed File System), GFS (Google File System)
• Special-purpose file systems: Optimized for specific use cases or devices
  • Examples: procfs (process file system) for process information, FUSE (Filesystem in Userspace) for user-level file system development

File System Operations

• Create: Create a new file or directory within the file system hierarchy
• Open: Open an existing file for reading, writing, or both
• Read: Read data from a file and transfer it to the application's memory space
• Write: Write data from the application's memory space to a file
• Close: Close an open file, releasing any associated resources and flushing any buffered data
• Delete: Remove a file or directory from the file system
• Rename: Change the name of a file or directory
• Seek: Move the file pointer to a specific position within a file for random access
• Truncate: Reduce the size of a file by discarding data beyond a specified point
• Sync: Ensure that all buffered data is written to the storage device for consistency

File Organization and Access Methods

• Sequential access: Files are accessed sequentially, reading data in order from the beginning to the end
  • Suitable for scenarios where data is processed in a linear fashion (logs, data streams)
• Random access: Files can be accessed at any arbitrary position by seeking to a specific offset
  • Enables efficient retrieval of specific portions of a file (databases, indexes)
• Indexed access: Files are accessed using an index or key to locate the desired data quickly
  • Commonly used in database systems and key-value stores
• Contiguous allocation: Files are stored in contiguous blocks on the storage device
  • Provides good sequential access performance but may lead to fragmentation over time
• Linked allocation: Files are stored as a linked list of blocks, with each block containing a pointer to the next block
  • Allows for efficient space utilization but may have slower sequential access performance
• Indexed allocation: Files are accessed using an index or table that maps file offsets to block addresses
  • Provides a balance between space efficiency and access performance

File System Implementation

• On-disk data structures: Used to store file system metadata and file data on the storage device
  • Examples: superblock, inode table, data blocks, free space bitmap
• In-memory data structures: Used to cache file system metadata and improve performance
  • Examples: inode cache, directory cache, buffer cache
• Allocation methods: Techniques used to allocate disk space for files and directories
  • Contiguous allocation: Files are stored in contiguous blocks on the disk
  • Linked allocation: Files are stored as a linked list of blocks
  • Indexed allocation: Files are accessed using an index or table of block addresses
• Free space management: Techniques used to keep track of and allocate free disk space
  • Bitmap-based: Uses a bitmap to represent free and allocated blocks
  • Linked list-based: Maintains a linked list of free blocks
• Directory implementation: Techniques used to organize and store directory information
  • Linear list: Directories are stored as a linear list of file entries
  • Hash table: Directories are stored using a hash table for fast lookup
  • B-tree: Directories are stored using a balanced tree structure for efficient searching

File System Management and Security

• Disk quotas: Limit the amount of disk space that a user or group can consume
  • Prevents individual users from monopolizing storage resources
• Backup and restore: Techniques for creating and managing file system backups
  • Full backup: Creates a complete copy of the entire file system
  • Incremental backup: Backs up only the changes since the last backup
  • Differential backup: Backs up changes since the last full backup
• File system consistency checking: Tools for verifying and repairing file system integrity
  • Examples: fsck (file system consistency check), chkdsk (check disk)
• Access control and permissions: Mechanisms for controlling access to files and directories
  • User and group ownership: Each file and directory is associated with an owner and group
  • Permission modes: Read, write, and execute permissions for owner, group, and others
  • Access control lists (ACLs): Fine-grained control over file and directory permissions
• Encryption: Techniques for protecting file system data confidentiality
  • File-level encryption: Individual files are encrypted using symmetric or asymmetric encryption
  • Full disk encryption: The entire storage device is encrypted, protecting all files and directories

Advanced Topics and Future Trends

• Log-structured file systems: Optimize write performance by treating the file system as a circular log
  • Example: F2FS (Flash-Friendly File System) for flash-based storage devices
• Copy-on-write (COW) file systems: Use COW techniques to improve data integrity and enable efficient snapshots
  • Examples: ZFS (Zettabyte File System), Btrfs (B-tree file system)
• Persistent memory file systems: Designed for non-volatile memory technologies (NVMe, 3D XPoint)
  • Example: PMFS (Persistent Memory File System) for byte-addressable persistent memory
• Distributed and parallel file systems: Provide scalable and high-performance file system access across multiple nodes
  • Examples: Lustre, GlusterFS, Ceph
• Erasure coding: Technique for improving data durability and reducing storage overhead compared to replication
  • Used in distributed file systems and object storage systems
• File system virtualization: Abstracts multiple file systems into a single logical file system
  • Enables transparent data movement, load balancing, and storage tiering
• File system compression and deduplication: Techniques for reducing storage space usage
  • Compression: Reduces file size by removing redundant data within files
  • Deduplication: Eliminates duplicate data across multiple files