Linear Tape File System

Magnetic tape data storage has been used for over 50 years, but typically did not hold file metadata in a form easy to access or modify independent of the file content data. Often external databases were used to maintain file metadata (file names, timestamps, directory hierarchy) to hold this data but these external databases were generally not designed for interoperability and tapes might or might not contain an index of their content. In Unix-like systems, there is the tar interoperable standard, but this is not well-suited to allow modification of file metadata independent of modifying file content data - and does not maintain a central index of files nor provide a filesystem interface or characteristics.

LTFS technology was first implemented by IBM as a prototype running on Linux and Mac OS X during 2008/2009. This prototype was demonstrated at NAB 2009. Based on feedback from this initial demonstration and experience within IBM the filesystem was overhauled in preparation for release as a product. The LTFS development team worked with the vendors of LTO tape products (HP and Quantum) to build support and understanding of the LTFS format and filesystem implementation leading up to the public release.

The LTFS Format Specification and filesystem implementation were released on April 12, 2010 with the support of IBM, HP, Quantum, and the LTO Consortium.[1]

LTFS v2.0.0 was released in March 2011, improving the text to clarify and remove ambiguity. It also added support for sparse files; persistent file identifiers; virtual extended attributes for filesystem metadata and control - and defined minimum and recommended blocksize values for LTFS volumes, for compatibility across various HBA hardware implementations.

The ISO/IEC 20919:2016 standard defines the LTFS Format requirements for interchanged media that claims LTFS compliance. It defines the data format, independent of the physical storage media and the software commands format, to make data truly interchangeable. The ISO standard was prepared by SNIA. It is based on LTFS v2.2, and was adopted to ISO by a joint technical committee ISO/IEC JTC 1 Information Technology.

The SNIA workgroup continues to develop LTFS and release updates. Version 2.0.0 defines rules for how the version number may change in future, and how compatibility is maintained across varying implementations. All implementations must:

• correctly read media that was compliant with any prior version
• write media that is compliant with the version they claim compliance with

While LTFS can make a tape appear to behave like a disk, it does not change the fundamentally sequential nature of tape. Files are always appended to the end of the tape. If a file is modified and overwritten or removed from the volume, the associated tape blocks used are not freed up, they are simply marked as unavailable and the used volume capacity is not recovered. Data is only deleted and capacity recovered if the whole tape is reformatted.

In spite of these disadvantages, there are several uses cases where LTFS formatted tape is superior to disk and other data storage technologies. While LTO seek times can range from 10 to 100 seconds, the streaming data transfer rate can match or exceed spinning-disk data transfer rates. Additionally, LTO cartridges are easily transportable and hold far more data than any other removable data storage format. The ability to copy a large file or a large selection of files (up to 1.5TB uncompressed data for LTO-5, and 18TB for LTO-9) to an LTFS formatted tape, allows easy exchange of data to a collaborator, or the saving of an archival copy.

Since LTFS is an open standard, LTFS formatted tapes are usable by a wide variety of computing systems and operating systems, avoiding the incompatibilities caused by previous proprietary LTO filesystems.[9]

Tape drives manufacturers often offer two different editions, one for Single Drives and one for Tape Libraries, based on the LTFS Reference Implementation.

• Thought Equity Motion[15] is executing a major film digitization and preservation project for the EYE Film Institute Netherlands. The project involves scanning more than 150 million discrete DPX files and storing them on LTO Gen5 using the LTFS format.[16] More than 1 petabyte of film will be scanned and archived over two years (2010–2012).

• IBM LTFS technology received a Pick Hit Award from Broadcast Engineering at NAB 2011.[17]
• IBM and FOX Networks received an Engineering Emmy Award in 2011 for a project that uses LTFS to store, exchange, and archive video content.[18]
• IBM received the 2011 Hollywood Post-Alliance (HPA) Engineering Excellence Award.[19]

As of standard version 2.5.1, LTFS does not support hard links.[7] If files are deleted, they become invisible to the user. However, the space occupied by a file is not freed.[7] To free up space a tape needs to be re-formatted.

• LTFS at LTO
• LTFS at SNIA
• LTFS for Dummies Book
• Implementations:
  • LinearTapeFileSystem on GitHub (LTFS Reference Implementation for stand alone tape drive)
  • IBM Spectrum Archive - IBM Spectrum Archive Single Drive Edition (SDE)
  • Oracle Tape Storage - Oracle Linear Tape File System, Open Edition
  • HP Linear Tape File System - HPE LTFS Software
  • Quantum Linear Tape File System - Quantum LTFS Source
• LTFS-Data-Management on GitHub