LTFS Reality Check - The LTO Standard Comes of Age...Or Does It?

Aug. 19, 2011 - PRLog -- The fifth generation of the LTO (Linear Tape-Open) standard is notable for its embrace of LTFS (Linear Tape File System). The goal of incorporating LTFS was to broaden LTO's range of use, increase flexibility and facilitate the exchange and transport of data, providing users disk-like access to browsing and OS level functionality without the need for software. But while LTFS does represent a major milestone for tape-based archiving, much work still remains.

Tape as a storage medium has historically had its share of detractors and adversaries, and the need for positive information has likely been a factor in spurring development of LTFS. This article will touch on the benefits of the marriage between LTO and LTFS, as well as the issues that still remain.

Tape and Its Reputation

One very interesting yet lesser known features of LTO is that the origin of the standard is actually based on combining the best features of several preceding technologies: the single track cartridge from DLT; the chip within the cartridge from AIT; the servo tracks from SLR; and error correction from DDS. This "Best of all Worlds" product is something of a rarity in the IT world, and created in LTO a highly professional and secure storage medium that has proven to be unmatched in cost and efficiency, particularly for long-term archiving and off-site storage. Yet tape is still much maligned by misconceptions concerning reliability, slow speed and cost.

Now let´s have a closer look at LTFS. What exactly is LTFS, how is it implemented and what is the benefit for the user? LTO5 introduced an option to partition tape media. Partition 0 (37GB in size) holds the index and Metadata, whereas partition 1 holds the actual data as well as a Backup copy of the index.

During a Read process, the index is read from partition 0, placed into RAM of the connected computer, and copied into a temp directory respectively. There the index is updated and written back to partition 0. Additionally, a backup copy of the index will be written after the actual stored data (on partition 1). The index itself is an XML structure (in UTF-8) that holds the file names and directories of all the files written to tape. For security reasons the index is kept twice on the tape

(Almost) Like a Disk

When and LTO tape is inserted into a drive, it mounts and displays its contents just like a disk (assuming the correct driver is installed) – the sees the files on the tape listed, just like a disk. This is certainly an important improvement, in that tapes that were written with older backup software did not mount by themselves. In any case, since every vendor created their own proprietary format or container files respectively on tape, this would have been of little use, and since the respective container files hold all the attributes, ACLs, extended finder attributes, etc., cross-platform compatibility was dependent on it.

At present LTFS still has its issues with special characters in file names, as well as in the file path. Extended attributes are not always supported between platforms, and this generates warning messages that can sometimes be ambiguous. These conditions are largely dependent upon which OS the LTO medium was written. To compensate, the user would have to document this information for data exchange, making it a cumbersome process.

Write, Read, Delete

To write a file to tape, you drag it onto the drive symbol or the window of the tape, just as with a disk. But this is where the similarity ends. Since tape is a linear medium, access to files works completely different than with disks. Files are always appended to the end of the previously written data partition. Deleting a file does not free up space – to delete a file and free up space it is necessary to delete the previous files and rewrite them back to tape again. Just reading a fully populated 1TB tape can take up to three hours. Writing back to tape takes equally long.

For users accustomed to the non-linear access of a hard disk, USB stick or SSD, this linear process can be cumbersome. Marketing tape-based media as comparable to hard disks in this respect leads to unnecessary misunderstanding and disappointment. For example: "The tape can be utilized just like a hard disk or other removable media, including directory tree structures." (www.trustlto.com). The issue of access rights further complicates things. Since the index is cached (in /tmp/ltfs) while the tape is in the drive the user has to have access rights to its directory, as well as for the mount point of the LTO drive. (Oracle LTFS)

Search and Find

As with CDs, DVDs or disks, each LTFS tape holds a directory of all files contained in the medium. When searching for a file that might be located on one of several tapes, each of those tapes must be individually placed in the drive, and its index read before the directory can be displayed. Depending on the number of files on the tape, this process can take up to several minutes. If several tapes are to be screened this can add up to half an hour or more just to locate a file. The actual reading process of the file also requires time, but is relatively fast at 140MB/s (assuming the host computer can support the data at that rate).

Traditional Backup software like Archiware´s PresSTORE has a clear advantage over this procedure. All written files and versions are stored in an index that can be browsed or searched by the user at any time. When the desired file is located, the respective tape is called up, and queued to the specific position where the file is located, since this information is also stored.

Scalability

On a smaller scale, administering a few tapes and putting them in the drive to locate a file seems manageable. As the number of tapes increases this process becomes increasingly unwieldy. If seek time is relevant, LTFS can only be used for archiving a small number of tape volumes. It is certainly not feasible on a larger scale in a professional application.

Traditional archive software like PresSTORE P4 compiles a catalog of all tapes that are part of an archive. This catalog can be browsed and searched anytime, supported with media previews to streamline the process. Searching by file name, date of archiving, path or metadata (even individually formulated) is standard. Any file can be located within seconds. Multiple files can be combined in a restore selection. The physical tape is only called by the software when starting the actual restore process, and in a tape library setting the tape will be mounted in the drive automatically.

LTFS is For Whom?

Considering all the aforementioned factors, attributes and limitations, it is clear that LTFS is only appropriate in smaller computing environments. Nevertheless, maintaining a consistent archive structure and documentation to facilitate file location and retrieval is still crucial in all environments. The additional security of redundant, off-site backup is also critical. The total number of tapes should still be manageable manually, and it is thus clear that long-term archiving with LTFS is not advisable at this point. While LTO5 is an interesting medium for data transport due to its high data density and robust technology, long-term experience with driver development and future compatibility are still unknown factors.

LTFS is still something of a novelty for LTO tape. At this stage neither the idea nor the implementation are convincing. It bears many of the attributes of a professional medium, but with a somewhat ill-fitting feature set for most users. Communicating the real advantages of LTO tape more effectively and to a broader audience would go a long way toward attracting new users. One approach would be to emphasize the unrivaled TCO of tape versus disk. Tape drives and libraries could be designed to be more attractive, with better desktop compatibility for new users and applications. There is much left to do.

# # #

About Archiware

Headquartered in Munich, Germany, Archiware is the premiere manufacturer of powerful, multi-platform data backup solutions including the award winning PresSTORE P4. PresSTORE P4 offers four modules to meet specific storage requirements:

P4 Synchronize lets you clone data between different locations (or from Xsan or metaSAN).

P4 Backup saves data from servers to disk or tape-based backup.

P4 Backup2Go provides backup for desktop and laptop workstations, with flexible scenarios for multiple workgroups.

P4 Archive provides cloning for off-site storage, creating valuable storage space while keeping data within reach.

P4 runs on Mac OS X, Linux, Windows and Solaris. It is embraced by customers worldwide in diverse industries including design, video, broadcast and print as well as universities and research institutes.

P4 is available through official resellers (www.archiware.com/reseller). Further information about P4 can be found at www.archiware.de.