Every four years, the Digital Preservation team collaborates with Library Technology Services (LTS) to implement a storage refresh for Harvard’s digital preservation repository, the DRS. While this is an established routine, the teams took advantage of the timing of this recent refresh to equip the DRS with an extended array of storage options and corresponding lower associated costs. On Tuesday, September 28, 2021, over 75 Harvard Library staff members tuned in to an open meeting hosted by Stephen Abrams, Head of Digital Preservation, to learn about the new DRS storage enhancements (see the event recording and slides on the Digital Preservation wiki),
Screenshot from the presentation, “Introducing the DRS Storage Refresh,” hosted by Stephen Abrams on September 28, 2021 over Zoom.
Why refresh storage?
Storage media, such as magnetic tape and spinning disk, are mechanical devices that are vulnerable to a spectrum of risks, such as obsolescence, natural disasters, damage during handling, degradation of moving parts over time, and simple aging. Regular storage refreshes are a central pillar of any digital preservation strategy to protect digital content from data loss associated with storage media failures. While a person in their home can get away with refreshing their storage devices (personal computer, external hard drive, etc.) every five - eight years, Harvard’s DRS protects over 10.5 million intellectual objects from over 60 curatorial Library units and therefore takes a more proactive approach and conducts refreshes regularly every four years.
More storage and cost flexibility
Historically, the refresh process involves outlining anticipated storage needs, drafting a Request for Proposal (RFP) for storage providers to respond to, procuring the new storage, and then migrating the content from the old hardware to the new hardware. Since the conception of the DRS over two decades ago, all content deposited has been managed homogeneously: every file has been replicated four times across different types of storage hardware in geographically diverse locations, and fixity checks are run constantly across everything. This is an excellent preservation strategy that mediates risks to digital content by minimizing single points of failure. However, it has been both a curatorial and cost barrier for depositors that do not require the highest level of preservation treatment for all of their digital content. With the most recent refresh, the Digital Preservation team outlined an array of new storage options based on several different types of content genres and functions. In future, files will be replicated a variable number of times consistent with their curatorial assessment and designation. Through a combination of new higher capacity hardware, fewer copies of certain categories of content, and a greater reliance on low power and environmentally-friendly tape media, Digital Preservation will be supporting lower DRS pricing.
Timeline of the implementation
The migration to the new storage is currently underway! However, migrating over 112 million files to an array of new storage hardware and verifying that the content is secure, safe, and complete along the way is an extended process. While the teams aim to have the migration complete within the next couple of months, the timeline is variable and prioritizes the safe migration and stringent verification measures over meeting a specific deadline. However, the new storage options and price levels will go into effect no later than July 2022 (Quarter 1 of Fiscal Year 2023). While that seems like a long way into the future, Digital Preservation Services have begun education and outreach about the new options early in order to support Library curatorial units in their collections and budget planning.
The Digital Preservation team and LTS will continue to provide updates as this refresh project progresses. This will include guidance on appropriate storage selection, training and education, and new cost estimates. Watch for updates through the hlcomms listserv as well as on the Digital Preservation Services wiki.
