CISL’s HPSS tape archive to reach end of life in 2021 - User actions needed

October 14, 2019

Given several factors, including that our tape archive vendor, Oracle, is getting out of the hardware market, NCAR’s HPSS tape archive will be shut off on October 1, 2021. Our archival strategy has been moving away from tape and thus, for user data, the NCAR Campaign Storage file system is the recommended location for longer-lived data. Our annual storage budget will be focused primarily on adding capacity to  Campaign Storage on a regular basis.

Move data now

CISL will not be migrating the 90 petabytes of data currently in the HPSS tape archive; it will be up to you, the user, to assess which data are worth saving and to move your files to the  Campaign Storage system and/or other institutional storage resources available to you. We also expect a substantial volume of data is no longer needed and can be deleted.

It’s important to note that tape archives have limited bandwidth. If you wait too long, you will run out of time to migrate your data and they will be lost when the system is shut down. To prevent this, complete the following action items as soon as possible: 

User action items

  1. Review the data you have on tape and decide what you need to keep beyond October 1, 2021.

  2. Delete data that you do not need to keep.

  3. Review your workflows and stop writing data to tape as soon as possible. HPSS will be put into read-only mode shortly.

  4. Copy data from tape that you need to keep as soon as possible and transfer it to your target storage system.

CISL is working on a number of resources, documentation, and tools to ease the transition. Information about these will be published as soon as it becomes available. Nevertheless,  you should take action immediately. None of these tools will help with the difficult decision about which data are scientifically worth keeping and will fit into the storage resources available to you.

Please contact CISL for advice on individual workflows, accessing data movers, and storage options. Additional information and documentation will be provided in the CISL Daily Bulletin as it becomes available.