Daily Bulletin Archive

October 14, 2019

Given several factors, including that our tape archive vendor, Oracle, is getting out of the hardware market, NCAR’s HPSS tape archive will be shut off on October 1, 2021. Our archival strategy has been moving away from tape and thus, for user data, the NCAR Campaign Storage file system is the recommended location for longer-lived data. Our annual storage budget will be focused primarily on adding capacity to  Campaign Storage on a regular basis.

Move data now

CISL will not be migrating the 90 petabytes of data currently in the HPSS tape archive; it will be up to you, the user, to assess which data are worth saving and to move your files to the  Campaign Storage system and/or other institutional storage resources available to you. We also expect a substantial volume of data is no longer needed and can be deleted.

It’s important to note that tape archives have limited bandwidth. If you wait too long, you will run out of time to migrate your data and they will be lost when the system is shut down. To prevent this, complete the following action items as soon as possible: 

User action items

  1. Review the data you have on tape and decide what you need to keep beyond October 1, 2021.

  2. Delete data that you do not need to keep.

  3. Review your workflows and stop writing data to tape as soon as possible. HPSS will be put into read-only mode shortly.

  4. Copy data from tape that you need to keep as soon as possible and transfer it to your target storage system.

CISL is working on a number of resources, documentation, and tools to ease the transition. Information about these will be published as soon as it becomes available. Nevertheless,  you should take action immediately. None of these tools will help with the difficult decision about which data are scientifically worth keeping and will fit into the storage resources available to you.

Please contact CISL for advice on individual workflows, accessing data movers, and storage options. Additional information and documentation will be provided in the CISL Daily Bulletin as it becomes available.

September 30, 2019

CISL is delaying the final rollout of its new user support environment, Jira Service Desk, until October 29. Based on feedback from early users, CISL will use the next month to simplify the experience for users, provide additional training for ticket resolvers, and improve email and other interactions with the system.

 

We appreciate your patience as we work to ensure we can continue to provide a high-quality customer experience with the new system. In the meantime, users can continue to email cislhelp@ucar.edu or call 303-497-2400 to request assistance.


When ready, the new ticket system will feature a friendly user interface, simple request forms, and knowledge base articles that answer common questions. It will replace the ExtraView ticketing system for all but a few internal purposes (including NETS requests that will still use netshelp@ucar.edu).

September 30, 2019

CISL will increase the file retention period for the GLADE scratch space from 90 days to 120 days effective Tuesday, October 1. Individual files will be removed from scratch automatically if they have not been accessed – read, copied or modified – for more than 120 days. To check a file's last access time, run the command ls -ul <filename>.

The updated retention policy is expected to improve overall file system utilization and ease user issues related to managing their data holdings.

September 30, 2019

No scheduled downtime: Cheyenne, Casper, Campaign Storage, GLADE, HPSS

September 30, 2019

No Scheduled downtime for Cheyenne, Casper, HPSS and GLADE

September 27, 2019

Users of NCAR’s High Performance Storage System (HPSS) are reminded that abandoned files will be deleted on Tuesday, October 1. HPSS files are considered abandoned when the project and/or file owner's user ID have been inactive for at least 12 months and CISL did not receive responses from either the files’ owners or the project leads after multiple subsequent notifications. CISL estimates that more than 7 PB of tape storage space will be reclaimed.  More than 460 users have received multiple notifications of the pending deletions. Extensions to the October 1 deadline will not be granted.

Information on the abandoned data holdings, by project and user, is available here.

September 26, 2019

The previously announced full-day GLADE file system outage has been rescheduled for Tuesday, October 29. The outage is required to allow CISL staff to update key system software components. The Cheyenne and Casper clusters will also be unavailable throughout the outage. The downtime will begin at 6:00 a.m. MDT and is expected to last until approximately 6 p.m.  Additional reminders and more details on the outage will be published in the Daily Bulletin in mid-October.

September 26, 2019

The GLADE file system will be unavailable on October 22 to allow CISL staff to update key system software components. The Cheyenne and Casper clusters will also be unavailable throughout the outage. The downtime will begin at 6:00 a.m. MDT and is expected to last until approximately 6 p.m.  Additional reminders and more details on the outage will be published in the Daily Bulletin next month.

September 18, 2019

Previously announced plans to phase in CISL’s project space purge policy beginning October 1 have been postponed indefinitely. Moving forward, rather than purge files from the /glade/p spaces, NCAR labs and university projects will be responsible for managing their own project spaces and storage allocations. 

To aid users and storage managers, CISL now generates weekly usage reports for each NCAR lab and university subdirectory in /glade/p. The reports provide a summary of when files were last accessed, how much space is used, and details for the top 25 users. The files are named access_report.txt and can be found in:

  • /glade/p/lab_name/group_name/

  • /glade/p/univ/project_code/

  • /glade/p/uwyo/project_code/

CISL support staff will also be regularly reviewing these access reports to identify projects who are using project space to store files for long periods and may be better served with a Campaign Storage allocation.

CISL is developing additional data management tools, including utilities to help users manage their data holdings. The first of these tools will be released to NCAR labs’ data managers for testing later this month and made available to all users shortly thereafter.

September 12, 2019

CISL has introduced new tools to support unattended workflows using the Globus command line interface: long-lived InCommon certificates and the Globus Campaign Storage interface (gci) utility. The gci utility streamlines transfers between the GLADE and Campaign Storage file systems. It automatically detects the presence of a valid certificate and reactivates the NCAR GLADE and Campaign Storage endpoints when necessary, eliminating the need for manual authentication. Another new CISL utility – gcert – simplifies the process of setting up the certificate. 

See Configuring Globus for unattended workflows for more information on the gci and gcert utilities and InCommon Certificates. Interested users can request an InCommon Certificate by contacting cislhelp@ucar.edu.

Pages