Daily Bulletin Archive

September 30, 2019

No Scheduled downtime for Cheyenne, Casper, HPSS and GLADE

September 27, 2019

Users of NCAR’s High Performance Storage System (HPSS) are reminded that abandoned files will be deleted on Tuesday, October 1. HPSS files are considered abandoned when the project and/or file owner's user ID have been inactive for at least 12 months and CISL did not receive responses from either the files’ owners or the project leads after multiple subsequent notifications. CISL estimates that more than 7 PB of tape storage space will be reclaimed.  More than 460 users have received multiple notifications of the pending deletions. Extensions to the October 1 deadline will not be granted.

Information on the abandoned data holdings, by project and user, is available here.

September 26, 2019

The previously announced full-day GLADE file system outage has been rescheduled for Tuesday, October 29. The outage is required to allow CISL staff to update key system software components. The Cheyenne and Casper clusters will also be unavailable throughout the outage. The downtime will begin at 6:00 a.m. MDT and is expected to last until approximately 6 p.m.  Additional reminders and more details on the outage will be published in the Daily Bulletin in mid-October.

September 26, 2019

The GLADE file system will be unavailable on October 22 to allow CISL staff to update key system software components. The Cheyenne and Casper clusters will also be unavailable throughout the outage. The downtime will begin at 6:00 a.m. MDT and is expected to last until approximately 6 p.m.  Additional reminders and more details on the outage will be published in the Daily Bulletin next month.

September 18, 2019

Previously announced plans to phase in CISL’s project space purge policy beginning October 1 have been postponed indefinitely. Moving forward, rather than purge files from the /glade/p spaces, NCAR labs and university projects will be responsible for managing their own project spaces and storage allocations. 

To aid users and storage managers, CISL now generates weekly usage reports for each NCAR lab and university subdirectory in /glade/p. The reports provide a summary of when files were last accessed, how much space is used, and details for the top 25 users. The files are named access_report.txt and can be found in:

  • /glade/p/lab_name/group_name/

  • /glade/p/univ/project_code/

  • /glade/p/uwyo/project_code/

CISL support staff will also be regularly reviewing these access reports to identify projects who are using project space to store files for long periods and may be better served with a Campaign Storage allocation.

CISL is developing additional data management tools, including utilities to help users manage their data holdings. The first of these tools will be released to NCAR labs’ data managers for testing later this month and made available to all users shortly thereafter.

September 12, 2019

CISL has introduced new tools to support unattended workflows using the Globus command line interface: long-lived InCommon certificates and the Globus Campaign Storage interface (gci) utility. The gci utility streamlines transfers between the GLADE and Campaign Storage file systems. It automatically detects the presence of a valid certificate and reactivates the NCAR GLADE and Campaign Storage endpoints when necessary, eliminating the need for manual authentication. Another new CISL utility – gcert – simplifies the process of setting up the certificate. 

See Configuring Globus for unattended workflows for more information on the gci and gcert utilities and InCommon Certificates. Interested users can request an InCommon Certificate by contacting cislhelp@ucar.edu.

September 9, 2019

CISL will preview the new Jira Service Desk ticket system at 2 p.m. Tuesday, September 17, in the Main Seminar Room at the Mesa Lab in Boulder. Participants who are unable to attend in person can see the presentation and submit questions via webcast at this URL.

The presentation will include a brief overview followed by a live demonstration of how to use the NCAR Research Computing portal to get help with Cheyenne, Casper, and related data-storage systems such as GLADE and Campaign Storage.

The Staff Support portal will have similar features for use by CISL personnel and NCAR directorate staff who need workstation support and by UCAR staff who need support for enterprise IT services operated by CISL. UCAR staff already use it to request help with UCAR IT services and business applications.

The presentation will be recorded and published online. As announced previously, CISL will begin using the new system on Tuesday, October 1.

September 6, 2019

CISL is now accepting large-scale allocation requests from university-based researchers for the 5.34-petaflops Cheyenne supercomputer and the Casper data analysis and visualization cluster. Submissions are due September 10. Researchers are encouraged to review these allocation instructions before preparing their requests.

In addition to requesting computing allocations, university projects should request long-term space on the NCAR Campaign Storage resource instead of HPSS. Unlike HPSS, Campaign Storage has no default minimum amount; users are asked to justify the amount requested. The CISL HPC Allocations Panel (CHAP) continues to closely scrutinize data management plans and storage requests, especially when proposals request a substantial portion of the storage capacity available to allocate.

At the fall meeting, CISL will allocate up to 340 million core-hours on Cheyenne, up to 2 PB of Campaign Storage space, and up to 200 TB of GLADE project space. Large allocations on Cheyenne are those requesting more than 400,000 core-hours. CISL accepts requests from university researchers for these large-scale allocations every six months. Please contact cislhelp@ucar.edu if you have any questions.

September 4, 2019

CISL will transition to using the Jira Service Desk ticket system on Tuesday, October 1, to help users quickly find the information or assistance they need. Users of high-performance computing (HPC) systems and UCAR staff will get support through new “NCAR Research Computing” and “Staff Support” portals respectively.

NCAR Research Computing portal

Through the NCAR Research Computing portal, the CISL User Services Section and High-end Services Section will support staff and external HPC users who need help working with Cheyenne, Casper, and related data-storage systems such as GLADE and Campaign Storage.

Staff Support portal

UCAR staff who need support for enterprise IT services operated by CISL, as well as CISL personnel and NCAR directorate staff who need workstation support, will use the Staff Support portal. UCAR staff already use it to request help with UCAR IT services and business applications.

The new system features a friendly user interface, simple request forms, and knowledge base articles that answer common questions. It will replace the ExtraView ticketing system for all but a few internal purposes (including NETS requests that will still use netshelp@ucar.edu). Users will log in with their UCAR usernames and CIT passwords, but can continue to request help by emailing cislhelp@ucar.edu at this time. Those who do not have CIT passwords can request them by email or by calling 303-497-2400.

Additional information on implementation and training opportunities will be available soon.

 
August 29, 2019

CISL will be releasing a new version of the PBS Pro qstat command that is designed to give HPC users more information about job status while putting less strain on the job scheduler and workload manager. The fully functioning qstat command was disabled in 2018 because its heavy use adversely affected overall system performance. The reduced resource requirements of the new version allow us to provide complete job listings again, although users can still limit the command’s output by specifying a job ID or their username.

Information from the PBS server is cached every minute, so listings will be updated at least every minute. Using the -f option to get a full listing still queries the PBS scheduler, so CISL asks that users run such full-listing operations sparingly. Run qstat -h to see all supported options.

The new version of qstat will be released on Wednesday, September 4, and it will be available in Cheyenne’s default user environment – that is, when the ncarenv module is loaded. It will also be available on Casper, unlike other PBS commands.

Pages