The Daily Bulletin

HPSS downtime: Tuesday, July 17th 7:30 a.m. - 10:00 a.m.

No downtime: Cheyenne, GLADE, Geyser_Caldera

A recording of the July 9 tutorial, “Using Globus and Campaign Storage,” is now available on the CISL web site. See the course page to review the presentation and download the slides. The 45-minute tutorial familiarizes users with the Globus file transfer system and the new Campaign Storage resource.

A new, larger GLADE scratch space is now available for immediate use as /glade/scratch_new/. The new space is built with the latest version of GPFS 5, providing more efficient use of the available storage and improved I/O performance. Users are encouraged to move their existing /glade/scratch files to the new space as soon as possible. As with the new /glade/p file space, users should take note of the changes for their impact on workflows and scripts.

Here are the key changes to be aware of:

  • The present /glade/scratch/ spaces will remain read/write for a period of 30 days.

  • In 30 days the present /glade/scratch/ spaces will be renamed /glade/scratch_old/ and will become read-only. The purge policy for files in /glade/scratch_old/ will be reduced to 30 days.

  • Also in 30 days, /glade/scratch_new will be renamed /glade/scratch.

  • Each user’s current scratch quotas will be preserved in the new scratch space.

  • Effective immediately, all requests for increases to scratch quotas will be made to the new scratch space.

  • /glade/scratch_old will be removed from the system in approximately 60 days, in early September.

CISL recommends using Globus as the most efficient way to transfer files across file systems. Globus monitors progress and automatically validates correctness of each file transfer. Users are asked to remove files from /glade/scratch_old/ once their transfers are complete.

Containers are a hot topic in high-performance and scientific computing, but while they can provide significant advantages they don't always live up to the hype. That’s why CISL is offering a hands-on class, “Containers and How They Work,” from 9 a.m. to noon MDT on Friday, July 20, at the NCAR Mesa Lab in Boulder.

The course explains what containers are and how they work, and it surveys some popular implementations with an eye toward supporting scientific workloads. Security and other operational concerns will also be covered for cluster administrators who are thinking about supporting containerized workloads on their systems. Topics to be covered include:

  • What are containers and how do they work?

  • Image formats (tar/filesystem/overlayfs/dense filesystem image/singularity)

  • Build vs. run [singularityhub]

  • Platform independence/reproducibility

  • User namespaces and rootless containers

  • Scheduler integration

  • Container runtimes (Docker, Charliecloud, Inception, Singularity, others)

  • OCI/Standards/runc

  • Applications

  • Education/outreach

  • Cloud

  • Reproducible science

The class is intended for anyone who is planning to deploy applications and create application environments using containers; developers and systems support staff getting started with containers; and others interested in learning about containers. Participants should be familiar with Linux and system operations and should bring a laptop and authentication token for connecting to Geyser and Caldera. Laptops should be fully charged as there may not be enough power receptacles in the seminar room.

Please use this form to register so CISL knows how many participants to expect. Space is limited to 50 participants and registration is open through July 16.

Globus team members will present a workshop at NCAR from 8:30 a.m. to 5 p.m. MDT on Wednesday, September 5, for system administrators who have deployed or are planning to deploy Globus, developers building applications for research, and others who are  interested in learning more about the service for research data management.

Place: Center Green Campus, CG1-1210-South-Auditorium, 3080 Center Green Drive, Boulder
Agenda: https://www.globusworld.org/tour/program?c=14  
Registration: https://www.globusworld.org/tour/register
Registration: No charge to attend; space is limited so register early

The session will include hands-on walkthroughs of:

  • Using Globus for file transfer, sharing and publication

  • Installing and configuring Globus endpoints

  • Incorporating Globus capabilities into your own data portals, science gateways, and other web applications

  • Automating research data workflows using Globus CLI and API — including how to automate scripted transfers to and from the new NCAR Campaign Storage

  • Using Globus in conjunction with the Jupyter platform

  • Integrating Globus services into your institutional repository and data publication workflows

  • Using Globus Auth authentication and fine-grained authorization for accessing your own services

Globus (www.globus.org) is a research data management service developed by the University of Chicago and used by hundreds of thousands of researchers at institutions in the U.S. and abroad.