Daily Bulletin Archive

January 11, 2013

With Yellowstone now in full production, we wanted to remind all users of Yellowstone, Geyser, and Caldera about the 90-day file retention policy for the NWSC /glade/scratch file space.

Each user has a /glade/scratch/username space by default, with an individual quota of 10 TB. The scratch file space is intended to support output from large-scale capability runs as well as computing and analysis workflows across CISL-managed resources. It is a temporary space for data that will be analyzed and removed within a short amount of time.

Files that have not been accessed in more than 90 days are removed from /glade/scratch/username automatically.

To help us avoid the need to shorten the 90-day retention period, please use this space conscientiously. Delete files that you no longer need as soon as you're done with them rather than leave large amounts of data sitting untouched for the full 90 days.

If /glade/scratch remains significantly above 50% full for more than a week, CISL will review and potentially shorten the 90-day retention period.

Information about all the Yellowstone GLADE file spaces is provided at https://www2.cisl.ucar.edu/resources/glade_yellowstone

January 4, 2013

To give your jobs the best chance to run as quickly as possible in the Yellowstone environment, use LSF's bslots command to check for available backfill windows. You may be able to adjust your job script wall-clock estimate and have your job fit an available window.

The bslots command displays a summary of slots that are not being used by running jobs and are not scheduled to be used for pending jobs, along with the associated backfill windows.

See our Running jobs documentation page for more information, including sample output.

January 4, 2013

The GLADE file spaces mounted on Bluefire and Lynx will be accessible until March 31, 2013, approximately 60 days after Bluefire is decommissioned. During February and March, files will be accessible only via the Mirage cluster nodes. Users are responsible for ensuring that files are migrated to new locations before the end of this period.

Important note regarding /ptmp :: Bluefire's /ptmp file system will only be available through January 31, 2013. When Bluefire is shut down, /ptmp will become unavailable. Users with data in /ptmp must be sure to move files before the end of January. /ptmp will be made read-only on January 14 to prevent new data from being stored there.

During February 2013, Bluefire's GLADE file systems will be mounted on the Mirage nodes with read/write access to permit users to complete some final analysis tasks.

During March 2013, the file systems will be set to read-only mode, to prevent new data from being generated. Users will still have access to Mirage for the purpose of migrating data; analysis and visualization software may not be accessible.

You should consider the following decommissioning plan for the Mesa Lab/Bluefire file systems.

* /glade/home :: Home directories will be available until March 31, 2013. After that point, CISL will retain an off-line copy of the data for six months, during which period users will need to contact cislhelp@ucar.edu to access their files.

* /glade/scratch :: The current 30-day retention policy will remain in effect. Files will continue to be removed 30 days after last access.

* /glade/user, /glade/proj2, /glade/proj3 :: Files in these spaces will be available until March 31, 2013. Users are responsible for migrating essential files.

On April 1, CISL will remove all remaining data in /glade/scratch, /glade/user, /glade/proj2, /glade/proj3. The bfft.ucar.edu service for transferring files will be turned off at the same time.

Please see http://www2.cisl.ucar.edu/resources/yellowstone/transition#glade for the various ways to migrate files to the Yellowstone GLADE file spaces.

January 3, 2013

Lynx downtime Wednesday, January 2 from 9:00am - 5:00pm

No Scheduled Downtime: Yellowstone, Geyser_Caldera, NWSC GLADE, HPSS, Bluefire, DAV, ML GLADE, Firefly

January 2, 2013

The help-desk/consulting office will close early today at 1:30pm in order for staff to begin the holiday.

Computing users may still submit tickets to cislhelp@ucar.edu and we will respond by Wednesday, January 2.

December 27, 2012

No Scheduled Downtime: Yellowstone, Geyser_Caldera, NWSC GLADE, HPSS, Bluefire, DAV, ML GLADE, Firefly, Lynx

December 18, 2012

Based on current Yellowstone successes, the current timeline for Bluefire will have that system being decommissioned on January 31, 2013. Users should plan their work accordingly.

CISL and IBM continue to make progress on Yellowstone, and the agreement with IBM is to provide maintenance for Bluefire for six weeks after Yellowstone is declared production-ready.

Most Bluefire users should now have access to Yellowstone and are strongly encouraged to begin ramping up on that system, while they focus on bringing their work on Bluefire to a meaningful stopping point.

December 18, 2012

CISL provides more IDL and MATLAB licenses for the Yellowstone environment than were available previously, allowing more users to run either application at the same time. Both are installed and ready for use.

Geyser is the recommended system for running large-scale data analysis and post-processing tasks. Many other applications have been installed, or will be soon, for analysis and visualization on the Geyser and Caldera clusters.

Even with increased licenses, the Consulting Services Group (CSG) asks that individuals use no more than one MATLAB license and no more than one MATLAB Toolkit license of each variety at any one time. If CSG receives a complaint of license domination, the responsible user will be asked to kill some of the jobs. If CSG cannot reach the responsible user, system administrators will kill all but one of the related jobs.

See this page for documentation about running applications on Geyser and Caldera.

December 18, 2012

The Research Data Archive (RDA) is being copied to the new GLADE system in a process that will take several days. Until that process is complete, please continue to use Bluefire or Mirage to get the data you need from /glade/data02/dsszone as documented here: Research Data Archive.

You can then copy (cp) the files you need to your new GLADE directory for use in the Yellowstone environment.

We will make another announcement when the RDA is publicly accessible directly from the new GLADE system.

December 18, 2012

As part of the migration to Yellowstone and its new underlying accounting system (SAM), HPSS is making the transition to recognizing only those projects that have been made active in the Yellowstone environment. For some users, this transition also means that your HPSS default project may have changed.

To check your active projects in HPSS, please visit http://sam.ucar.edu and log in. From the user preferences screen, you can view or change your current default project. Please note that default project changes may take several days to propagate. We apologize for the inconvenience.

Please see the HPSS documentation at https://www2.cisl.ucar.edu/docs/hpss/projects for help on setting your project for an HSI session.

NCAR staff are most likely to be affected, since some legacy Bluefire projects were not migrated to the new accounting system. Users should start associating HPSS files with projects that are active on Yellowstone. NCAR staff may contact their divisional allocation administrators if they have questions, as well as cislhelp@ucar.edu.