Daily Bulletin Archive

Feb. 22, 2018

Documentation for how to use CISL’s peak_memusage tool now includes information about running it with Slurm jobs on Geyser and Caldera. Examples of PBS sample scripts for Cheyenne jobs also have been updated. The utility helps users determine how much memory a program needs in order to run successfully. See Checking memory use for details

Feb. 22, 2018

Data sets that are provided to researchers through the CMIP Analysis Platform can now be found on the GLADE disk storage system in /glade2/collections/cmip. The original location (/glade/p/CMIP) will be removed on February 28.

By hosting climate data on GLADE, the CMIP Analysis Platform enables researchers  to work with it on the Geyser and Caldera analysis and visualization clusters without needing to transfer large data sets from Earth System Grid Federation (ESGF) sites to their local machines.

See Adding data sets to request the addition of data sets that are not already available on GLADE.

Feb. 20, 2018

No downtime: Cheyenne, GLADE, Geyser_Caldera and HPSS

Feb. 20, 2018

The Cheyenne “standby” batch queue has been removed from the system until further notice due to recently discovered difficulties with scheduling jobs in that queue. The other batch queues remain available to users: premium, regular, economy, and share. See Job-submission queues and charges for more complete information on Cheyenne’s batch queues.

Feb. 20, 2018

CISL will reactivate the purge policy for the GLADE scratch file space on Wednesday, February 7. The purge policy was turned off following the December 30 power outage at the NWSC facility so that users would not suddenly lose files when Cheyenne, Geyser, Caldera, and Glade were restored to service.

The purge policy data-retention limit will be increased from 45 days to 60 days and use two time and date factors: a file’s creation date and its last access date. Previously only the last access date was considered.

Files that were created more than 60 days ago and have not been accessed for more than 60 days will be deleted. CISL monitors scratch space usage carefully and reserves the right to decrease the 60-day limit as usage increases. Users will be informed of any change to the purge policy.

GLADE scratch space is for temporary, short-term use and not intended for long-term storage needs.

Feb. 16, 2018

All research projects are undertaken with the hope to produce findings and products of lasting value. It is often unthinkable to consider that someone could forget the details relating to a project, especially how the results are produced. However, the state of becoming an “unloved data set” is often reached unintentionally over time. Specifically, if the research projects lose sight of data management actions, research results and products could be at risk of becoming forgotten or “unloved” when the team moves on to new projects.

The Data Stewardship Engineering Team (DSET) is a cross-organizational team formed by the NCAR Directors. DSET’s charter specifies that the DSET leads the organization’s efforts to provide enhanced, comprehensive digital data discovery and access, and the team is focused on providing a user-focused, integrated system for the discovery and access of digital scientific assets.

The DSET and the DASH services are here to help in promoting NCAR’s scientific results and allow them to be used, so that they would be valued for the long term.

If you would like to learn more about DSET/DASH and its services after the LYD week, please contact us at datahelp@ucar.edu.

Thank you for participating in Love Your Data Week by reading this and the previous four posts. If you have missed any of the five posts during this week, they are available in Staff Notes as well as the Daily Bulletin archive, or please feel welcome to contact the Data Curation & Stewardship Coordinator.

Feb. 15, 2018

Finding the right data for a particular data story depends on many factors, including what were the research questions that produced the data, who was on the research project team, what are the terms and conditions for gaining access to the data, what data formats are available for use, and so on. Ultimately, the determination of whether a data set could be “right” for a data story relies both on the information from the original data producers and the information that the potential data users are able to access and understand.

Allowing NCAR’s data to be accessible by NCAR’s immediate communities is a significant first step. As the Digital Asset Services Hub (DASH) services progress in their development, the DASH would like to continue to help the NCAR community to fulfill and optimize the full potential of NCAR’s research data. This can include contributing to data efforts outside of NCAR, including assisting in education, communication, and increasing awareness for the Earth Sciences as a whole.

To learn more about how DASH is participating in data initiatives outside of NCAR, such as having the Data Curation & Stewardship Coordinator serve as a mentor and be on data advisory boards, please contact us at datahelp@ucar.edu.

The last LYD Week post is tomorrow and will be about “We are Data.”

Feb. 15, 2018

XSEDE is offering introductory and advanced training sessions this Thursday and Friday via webcast from the Texas Advanced Computing Center. The focus of these training sessions will be on programming for manycore architectures such as Intel's Xeon Phi and Xeon Scalable processors. Both classes run from 7 a.m. to 11 a.m. MST. See these links for registration and class details:

Feb. 14, 2018

Data stories could be told by anyone who could understand and work with data, and the stories could be about any issues that are pertinent to the storyteller. The diversity of the data being used by the broad range of data users is a key factor that makes data stories engaging.

It is important to note that a storyteller is also a data user, and to be a data user, data must be shared and made accessible first. The more types of data that are made available, the higher the possibility that someone can create a compelling story by using data.

The DASH Search system from the Digital Asset Services Hub (DASH) is NCAR’s new metadata registry that facilitates the discovery, identification, and understanding of the research products and output from NCAR labs via a centralized system. The DASH Search system uses the NCAR Dialect to describe and record the resources that are available from NCAR. Once the metadata records of the available resource are submitted to the DASH Search, a potential user could effectively and efficiently locate the desired data using the information in the metadata records. Continuing to increase the access of NCAR’s data via the DASH Search system will help in communicating our science to our community and beyond, including through data stories.

To learn more about DASH Search, please visit https://data.ucar.edu/ or if you would like to submit a metadata record of your data to DASH Search, please contact us at datahelp@ucar.edu.

Day 4’s post will discuss “Connected conversations.”

Feb. 13, 2018

Before using data to tell a story, the data should be evaluated for its quality. Although data quality can be difficult to measure, quality attributes of the data, including completeness, accuracy, credibility, and consistency, are key for building a trustworthy story. Without high-quality data, readers could easily lose confidence in the story, or worse yet, quickly deem the story and its data as hearsay.

In order to achieve high- quality data and mitigate the chance for the data to be misused, it is critical to also have high-quality documentation or metadata for the data. At NCAR, the NCAR Dialect is the designated metadata standard used by the Digital Asset Services Hub (DASH) services, including the DASH Search system. The NCAR Dialect is a customized metadata schema that is designed based on international metadata standards for scientific data. The NCAR Dialect is capable of recording in-depth descriptions to assist with data understandability as well as capturing information that is essential for identification and discovery of the assets. The DASH Search Request to Submit Form demonstrates the elements that are included in the NCAR Dialect.

To learn more about the NCAR Dialect or if you would like to submit a metadata record of your data to DASH Search, please contact us at datahelp@ucar.edu.

Coming up for Day 3 is a post on “Telling Stories with Data.”