How to Request the Restore of Data
CCR provides a wide variety of options for researchers to store their data. See below for a comparison of the various options. If you are unsure what option works best for your group, please contact CCR Help.
1. Vast Data Network-Attached Storage (NAS) - CCR provides a total of 2 PB of available disk storage to all the clusters in the Center for user home directories and group shared project directories. This is an enterprise-level, highly redundant storage solution designed for 99% uptime. All directories are backed up off-site nightly by UBIT for disaster recovery purposes.
Home Directories:
- Directories are available on the clusters and accessible from all compute nodes.
- Default user quota is 10GB.
- To protect the file system, user home directories are limited to 10 million files.
- Backed up by UB's Central Computing Enterprise Infrastructure Services department
Home directory location: /user/UBIT_username
Example: /user/ccrguest
Project Directories:
- Additional disk space is available for research groups in the project directories. PIs should request an allocation for 'Project Storage' to have a directory created for their group.
- The default project directory quota is 1TB.
- To protect the file system, project directories are limited to 200 million files. Any directory containing over 50 million files IS NOT BACKED UP.
- Backed up by UB's Central Computing Enterprise Infrastructure Services department
- If you require more than 1TB of storage, additional space can be purchased at the current approved rate. Please see this document for more information
Projects directory location - depends on your affiliation:
- /projects/academic/group_name - for all faculty of UB
- /projects/industry/group_name - for all industry partners (could be project name, company name, or other group name - if you're not sure, ask CCR staff)
- /projects/rpci - for all Roswell Park affiliated users
2. Global Scratch Space - Panasas
- CCR provides 1.4PB of high performance global scratch space.
- Accessible from all front end servers and compute nodes.
- Available to all users for temporary use. PIs should request an allocation for 'Global Scratch Storage' to have a directory created for their group.
- To protect the file system, users are limited to 2 million files and groups limited to 4 million files.
- Designed for high speed, parallel computing, large datasets, with the fastest I/O at the Center
- Data that has not been accessed in more than 60 days is subject to removal by an automated scrubber script. Please remove all data promptly after your job completes.
- There is NO backup of data in /panasas/scratch
- There is no guarantee of uptime
Global scratch location: /panasas/scratch/grp-PIusername or company name
NOTE: It is an outright violation of University at Buffalo computing policy to attempt to circumvent this policy. Anyone found attempting to modify file dates to get around the scrubber will have their data removed without notice and their account deactivated. We actively monitor for this!
3. Local Scratch Space:
- All servers and compute nodes in all the clusters have local disk space (/scratch).
- This scratch space is available to batch jobs running on the clusters.
- Data on local scratch may be subject to removal as soon as the job completes
- There is NO backup of data in /scratch.
More details about scratch usage at CCR and policies
4. Cloud storage:
- Accessible only from cloud instances
- Cost is $100/TB PER YEAR
- There is NO backup of data for the research cloud
- There is no guarantee of uptime or performance on cloud storage
- Available only to research groups with active cloud subscriptions
- More details about the cloud cost structure
Comparison of CCR Storage Options
Availability | Cost | Data Backup | Accessibility | Additional Notes | |
Vast Data NAS | Designed for 99% uptime 2 scheduled maintenance outages/year | Home directory (10GB): Free Project directory (1TB): Free Extra project space: See here for pricing | Yes | All CCR servers, cluster front-ends, and cluster nodes | Backups off-site daily by UBIT. Data retained securely for 6 months |
Panasas high-speed scratch | Volatile, no guarantee of uptime | Free for use while running jobs on the clusters, 10TB limit per group | No | All cluster front-ends and cluster nodes; some servers | Short term data storage only. Files over 60 days in age are automatically deleted daily |
Local scratch directory | Volatile, no guarantee of uptime | Free; limited by space available on node/server which varies across nodes | No | Local disks on all servers and cluster nodes | Data is removed when jobs complete. Users should copy all data from local disks at job end |
Cloud storage | Redundant, fiber-connected Dell Equilogic storage system 2 scheduled maintenance outages/year | $100/TB PER YR* | No | Only accessible from your cloud instances | Data deleted if cloud subscription is not renewed |
*Price for groups paying with UB funds (State, RF, UBF, etc). For all others, please contact CCR Help for a quote