What: New 1.4PB Panasas high speed scratch storage to replace GPFS scratch
When: Tuesday, September 1, 2020
Where: During the downtime on Tuesday, September 1, 2020 users will be provided access to their new scratch directories:
Academic & RPCI users: /panasas/scratch/grp-PIusername
Industry users: /panasas/scratch/grp-CompanyName
UPDATE: If you're a new group at CCR after 9/2020 you will need to request an allocation for the 'Global Scratch' resource in ColdFront. Instructions can be found here.
New Scratch Policy:
Users should be aware that there are new policies being enforced with this storage. The primary differences between this and /gpfs/scratch are:
- Groups have a 10TB and 4 million files/group quota. There is also a 2 million files/user quota
- Users must use their group's directory as described above. You can no longer make your own directories in the root scratch directory. Please make yourself a directory within your group's allocated area.
- The maximum file time allowed in the scratch spaces has been increased from 30 days to 60 days
Please see this article for more details
Change to Global Scratch Slurm Variable:
The Slurm variable $GLOBAL_SCRATCH is currently set to /gpfs/scratch. During the September 1st downtime, this will change to: /panasas/scratch
If you are using this variable you will need to append your new scratch directory name to it in your batch scripts. For example: $GLOBAL_SCRATCH/grp-PIusername
If you have this set to something else, your files will not be found and you will not be able to write new files. Therefore, your job will fail.
What about my files in /gpfs/scratch?
GPFS scratch will be available until Tuesday, September 29, 2020 at which time it will be decommissioned. Users MUST move any files they wish to keep over to Panasas BEFORE 9/29 or they will lose access to them. Remember: scratch spaces are NOT BACKED UP
We strongly encourage users to begin using Panasas immediately. The GPFS storage has been off warranty since January and is showing it's age. Any failures of the system could result in data loss and we will be unable to retrieve or repair your files.
Permissions on files in /panasas/scratch
Just like the permissions in the project directories, all files and directories need to be group-owned by your research group (usually this is the group that starts with grp-). If your files are group-owned by your account's private group (same as your username), you will get "out of space" error messages. Use the iquota command to check your quotas and usage. More information on this can be found in these articles:
We highly recommend you COPY files from /gpfs/scratch to /panasas/scratch rather than MOVE files as this will create the correct permissions inherited from the top directory.