When building your own Singularity containers, you MUST specify several cache and temporary directory locations, even if you have plenty of space in the directory where your container is being built.  If you don't, you'll fill up system temporary directories causing problems for other users.


Setting environment for temp folders:

We recommend using /gpfs/scratch if you're unsure how large the container will grow to.  Just remember that scratch space is for temporary use (no more than 30 days), is not backed up and is considered volatile.  See here for more info:  Scratch Usage Policy


If you're using your home or project space, make sure to check your quotas:  How to check quotas


Where ever you decide to set these variables, make sure the directory exists first. If not, create it:


SINGULARITY_LOCALCACHEDIR=/gpfs/scratch/[username]

SINGULARITY_CACHEDIR=/gpfs/scratch/[username]

SINGULARITY_TMPDIR=/gpfs/scratch[username]


export SINGULARITY_LOCALCACHEDIR

export SINGULARITY_CACHEDIR

export SINGULARITY_TMPDIR


This screenshot is courtesy of Syslabs:



More Singularity documentation can be found here: 

https://sylabs.io/docs/