BlueCrystal 5 FAQs
This FAQs page is specific to BlueCrystal 5 (BC5) and is intended to provide timely facility-specific information for users of the system during the early access and post-launch periods.
The content on this page will be updated over time as the system configuration is tuned and issues are encountered and resolved. The content will eventually be merged with the main documentation and this page ultimately deleted.
Early Access
Please note that access to BC5 is in early access while the facility is configured, tuned, and brought to full capacity.
If you encounter issues with the service, please check the section on known issues and workarounds to see if there is a solution or workaround. If your issue is not covered in known issues and workarounds or if you would like to provide general feedback on the service and user experience, please contact the BriCS helpdesk.
- Who can access BC5?
- How do I get access to BC5?
- How do I find the allocated resources for my project on BC5?
- How do I log in to BC5?
- How can I use my software/module?
- How can I run jobs on BC5?
- Where can I store my data on BC5?
- How do I get support?
- The documentation isn't quite correct?
- Known issues and workarounds
Who can access BC5?¶
BlueCrystal 5 (BC5) is available for use by staff and students at University of Bristol. Academic staff at University of Bristol can apply for resources on BC5 by following the instructions at https://apply.isambard.ac.uk/bristol/.
How do I get access to BC5?¶
PIs¶
Prospective PIs can apply for a project on BC5 via http://apply.isambard.ac.uk/bristol/. Once a project is allocated on BC5, you will receive an email invite to join the project in the BriCS portal.
Follow the instructions in the setup tutorial to access the portal and accept the invitation.
Project members¶
To join the project and access allocated resources, the project's PI will need to send you an invite following the instructions in the managing projects guide.
Once you have received the invite, follow the instructions in the setup tutorial to access the portal and accept the invitation.
How do I find the allocated resources for my project on BC5?¶
Information about allocated and consumed resources for projects can be viewed in the BriCS portal.
See the accounting guide for information on how resource usage is calculated and how to access this information.
How do I log in to BC5?¶
First, ensure that your computer is connected to the University of Bristol network. In general, this means either connecting to a wired/wireless network from within University of Bristol premises, or connecting to the University of Bristol VPN service.
Second, follow the instructions in the login guide to connect using clifton and SSH.
How can I use my software/module?¶
BriCS follows a self-service model for user software stacks on BlueCrystal 5. This means that we will not be maintaining a set of module files for users of the system, but will instead be providing the tools you need to build your own software.
EESSI¶
We have installed EESSI (European Environment for Scientific Software Installations) which provides an externally-maintained standard set of packages, including build tools, compilers and commonly-used software.
You can search the online list of available software to see what's available.
To access EESSI, you can run:
after which you can run module avail to see the list of modules available to load.
Note that you will also need to run these commands in any Slurm job scripts that you run that use EESSI.
Installing your own software¶
We also provide guidance and support to enable users to install and manage their own application software. See the main FAQs for details.
How can I run jobs on BC5?¶
Manage jobs with Slurm commands¶
BC5 uses the Slurm workload manager to run and manage jobs.
Jobs can be submitted, queried, and managed from the command line using standard Slurm commands (e.g. sbatch, squeue, scancel). The Slurm job management guide introduces the main commands used for day-to-day interaction with the workload manager. Note that while this guide should generally apply to BC5, some examples contain details specific to Isambard-AI/Isambard 3 and will require minor modification to run on BC5.
Submit jobs to partitions¶
Several partitions (also known as "queues") are available on BC5, grouping together the different types of compute nodes. The partition you submit a job to should be based on the type of compute node(s) you want your job to use.
Use the --partition option to select a specific partition for commands like sbatch and srun. If this is not specified, jobs will be submitted to the default partition.
| Partition name | Nodes | Maximum walltime | Notes |
|---|---|---|---|
| general (*) | 371 × AMD Epyc CPU: 32 core, 128 GiB RAM | 24h | For general purpose CPU workloads |
| highmem | 4 × AMD Epyc CPU: 56 core, 448 GiB RAM | 24h | For CPU workloads with high memory requirements |
| gpu | 4 × AMD Epyc CPU + NVIDIA GPU: 16 core, 128 GiB RAM, 1 × NVIDIA A100 GPU | 24h | For GPU workloads |
(*) Denotes the default partition that jobs are submitted to if no partition is specified.
Where can I store my data on BC5?¶
BC5 offers the same storage spaces described in the storage spaces page. These are backed by a CephFS distributed filesystem.
| Storage space | Filesystem path |
|---|---|
| User home | /home/<PROJECT>/<USER>.<PROJECT> |
| User scratch | /scratch/<PROJECT>/<USER>.<PROJECT> |
| Project | /projects/<PROJECT> |
| Project public | /projects/public/<PROJECT> |
The above storage spaces are accessible on both login and compute nodes.
The details of BC5's storage system and storage policies (e.g. quotas, expiry times) will be configured and tuned during the early access period, so information on the storage spaces page may not apply to BC5.
How do I get support?¶
Support for BC5 is provided through the BriCS helpdesk.
See the Getting Support page for advice on how submit a support ticket.
The documentation isn't quite correct?¶
The BriCS documentation site will be updated to add and integrate information about BC5 throughout the early access period.
Please raise a helpdesk ticket if you spot an issue/omission or would like to provide feedback.
Known issues and workarounds¶
During the early access period we would be grateful to receive reports of any issues you encounter while using BC5 service. Please contact the BriCS helpdesk to open a support ticket if you encounter any issues.
While working to resolve reported issues we will record known issues and workarounds in the table below.
| Issue | Description | Workaround |
|---|---|---|
| Storage space environment variables are not available on BC5 | The environment variables containing BriCS storage space paths listed on the storage spaces page (e.g. SCRATCH, PROJECTDIR) are currently not automatically set on BC5. The HOME variable is set correctly. |
Use the paths to the storage spaces directly where needed. If your workflow depends on the environment variables, please set these variables manually. |
| Node-local scratch storage is not available on BC5 | The node-local per-user /local/user/<UID> storage space is not configured on BC5 compute nodes currently. |
Use one of the other available user storage spaces (e.g. user home) for storing temporary workload data. |