JupyterHub
Prerequisites
To work through this guide you must
- Be a member of a project in the BriCS portal that has access to an Isambard-AI Phase 2 resource
- Have set a UNIX username for your account
Please see the Setup tutorial and Setting your UNIX username guide for details.
JupyterHub is a platform that allows users to start, access, and manage interactive Jupyter notebook sessions through a web interface. JupyterHub handles authentication, notebook instance spawning, and proxying on behalf of the user to provide a full web-based interactive notebook experience.
BriCS provides a JupyterHub service that allows users to authenticate with BriCS's federated identity solution and start interactive Jupyter notebook sessions on Isambard-AI phase 2.
Step 1: Connect to JupyterHub¶
To connect go to https://apps.isambard.ac.uk/aip2-jupyter in a web browser. This will open the usual identity provider selection page:
Select your identity provider and sign in following the usual process for your identity provider (see the Setup tutorial for more information on signing in with an identity provider).
Step 2: Configure your notebook session¶
After you have signed in, a "Server Options" page will open.
Use the settings on this "Server Options" page to configure your notebook session.
| Setting | Description |
|---|---|
| Project | Select a project to start the notebook session in. The job will have access to storage spaces associated with the project and will use compute resources allocated to the project. |
| Job duration | Select the length of time the notebook session will run for. After this time the session will be ended and to continue working you will need to start a new session. |
| Number of GH200s | Select the number of Grace-Hopper superchips that will be allocated to the notebook session. The job will be allocated exclusive access to all resources on the selected number of GH200s (CPU cores, memory, GPU). Up to 4 GH200s may be requested (1 Isambard-AI compute node). |
| Partition | Optionally enter a Slurm partition to submit the notebook session to. This is usually not needed, as leaving empty will submit to the default partition. |
| Reservation | Optionally enter a Slurm reservation to submit the notebook session to. This is only necessary if your project has been granted access to a Slurm reservation and you want to run your notebook session in this reservation. |
Step 3: Start the interactive notebook session¶
Once you have configured your notebook session, select the "Start" button.
This will open a status page which will display the status of your notebook session while it is being started.
After a short period your notebook session should start and your browser will automatically be redirected to a JupyterLab session running on an Isambard-AI compute node with the requested resources.
Step 4: Use the interactive notebook session¶
Your interactive JupyterLab server is now running on an Isambard-AI compute node with the resources you requested on the "Server Options" page. The JupyterLab web interface provides access to the allocated compute resources via interactive notebook, console, and terminal interfaces. The storage spaces for the selected project are available for use in the JupyterLab session.
Use a Jupyter kernel¶
Jupyter notebooks and console sessions can be started with a Jupyter kernel based on a Conda environment or Python virtual environment. This enables software installed in the environment (e.g. for machine learning, data analysis, or other workloads) to be used interactively in the notebook or console.
Some Jupyter kernels are created and managed by the BriCS team and will automatically be available for use. For example, the kernel used for the Interactive Chatbot Tutorial is available to all notebook users and can be selected to start a notebook or console session from the JupyterLab Launcher:
To use your own software environment in a Jupyter notebook or console, you can create a custom Jupyter kernel from a Conda environment or Python virtual environment. Instructions on how to do this are provided in the Use a custom Jupyter kernel in a JupyterLab session section in the Jupyter Notebooks guide. These instructions involve running commands in a terminal, which can be done in terminal tab in your JupyterLab session or in a terminal connected to Isambard-AI via SSH.
Once created, the custom kernel can be selected from the Launcher tab, e.g. for a Jupyter kernel created from Python virtual environment data-analysis-venv:
With an open notebook, the Jupyter kernel can be chosen by selecting the kernel name in the top right of the notebook tab
and then selecting the kernel from the drop down menu in the displayed kernel selection dialog box:
View GPU utilisation¶
The JupyterLab NVdashboard extension is available by default in notebook sessions started with JupyterHub. The extension allows GPU resource utilisation to be monitored in realtime with graphical interfaces and can be accessed by selecting the GPU icon in JupyterLab's left sidebar.
How much time do I have left?
To check how much time your current notebook session has been running and how much time is remaining, open a Terminal tab in JupyterLab and run the following squeue command
user.project@nid001040:~> squeue --me --name="spawner-jupyterhub"
JOBID USER PARTITION NAME ST TIME_LIMIT TIME TIME_LEFT NODES NODELIST(REASON)
10000 user.project workq spawner-jupyterhub R 1:00:00 1:00 59:00 1 nid001000
When the session reaches the time limit it will be ended without warning, so please remember to save your work frequently.
Step 5: Stop the interactive notebook session¶
When you are finished with your interactive notebook session, open the "File" menu in JupyterLab and select "Hub Control Panel".
This will open the a new page with a red "Stop My Server" button. Select the "Stop My Server" button to stop the interactive notebook session.
After the notebook session has stopped, the page will display a "Start My Server" button.
Select the "Start My Server" button to return to the "Server Options" page to configure a new notebook session (see Configure your notebook session).