Skip to content

Python

Using Python virtual environments with venv

For Python specifically, you can set up a virtual environment, which acts similarly to a personal module but for using Python packages. In these virtual environments, you can install whichever Python packages you want using pip, and the result exists as a directory that you can load into sessions.

$ module load python # (1)!
$ python3 -m venv myenv # (2)!
$ source myenv/bin/activate # (3)!
$ pip install <my-python-package>
  1. This will load the cluster default python version. If you need a specific version, you can instead use module load python/3.12.4.
  2. this creates a new virtual environment name myenv
  3. This activates the environment, to deactivate it run deactivate

Using Python virtual environments with conda

A similar process can be used with miniconda to create and activate a conda environment

$ module load miniconda3
$ conda create --name <my-env>
$ conda init bash # (1)!
$ conda activate <my-env> # (2)!
$ conda install <my-python-package>
  1. Note that this modifies your .bashrc so that every time you log in conda will be sourced as your python distribution.
  2. run conda deactivate to exit your conda environment.

For more information, you can look at the conda env documentation

Jupyter Notebooks vs. .py Files

We recommend using Jupyter notebooks for exploration and data visualization. Notebooks provide an interactive environment that’s ideal for experimenting and analyzing data. For larger scripts or batch jobs, .py files are often a better fit, as they are easier to run on compute nodes in non-interactive sessions.

Running Jupyter Notebooks on a Compute Node

If you'd like to run a Jupyter notebook on a compute node, you can do so using the sjupyter command. This requires an active virtual environment that includes Jupyter:

$ sjupyter

This allows you to access compute resources within a notebook environment for more intensive interactive work.