Episode 2: LAMMPS with MACE on Leonardo Booster

This recipe builds LAMMPS with ML-IAP and MACE on Leonardo Booster, converts a model, and runs a copper simulation. We tested it on one A100 GPU on 13 October 2026 using the CUDA 12 stack below.

Before you start

Prerequisites

  • A Leonardo Booster account and a Slurm allocation with GPU access.

  • Familiarity with the installation workflow in Episode 1: Installing LAMMPS on EuroHPC.

  • Access to the compiler, MPI, CUDA, CMake, and Python modules listed below.

  • A working directory with enough space for the source trees, build, environment, Python wheels, and model file.

Tested configuration

Important

Start with the NVIDIA driver. Our Booster node had driver 535.274.02, which led us to choose a CUDA 12 toolkit and CUDA 12 Python packages. Under NVIDIA’s normal minor-version compatibility rules, CUDA 12.x requires driver branch 525 or newer, while CUDA 13.x requires branch 580 or newer.

Check nvidia-smi inside your GPU allocation before choosing software. See NVIDIA’s compatibility table.

The tested configuration was:

Component

Tested version

GPU and NVIDIA driver

A100-SXM-64GB; 535.274.02

CUDA toolkit / PyTorch CUDA runtime

12.2 / 12.1

GCC / OpenMPI

12.2.0 / 4.1.6

Python / CMake

3.11.7 / 3.27.9

LAMMPS

patch_30Sep2026

MACE

v0.3.13

PyTorch

2.5.1+cu121

cuEquivariance

0.4.0, with the CUDA 12 backend

CuPy / NumPy / e3nn / Cython

13.3.0 / 1.26.4 / 0.4.4 / 3.0.12

Within CUDA 12, PyTorch 2.5.1+cu121 and cuEquivariance 0.4.0 passed our dependency and GPU checks. The other Python pins follow the MACE and build requirements; GCC, MPI, CMake, and Python come from Leonardo’s available modules. The LAMMPS and MACE release pins identify the sources we used.

We compiled LAMMPS with CUDA 12.2; the PyTorch wheel bundles a CUDA 12.1 runtime. That combination passed the tests below. See CUDA Driver, Toolkit, and Compute Capability Mapping (2026) for the driver, toolkit, and GPU architecture distinctions. If you change versions, repeat pip check and the GPU runtime check in step 5.

1. Prepare the source and working directory

Run the preparation steps on a login node, where downloads are available. Record the project root so later directory changes are explicit:

mkdir -p lammps-mace-work
cd lammps-mace-work
export LAMMPS_MACE_WORKDIR="$PWD"

git clone --branch patch_30Sep2026 --depth 1 https://github.com/lammps/lammps.git
git clone --branch v0.3.13 --depth 1 https://github.com/ACEsuit/mace.git

git -C lammps rev-parse HEAD
git -C mace rev-parse HEAD

The tested source revisions were:

  • LAMMPS: 0165101ce505734e8fb249694b22e295a7ab4182.

  • MACE: b5faaa076c49778fc17493edfecebcabeb960155.

Use these releases for this example. An unpinned develop checkout can change both the build and the Python interface.

As you complete the steps, the directory layout will be:

lammps-mace-work/
├── lammps/              # source
├── mace/                # source
├── constraints.txt      # tested Python version constraints
├── lammps_mace_venv/     # Python environment
├── environment.sh       # shared build and runtime environment
├── wheelhouse/          # offline Python wheel-build tools
├── models/              # original and converted model
├── build/               # CMake build
├── lammps-mace/          # installed executable and shared library
├── build.sbatch         # build job
├── check_runtime.py     # GPU and Python interface checks
├── job.sbatch           # simulation job
└── test/                # LAMMPS input and output

2. Prepare the Python environment and downloads

Load modules and create the environment

Run from LAMMPS_MACE_WORKDIR:

module purge
module load cmake/3.27.9
module load gcc/12.2.0
module load cuda/12.2
module load openmpi/4.1.6--gcc--12.2.0-cuda-12.2
module load fftw/3.3.10--openmpi--4.1.6--gcc--12.2.0-spack0.22
module load python/3.11.7

export PYTHONNOUSERSITE=1
python -m venv lammps_mace_venv
source lammps_mace_venv/bin/activate
python -m pip install --upgrade pip

Check python --version and python -m pip --version: Python should report 3.11, and pip should point into lammps_mace_venv. Use this same environment when building LAMMPS and running its Python interface.

Resolve the runtime packages before compiling

Save the following as constraints.txt in the project root:

numpy==1.26.4
e3nn==0.4.4
torch==2.5.1+cu121
cuequivariance==0.4.0
cuequivariance-torch==0.4.0
cuequivariance-ops-torch-cu12==0.4.0
cupy-cuda12x==13.3.0
Cython==3.0.12

Ask pip to resolve MACE and the GPU packages together. First inspect the dry-run report, then install using the same constraints:

cd "$LAMMPS_MACE_WORKDIR"
python -m pip install --dry-run --report resolver-report.json \
  -c constraints.txt --extra-index-url https://download.pytorch.org/whl/cu121 \
  ./mace torch cuequivariance cuequivariance-torch \
  cuequivariance-ops-torch-cu12 cupy-cuda12x Cython

python -m pip install --report installed-report.json \
  -c constraints.txt --extra-index-url https://download.pytorch.org/whl/cu121 \
  ./mace torch cuequivariance cuequivariance-torch \
  cuequivariance-ops-torch-cu12 cupy-cuda12x Cython

python -m pip check
python -m pip freeze --all > requirements-resolved.txt

Continue only if pip check reports no conflicts. The constraints select the main components; requirements-resolved.txt records the full environment, including dependencies selected by pip. Keep it with the build records. Do not change NumPy or Cython after configuring the Python-enabled build.

Note

cuEquivariance 0.8.1 required a newer cuBLAS dependency than the one pinned by PyTorch 2.5.1+cu121 in our installation attempt. Version 0.4.0 passed the runtime and numerical tests below. Using --no-deps would bypass the dependency check; it does not resolve that conflict.

Prepare downloads for the compute node

The allocated compute nodes used in this test could not download from PyPI. LAMMPS’s install-python target creates a temporary environment to build its Python wheel, so download its build requirements now:

mkdir -p wheelhouse models test
python -m pip download --only-binary=:all: --dest wheelhouse \
  -r lammps/python/wheel_requirements.txt

wget -O models/mace-mp-0b3-medium.model \
  https://huggingface.co/mace-foundations/mace-mp-0/resolve/main/mace-mp-0b3-medium.model

printf '%s  %s\n' \
  '2f2be696351ac9e94fbe01cdfb6f017679acdbd2db7645209ef55fec9826b012' \
  'models/mace-mp-0b3-medium.model' | sha256sum --check

The checksum should report OK. Convert the model later, inside a GPU allocation.

3. Save a shared build and runtime environment

Save the following as environment.sh in the project root. Both Slurm jobs will source this file, so their modules, Python interpreter, and library paths stay consistent:

export LAMMPS_MACE_WORKDIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

module purge
module load cmake/3.27.9
module load gcc/12.2.0
module load cuda/12.2
module load openmpi/4.1.6--gcc--12.2.0-cuda-12.2
module load fftw/3.3.10--openmpi--4.1.6--gcc--12.2.0-spack0.22
module load python/3.11.7

export PYTHONNOUSERSITE=1
source "$LAMMPS_MACE_WORKDIR/lammps_mace_venv/bin/activate"
export NVCC_WRAPPER_DEFAULT_COMPILER="$(command -v g++)"
export PATH="$LAMMPS_MACE_WORKDIR/lammps-mace/bin:$PATH"

LMP_CXX_RUNTIME="$(g++ -print-file-name=libstdc++.so.6)"
export LD_LIBRARY_PATH="$(dirname "$LMP_CXX_RUNTIME"):$LAMMPS_MACE_WORKDIR/lammps-mace/lib64:$LAMMPS_MACE_WORKDIR/lammps-mace/lib:${LD_LIBRARY_PATH:-}"
export LD_PRELOAD="$LMP_CXX_RUNTIME${LD_PRELOAD:+:$LD_PRELOAD}"

export OMP_NUM_THREADS=1
export XDG_CACHE_HOME="$LAMMPS_MACE_WORKDIR/cache"
export TORCH_HOME="$XDG_CACHE_HOME/torch"
export TORCH_EXTENSIONS_DIR="$XDG_CACHE_HOME/torch_extensions"
export CUPY_CACHE_DIR="$XDG_CACHE_HOME/cupy"
export CUDA_CACHE_PATH="$XDG_CACHE_HOME/cuda"

The GCC runtime selection is needed for this Leonardo module combination. During testing, Python loaded an older libstdc++ and importing LAMMPS failed with a missing GLIBCXX_3.4.29 symbol. Adding the GCC 12 library directory to LD_LIBRARY_PATH alone did not fix it; preloading the library selected by g++ did. Reassess this workaround if the centre changes its modules.

4. Compile LAMMPS in an allocation

The build enables only the packages needed by this recipe:

Option

Purpose

kokkos-cuda.cmake preset

Enable KOKKOS/CUDA and select nvcc_wrapper.

BUILD_MPI, BUILD_OMP

MPI and OpenMP support.

BUILD_SHARED_LIBS

A shared LAMMPS library for the Python interface.

PKG_PYTHON

Python support.

PKG_ML-IAP, MLIAP_ENABLE_PYTHON

The Python-enabled ML-IAP interface.

PKG_ML-SNAP

A supporting package enabled in the tested build.

Kokkos_ARCH_AMPERE80, Kokkos_ARCH_ICX

GPU and host architecture settings used on Booster.

Save the following as build.sbatch in the project root. Replace the allocation placeholder with your account. The debug allocation keeps compilation off the login node and provides the same environment used for the GPU test.

--qos=boost_qos_dbg selects Booster’s debug Quality of Service (QoS) within the boost_usr_prod partition. It is intended for short tests and has a maximum walltime of 30 minutes. For production runs, remove or replace this directive with a QoS appropriate to your allocation and adjust the time limit. Check the Leonardo scheduling limits before submitting.

#!/bin/bash
#SBATCH --job-name=lammps-mace-build
#SBATCH --partition=boost_usr_prod
# Debug QoS for short tests; maximum walltime 30 minutes.
#SBATCH --qos=boost_qos_dbg
#SBATCH --account=EUHPC_XXX_XXX
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=8
#SBATCH --gres=gpu:1
#SBATCH --mem=32G
#SBATCH --time=00:30:00

set -euo pipefail
cd "$SLURM_SUBMIT_DIR"
source ./environment.sh

export PIP_CONFIG_FILE=/dev/null
export PIP_NO_INDEX=1
export PIP_FIND_LINKS="$LAMMPS_MACE_WORKDIR/wheelhouse"
export PIP_DISABLE_PIP_VERSION_CHECK=1
export PIP_CACHE_DIR="$LAMMPS_MACE_WORKDIR/cache/pip"

cmake -S lammps/cmake -B build \
  -C lammps/cmake/presets/kokkos-cuda.cmake \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CXX_USE_RESPONSE_FILE_FOR_OBJECTS=OFF \
  -DCMAKE_INSTALL_PREFIX="$LAMMPS_MACE_WORKDIR/lammps-mace" \
  -DPython_EXECUTABLE="$LAMMPS_MACE_WORKDIR/lammps_mace_venv/bin/python" \
  -DBUILD_MPI=ON -DBUILD_SHARED_LIBS=ON -DBUILD_OMP=ON \
  -DPKG_PYTHON=ON -DPKG_ML-IAP=ON -DMLIAP_ENABLE_PYTHON=ON \
  -DPKG_ML-SNAP=ON \
  -DKokkos_ARCH_AMPERE80=ON -DKokkos_ARCH_ICX=ON \
  -DKokkos_ENABLE_OPENMP=ON \
  -DMPI_CXX_COMPILER="$(command -v mpicxx)"

cmake --build build --parallel "$SLURM_CPUS_PER_TASK"
cmake --install build
cmake --build build --target install-python
python -m pip check
python -m pip freeze --all > requirements-final.txt

The CUDA preset selects the compiler wrapper; leave CMAKE_CXX_COMPILER unset. The response-file option avoids a linking failure encountered with nvcc_wrapper on this toolchain. Additional packages such as KSPACE or MOLECULE can be enabled later if your input needs them.

Submit from the project root:

cd "$LAMMPS_MACE_WORKDIR"
sbatch build.sbatch

Wait for successful completion before submitting the simulation job. Check the Slurm output for the intended Python interpreter, CUDA toolkit, and MPI implementation, and confirm that lammps-mace/bin/lmp was installed. If compilation exceeds the debug time limit, request a suitable allocation; rerunning the same script reuses completed build objects.

5. Verify the runtime environment

Save the following as check_runtime.py in the project root. The simulation job in the next step runs it on the allocated GPU before converting the model:

from importlib import metadata

import cupy as cp
import cuequivariance_torch
import mace
import torch
from lammps import lammps

for package in ["torch", "mace-torch", "cuequivariance", "cupy-cuda12x"]:
    print(f"{package}:", metadata.version(package))

assert torch.cuda.is_available(), "Run this check in a GPU allocation"
assert torch.cuda.device_count() == 1, "This example expects one visible GPU"
print("GPU:", torch.cuda.get_device_name(0))
print("PyTorch CUDA runtime:", torch.version.cuda)
assert torch.ones((2, 2), device="cuda").sum().item() == 4.0
assert cp.arange(10).sum().item() == 45
cp.cuda.runtime.deviceSynchronize()

engine = lammps(cmdargs=["-log", "none", "-screen", "none"])
try:
    required = {"KOKKOS", "ML-IAP", "ML-SNAP", "PYTHON"}
    assert required.issubset(set(engine.installed_packages))
finally:
    engine.close()

print("Runtime checks passed")

This checks actual CUDA operations and loads the LAMMPS shared library. An import-only check cannot establish that the GPU runtime works.

6. Run a test simulation

Create the LAMMPS input

Save the following as test/in.cu_mace_mliap_kk. It creates a periodic copper system with 2,048 atoms and runs 1,000 time steps:

units           metal
atom_style      atomic
boundary        p p p
newton          on

lattice         fcc 3.615
region          box block 0 8 0 8 0 8
create_box      1 box
create_atoms    1 box
mass            1 63.546

pair_style      mliap unified mace-mp-0b3-medium.model-mliap_lammps.pt 0
pair_coeff      * * Cu

neighbor        2.0 bin
neigh_modify    every 1 delay 0 check yes
velocity        all create 300.0 12345 mom yes rot no dist gaussian

timestep        0.001
thermo          100
thermo_style    custom step atoms temp pe etotal press vol
fix             1 all nvt temp 300 300 0.1
run             1000

Create the Slurm job script

Save the following as job.sbatch in the project root. Replace the allocation placeholder as in the build job. Submit both jobs from the project root so SLURM_SUBMIT_DIR identifies the same directory.

#!/bin/bash
#SBATCH --job-name=lammps-mace-test
#SBATCH --partition=boost_usr_prod
# Debug QoS for short tests; maximum walltime 30 minutes.
#SBATCH --qos=boost_qos_dbg
#SBATCH --account=EUHPC_XXX_XXX
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=8
#SBATCH --gres=gpu:1
#SBATCH --mem=32G
#SBATCH --time=00:30:00

set -euo pipefail
cd "$SLURM_SUBMIT_DIR"
source ./environment.sh

nvidia-smi
srun python check_runtime.py
srun python mace/mace/cli/create_lammps_model.py \
  models/mace-mp-0b3-medium.model --format=mliap </dev/null

ln -sfn ../models/mace-mp-0b3-medium.model-mliap_lammps.pt \
  test/mace-mp-0b3-medium.model-mliap_lammps.pt

srun --chdir="$LAMMPS_MACE_WORKDIR/test" \
  lmp -k on g 1 -sf kk -pk kokkos newton on neigh half \
  -in in.cu_mace_mliap_kk -log log.lammps

MACE 0.3.13’s converter detects the available GPU. This model has a single default head, so conversion can run without interactive input. The converted filename is the original model filename followed by -mliap_lammps.pt.

Submit and inspect the result

cd "$LAMMPS_MACE_WORKDIR"
sbatch job.sbatch

After the job finishes, inspect its Slurm output and test/log.lammps. Confirm that the runtime checks passed, the model loaded, and the simulation reached step 1,000 without errors.

What was validated

This stack completed the 2,048-atom, 1,000-step copper run on one A100 GPU. A separate numerical check compared LAMMPS ML-IAP/KOKKOS/cuEquivariance against MACE’s Python/e3nn backend using the same original model, double precision, and a displaced 32-atom periodic copper configuration:

Quantity

Absolute difference

Total energy

2.79 × 10⁻¹² eV

Largest force-component difference

3.06 × 10⁻¹² eV/Å

Both passed a tolerance of 10⁻⁶ in their respective units. We tested this model on one GPU; multi-GPU scaling and other models were not tested.

Completion checklist

  • Source revisions match the tested releases, and the model checksum matches.

  • pip check reports no dependency conflicts before and after compilation.

  • CMake uses the intended Python, MPI, CUDA, and KOKKOS compiler wrapper.

  • The runtime check performs CUDA operations and loads the LAMMPS library.

  • Model conversion succeeds inside a GPU allocation.

  • The Slurm job uses your allocation and completes step 1,000.

Further reading