{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "eb299b08-7ca7-444e-9c28-5796d6c8b110",
   "metadata": {},
   "source": [
    "# Setting Up Programming Environment\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "256c50a5-2118-4e84-9332-ea14d4b5c0b7",
   "metadata": {},
   "source": [
    "In order to run hands-on exercises in this course, you need the Python package and several depenencies.\n",
    "- If you use your own computer to run exercises, you should follow the instructions described below to install relevant packages and setup specific programming environment before running hands-on exercises.\n",
    "- You can use an HPC cluster if you have access to one to run hands-on exercises. Below we provide a short description to login to the [**LUMI**](https://www.lumi.csc.fi/public/) cluster, load the modules, and run interactive/batch jobs."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "115d51c8-ff50-4180-a9e8-2f83ed582b2e",
   "metadata": {},
   "source": [
    "## Local Installation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d5ffe2d-e7ad-40c0-8c3e-6111bc285e2a",
   "metadata": {},
   "source": [
    "### Install miniforge"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f1ff55ef-480c-4259-96cb-3ac79226e14e",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "If you already have a preferred way to manage Python versions and libraries, you can stick to that. If not, we recommend that you install Python3 and all libraries using Miniforge, a free minimal installer for the package, dependency and environment manager conda.\n",
    "\n",
    "Please follow the [installation instructions](https://conda-forge.org/download/) to install Miniforge.\n",
    "\n",
    "Make sure that conda is correctly installed:\n",
    "\n",
    "```console\n",
    "$ conda --version\n",
    "conda 24.11.2\n",
    "````"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f8b6248-1894-4aff-96d4-db6241e50fb4",
   "metadata": {},
   "source": [
    "### Install python programming environment on personal computer"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e954d9fe-e3e1-4d9a-a4f4-978d836834e5",
   "metadata": {},
   "source": [
    "**For Mac users**\n",
    "\n",
    "With conda installed, install the required dependencies by running:\n",
    "```console\n",
    "$ conda env create --yes -f https://raw.githubusercontent.com/ENCCS/hpda-python/main/content/env/environment.yml\n",
    "```\n",
    "\n",
    "This will create a new environment pyhpda which you need to activate by:\n",
    "```console\n",
    "$ conda activate pyhpda\n",
    "```\n",
    "\n",
    "Ensure that the Python version is fairly recent:\n",
    "```console\n",
    "$ python --version\n",
    "Python 3.12.8\n",
    "```\n",
    "\n",
    "Finally, open Jupyter-Lab in your browser:\n",
    "```console\n",
    "$ jupyter-lab\n",
    "```\n",
    "\n",
    "If you use VS code, you can come to the installed `pyhpda` programming environment via choosing `Select Kernel` ar the upper right corner, `Python Environents` and you will find the pre-installed `pyhpda` programming environment."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9650f838-f566-4dc0-b199-0e79418d06fe",
   "metadata": {},
   "source": [
    "**For Linux users**\n",
    "\n",
    "Please provide detailed instructures for linux users to install packages."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2feaa0c6-6887-4919-b8e6-6aca6cf61bbc",
   "metadata": {},
   "source": [
    "**For Windows users**\n",
    "\n",
    "Please provide detailed instructures for windows users to install packages."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ccec462-c34f-4145-bb43-3f3da96054f5",
   "metadata": {},
   "source": [
    "## Using HPC Cluster"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce6ea35c-7532-49fa-97aa-883ebb32b24b",
   "metadata": {},
   "source": [
    "### LUMI"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37521f9f-796b-4d4b-ac85-f7fe9f57e5e0",
   "metadata": {},
   "source": [
    "#### Login to LUMI cluster"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e132505-f95d-4d6b-883b-5b7f7dd8dede",
   "metadata": {},
   "source": [
    "Follow practical instructions [**HERE**](https://enccs.se/tutorials/2024/02/log-in-to-lumi-cluster/) to get your access to LUMI cluster.\n",
    "- On Step 5, you can login to LUMI cluster through terminal.\n",
    "- On Step 6, you can login to LUMI cluster from the web-interface."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aa5f824f-0d99-423f-8502-40c138e59cbf",
   "metadata": {},
   "source": [
    "#### Running jobs on LUMI cluster"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56e6b7aa-f27f-4377-84d1-76e419b61c85",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "If you want to run an interactive job asking for 1 node, 1 GPU, and 1 hour:\n",
    "\n",
    "```console\n",
    "$ salloc -A project_XXXXX -N 1 -t 1:00:00 -p standard-g --gpus-per-node=1\n",
    "\n",
    "$ srun <some-command>\n",
    "```\n",
    "\n",
    "Exit interactive allocation with `exit`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3180ac9d-e025-49c5-8390-588599e499c0",
   "metadata": {},
   "source": [
    "You can also submit your job with a batch script submit.sh:\n",
    "\n",
    "```bash\n",
    "#!/bin/bash -l\n",
    "#SBATCH --account=project_XXXXX\n",
    "#SBATCH --job-name=example-job\n",
    "#SBATCH --output=examplejob.o%j\n",
    "#SBATCH --error=examplejob.e%j\n",
    "#SBATCH --partition=standard-g\n",
    "#SBATCH --nodes=1\n",
    "#SBATCH --gpus-per-node=1\n",
    "#SBATCH --ntasks-per-node=1\n",
    "#SBATCH --time=1:00:00\n",
    "\n",
    "srun <some_command>\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e7f7b1fc-5c17-4ad6-ae70-2fbb95cd378f",
   "metadata": {},
   "source": [
    "Some useful commands are listed below:\n",
    "- Submit the job: `sbatch submit.sh`\n",
    "- Monitor your job: `squeue --me`\n",
    "- Kill job: `scancel <JOB_ID>`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "162e5901-d6fc-413e-a543-d451c26d0020",
   "metadata": {},
   "source": [
    "#### Using `pyhpda` programming environment on LUMI cluster"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "049c05ca-dada-424c-870b-87974b0770ff",
   "metadata": {},
   "source": [
    "We have installed the `pyhpda` programming environment on LUMI. You can follow instructions below to activate it and login to LUMI cluster, either via terminal or through the web-interface.\n",
    "\n",
    "**Login to LUMI cluster via terminal** and then the commands below to check and activate the pyhpda environment.\n",
    "You should get similar output as shown below.\n",
    "\n",
    "```console\n",
    "$ /projappl/projecXXXXX10/miniconda3/bin/conda init\n",
    "\n",
    "$ source ~/.bashrc\n",
    "\n",
    "$ which conda\n",
    "/project/project_XXXXX/miniconda3/condabin/conda\n",
    "\n",
    "$ conda activate pyhpda\n",
    "\n",
    "$ which python\n",
    "/project/project_XXXXX/miniconda3/envs/pyhpda/bin/python\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb1f2c22-d40b-4b45-843f-600e3a40f68e",
   "metadata": {},
   "source": [
    "**Login to LUMI cluster via [web-interface]()** and then select `Jupyter` (not `Jupyter for courses`) icon for an interactive session, and provide the following values in the form to launch the jupyter lab app.\n",
    "- Project: `project_XXXXX`\n",
    "- Partition: `interactive`\n",
    "- Number of CPU cores: `2`\n",
    "- Time: `4:00:00`\n",
    "- Working directory: `/projappl/project_XXXXX`\n",
    "- Python: `Custom`\n",
    "- Path to python: `/project/project_XXXXX/miniconda3/envs/pyhpda/bin/python`\n",
    "- `check` for *Enable system installed packages on venv creation*\n",
    "- `check` for Enable packages under *~/.local/lib on venv start*\n",
    "- Click the `Launch` button, wait for minutes until your requested session was created.\n",
    "- Click the `Connect to Jupyter` button, and then select the Python kernel `Python 3 (venv)` for the created Jupyter notebooks."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8aa74f41-7534-4011-8071-82212bc92eb0",
   "metadata": {},
   "source": [
    "### Leonardo Booster"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "38fb1020-5a61-4263-b060-4f09f49502ed",
   "metadata": {},
   "source": [
    "#### Login to Leonardo Booster cluster"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8792c807-a7eb-4659-8efb-b9bd2927c95f",
   "metadata": {},
   "source": [
    "Follow instructions at [HERE](https://enccs.se/news/2023/09/how-to-login-to-leonardo-supercomputer/) to get your access to Leonardo Booster cluster."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2896759d-9e66-4e97-94ad-5164386fd488",
   "metadata": {},
   "source": [
    "#### Running jobs on Leonardo Booster cluster"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f261832b-f789-4e69-867d-aee830a5fa9f",
   "metadata": {},
   "source": [
    "Here are instructions to run jobs on Leonardo Booster cluster\n",
    "- xxx\n",
    "- xxx"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0dd4a7f-4447-4c3a-b44f-b1893fece85b",
   "metadata": {},
   "source": [
    "#### Using `pyhpda` programming environment on Leonardo Booster cluster"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19bbe498-d79a-455e-a5b0-da3b5992bb6c",
   "metadata": {},
   "source": [
    "Here are instructions to install the `pyhpda` programming environment on Leonardo Booster cluster\n",
    "- xxx\n",
    "- xxx"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}