CASTIEL2 Multi-GPU AI
The lesson materials
1M: Access to Leonardo
1A: Introduction to Deep Learning
2M: PyTorch Distributed Data Parallel
2A: Model parallelism with Pytorch
Code
Slides
3M: PyTorch Lightning
3A: Fine-tuning neural networks
3A: Accelerate FSDP Fine-Tuning
4M: Computer Vision with CNN
4A: MLOps on HPC
5M: Introduction to Ray
5M: Retrieval Augmented Generation (RAG)
5A: Hyperparameter tuning
Reference
Quick Reference
Instructor’s guide
Directives
CASTIEL2 Multi-GPU AI
2A: Model parallelism with Pytorch
Edit on GitHub
2A: Model parallelism with Pytorch
Code
Note
content/hu/exercise
Slides
MP_MGPU.pdf