0. GPU programming

Objectives

Understand GPU programming concepts.
Present an overview of GPU programming models and libraries.
Introduce a few well-known deep learning libraries.

Deep learning libraries

There are many software libraries available for deep learning including:

TensorFlow

TensorFlow was developed by Google and is one of the older deep learning libraries, ported across many languages since it was first released to the public in 2015. It is very versatile and capable of much more than deep learning but as a result it often takes a lot more lines of code to write deep learning operations in TensorFlow than in other libraries. It offers (almost) seamless integration with GPU accelerators and Google’s own TPU (Tensor Processing Unit) chips that are built specially for machine learning.

PyTorch

PyTorch was developed by Facebook in 2016 and is a popular choice for deep learning applications. It was developed for Python from the start and feels a lot more “pythonic” than TensorFlow. Like TensorFlow it was designed to do more than just deep learning and offers some very low level interfaces. PyTorch Lightning offers a higher level interface to PyTorch to set up experiments. Like TensorFlow it is also very easy to integrate PyTorch with a GPU. In many benchmarks it outperforms the other libraries.

Keras

Keras is designed to be easy to use and usually requires fewer lines of code than other libraries. We have chosen it for this lesson for that reason. Keras can actually work on top of TensorFlow (and several other libraries), hiding away the complexities of TensorFlow while still allowing you to make use of their features.

The processing speed of Keras is sometimes not as high as with other libraries and if you are going to move on to create very large networks using very large datasets then you might want to consider one of the other libraries. But for many applications, the difference will not be enough to worry about and the time you will save with simpler code will exceed what you will save by having the code run a little faster.

Keras also benefits from a very good set of online documentation and a large user community. You will find that most of the concepts from Keras translate very well across to the other libraries if you wish to learn them at a later date.

Installing Keras and other dependencies

Follow the setup instructions to install Keras, Seaborn and scikit-learn.

Testing Keras Installation

Keras is available as a module within TensorFlow, as described in the setup instructions. Let’s therefore check whether you have a suitable version of TensorFlow installed. Open up a new Jupyter notebook or interactive python console and run the following commands:

import tensorflow
print(tensorflow.__version__)

2.17.0

You should get a version number reported. At the time of writing 2.17.0 is the latest version.

Testing Seaborn Installation

Lets check you have a suitable version of seaborn installed. In your Jupyter notebook or interactive python console run the following commands:

import seaborn
print(seaborn.__version__)

0.13.2

You should get a version number reported. At the time of writing 0.13.2 is the latest version.

Testing scikit-learn Installation

Lets check you have a suitable version of scikit-learn installed. In your Jupyter notebook or interactive python console run the following commands:

import sklearn
print(sklearn.__version__)

1.5.1

You should get a version number reported. At the time of writing 1.5.1 is the latest version.

Keypoints

TODO…