Training machine learning models has become a task so computationally expensive that it no longer suffices to utilize a single GPU. Using multi-GPU computer architectures has become the state-of-the-art, however it introduces new challenges, for instance it is usually harder to optimize in a distributed setting and generalization abilities of the models suffer.

The student has successfully taken classed on machine learning or data-mining, including for instance "Maschinelles Lernen", "Wissensentdeckung in Datenbanken" or a Fachprojekt, Projektgruppe, Proseminar, Seminar or Bachelor-Thesis in that area of research.

The student can write python code.

At LS8, a machine with 4 GPUs is available, but often only a single GPU is used. However, extremely large-scale learning tasks have to be solved.

In this "Studienarbeit", the student is asked to perform a literature review on techniques for large-batchsize, multi-GPU deep learning, and apply a technique on a given large-scale learning task, thereby demonstrating the benefits and down-sides of multi-gpu learning. Many software-frameworks are readily available, we suggest using extensions of pyTorch.