OPTIONAL: Going Parallel

Overview

Teaching: 10 min
Exercises: 20 min

Questions

• How can I use multiple GPUs at once?

• What are the limitations of multi-GPU machine learning?

Objectives

• Implement a multi-GPU training loop

under construction

Key Points

• Using Multiple-GPUs with PyTorch is trivial

• Multi-GPU training is subject to processing bottlenecks.