Skip to main content

Training Deep Neural Network(DNN) from Scratch

1. Google Colab​

Deep learning is an expensive practice. It only burst about 10 years ago into our lives because GPUs became strong enough to allow the magic it is. As most of us do not possess a computer with a GPU, Google offers a free platform that allows you to use their cloud GPUs. Weak as they might be, they are still powerful enough to ease our training processes and make them 10x faster. This will be crucial for the later exercises of the course. Therefore, we recommend that you become familiar with it early. However, exercises 1-5 do not require such capabilities.

In order to use the platform, open a folder in your Google Drive main page, under the name TPDNN within another folder name YouthInAI for consistency with the exercises. In there, paste the exercises folders. Then, you could simply open the notebooks with the colab-notebook. There, you should follow the instructions we've assembled for you in each notebook.

Pay attention to the fact that files require a few seconds to save to the Colab cloud disk. Therefore, run the zipping cell in the notebook after you've waited a few seconds, letting the previous cell save your models to the disk. Otherwise, you will have trouble submitting your code without your models.

NOTE: Pytorch does NOT support MacBooks with the M1 or M2 cpus. Therefore, in order to utilize a GPU --> use Colab.

2. The directory layout for the lab exercises​

The exercises are organized to work with the file structure shown below. By unzipping the first exercise, you automatically get some of the folders. For the remaining exercises, you need to download and unzip the exercise folder and place it in the YouthInAI/TPDNN/ folder.

3. Dataset Download​

Datasets will generally be downloaded automatically by exercise notebooks and stored in a common datasets directory shared among all exercises.

4. Acknowledgments​

We want to thank the Stanford Vision Lab, TU Munich Chair of Computer Graphics and Visualization and PyTorch for allowing us to build these lab exercises on material they had previously developed.