Week 5: Introduction to Deep Learning in Python

An introduction to deep learning in Python by Dr. Yi-Xin Liu at Fudan University (lyx@fudan.edu.cn).

This is a part of the course: Road to Scientific Research: Powerful Computer Applications (XDSY118019.01).

Lecture date: 2024.10.10

Introduction to Deep Learning

• Deep Learning In 5 Minutes | What Is Deep Learning? | Deep Learning Explained Simply | Simplilearn (length: 5:51)

Fundamentals of Deep Learning

• 3Blue1Brown - But what is a neural network? | Chapter 1, Deep learning (length: 19:13)

Fundamentals of Deep Learning

• 3Blue1Brown - Gradient descent, how neural networks learn | Chapter 2, Deep learning (length: 21:00)

Fundamentals of Deep Learning

• 3Blue1Brown - What is backpropagation really doing? | Chapter 3, Deep learning (length: 13:54)

Fundamentals of Deep Learning

• 3Blue1Brown - Backpropagation calculus | Chapter 4, Deep learning (length: 10:17)

Deep Learning Experiments

Do the following experiments in A neural network playgound (http://playground.tensorflow.org/):

• Choose Gaussian data, use linear activation function to train.
• Choose Circle data, use linear activation function to train. Does it succeed? How to obtain a successful training?
• (Optional) Choose Spiral data, find a successful classification neural network model.

Deep Learning with Pytorch

Why PyTorch?

PyTorch is currently the most popular deep learning framework not only in Python but all programming languages.

• See the overall trend for PyTorch and other competitors (TensorFlow, JAX, MxNet, Caffe2, MindSpore, etc.) here.
• Watch the video for the current state of machine learning frameworks here.

Introduction to PyTorch

• Introduction to PyTorch (length: 23:22)

Additional Resources for PyTorch

• PyTorch Beginner Series
• Youtube: Learn PyTorch for deep learning in a day. Literally.
• Online Course: Learn PyTorch for Deep Learning: Zero to Mastery

A Simple Walkthrough

Go to 03_python_deep_learning.ipynb.

More Resources for Machine Learning

Why Biological Neurons Are Deep Neural Networks (Youtube Video)

General Machine Learning

• scikit-learn: Machine learning in Python
• Gradient Boost: XGBoost, LightGBM, CatBoost
• Kaggle: Largest AI & ML community, competitions, datasets, notebooks, etc.
• Hugging Face: AI community for ML models and datasets

LLM

• Transformer
• Ollama: open-source machine learning models
• ChatGPT: AI-powered chatbot by OpenAI