This course was created with the
course builder. Create your online course today.
Start now
Create your course
with
Autoplay
Autocomplete
Previous Lesson
Complete and Continue
Testing & Monitoring Machine Learning Model Deployments
Introduction
Course Introduction (3:08)
Course Curriculum (2:35)
Course Requirements (1:36)
Approaching This Course (Important) (3:26)
Complete Course Notes
All Course Slides
FAQ: I would like to learn more about the topics not covered
Course Scenario & Model Lifecycle
Deploying a Model to Production (8:31)
Course Scenario - Predicting House Prices (9:27)
Setup A - Python Install (Do Not Skip) (3:47)
Setup B: Git Installation (Advanced Users Can Skip) (3:02)
Course Github Repo & Data (2:38)
Download Data and Github Link
Setup C: Install Jupyter (Advanced users can skip) (2:13)
Setup D: Install Initial Dependencies (Advanced users can skip) (2:19)
Introduction to the Dataset & Model Pipeline (13:21)
ML System Lifecycle (5:51)
Testing Concepts for ML Systems
Overview (0:48)
Testing Focus in this Course (1:25)
Why Test? (3:44)
The Value of Testing
Testing Theory (3:47)
Testing ML Systems (Important) (6:31)
Testing Concepts - Exercise 1 Instructions (12:07)
Testing Concepts Exercise 1 - Solution (3:13)
Unit Testing Concepts - Exercise 2 Instructions (4:24)
Testing Concepts Exercise 2 - Solution (4:04)
Testing Concepts - Exercise 3 Instructions (5:11)
Testing Concepts Exercise 3 - Solution (5:04)
Testing Concepts Exercise 4 - Instructions (3:23)
Testing Concepts Exercise 4 - Solution (1:27)
Summary (0:26)
Unit Testing ML Systems
Overview (0:45)
Python Code Conventions (2:26)
Intro to Pytest (11:49)
Setup - Download Dataset from Kaggle (3:22)
Using Tox (5:47)
Codebase Overview (13:41)
Preprocessing & Feature Engineering Testing Theory (3:24)
Unit Testing Preprocessing & Feature Engineering Code (11:06)
Git Hygiene
Config Tests Theory (3:00)
Unit Testing Config Code (9:57)
Testing Input Data Theory (3:06)
Unit Testing Input Data Code (8:35)
Testing Model Quality Theory (2:19)
Unit Testing Model Quality Code (10:10)
Repo Tooling (2:41)
Wrap Up (1:41)
Docker Refresher
Docker Section Overview (0:45)
Docker Recap (6:09)
Why Use Docker? (7:24)
Introduction to Docker Compose (4:28)
Docker & Docker Compose Installation (5:56)
[Windows only] Docker Setup (3:48)
Docker Exercise Instructions (3:29)
Docker Exercise Solution (3:23)
Integration Testing ML Systems
Overview (0:40)
API Conceptual Overview (2:16)
Integration Testing - Code Base (6:47)
Using the API Part 1 (1:55)
Windows Specific Docker Setup
Using the API Part 2 (2:56)
Integration Tests Theory (1:52)
Integration Tests Code (10:21)
Integration Tests Benchmark Theory (1:33)
Differential Testing
Overview (0:32)
Differential Testing Theory (3:19)
Differential Testing Implementation (7:37)
Shadow Mode
Shadow Mode Overview (0:44)
Shadow Mode Theory (4:23)
Testing Models in Production (9:32)
Tests in Shadow Deployments (15:08)
Shadow Mode Code Overview - DB Setup (13:13)
Shadow Mode - Setup Tests (11:40)
Shadow Mode - Asynchronous Implementation (4:25)
Populate Database with Shadow Predictions (5:22)
Jupyter Demo - Setup (5:02)
Jupyter Demo - Tests in Shadow Mode (14:18)
Monitoring Metrics with Prometheus
Overview (1:36)
Why Monitor ML Models? (5:34)
Monitoring Theory (8:29)
Metrics for ML Systems (6:03)
Prometheus & Grafana Overview (6:42)
WINDOWS Setup - More Port Mapping (2:27)
Basic Prometheus Setup (5:33)
Adding Prometheus Metrics (8:22)
Setup Grafana (7:21)
Infrastructure Level Metrics (6:44)
Adding Metrics Monitoring to Our Example Project (7:30)
Creating an ML System Grafana Dashboard (15:44)
Monitoring Logs with Kibana
Monitoring - Logs for Machine Learning (4:03)
Monitoring - Elastic Stack Overview (4:41)
Kibana Exercise (9:43)
Integrating Kibana into our Example Project (9:36)
Setting Up a Kibana Dashboard for Model Inputs (14:03)
Conclusion
Conclusion (1:01)
The Value of Testing
Lesson content locked
If you're already enrolled,
you'll need to login
.
Enroll in Course to Unlock