Phase 3: Automation
Phase 3: CI/CD/CT (Automation)
In this phase, we automate the boring stuff. We move from manual updates to pipelines that respond to changes in code or data.
🟢 Level 1: Continuous Integration (CI) for ML
CI in ML is more than just linting code. It’s about validating the Logic and the Data Schema.
1. The ML Test Suite
- Code Tests: Standard Pytest for utility functions.
- Data Tests: Pandera or Great Expectations (e.g., “Feature X should never be negative”).
- Model Tests: Unit tests for the model output (e.g., “The model should return a probability between 0 and 1”).
🟡 Level 2: Continuous Deployment (CD)
Deploying a model isn’t just updating a binary. It’s about updating an API.
2. Deployment Strategies
- Shadow Deployment: Run the new model alongside the old one. Log results but only return the old model’s output to the user.
- Canary Deployment: Send 5% of traffic to the new model. If it doesn’t crash, increase to 100%.
🔴 Level 3: Continuous Training (CT)
This is the “Holy Grail” of MLOps. The pipeline automatically retrains the model when new data arrives or performance drops.
3. Triggers for CT
- Schedule: Retrain every Sunday.
- Event: New data landed in S3.
- Metric: Model accuracy in production dropped below 80%.