ML Process Maturity Model
Overview
A framework for assessing how well software teams practice machine learning, based on six dimensions applied to each stage of the ML workflow.
The Six Dimensions
For each workflow stage, assess whether it:
| Dimension | Question |
|---|---|
| Goals Defined | Does the team have clear goals for this activity? |
| Consistently Implemented | Does the team do this activity the same way each time? |
| Documented | Are practices related to this activity written down? |
| Automated | Is this activity mostly automated? |
| Measured & Tracked | Does the team measure effectiveness at this activity? |
| Continuously Improved | Does the team actively improve practices here? |
These dimensions are loosely based on CMM (Capability Maturity Model) and Six Sigma concepts.
Activity Maturity Index (AMI)
The AMI is the average score across all six dimensions for a given workflow stage. The study found strong correlation (0.50–0.76) between AMI and self-reported activity effectiveness, suggesting it captures something real.
Why This Matters
Different teams need different interventions. A team struggling with undefined goals needs different help than a team with good processes that aren’t automated. The framework enables:
- Self-assessment of current state
- Identification of specific improvement areas
- Comparison across workflow stages (data activities vs. model activities)
- Tracking progress over time
Limitations
The model is descriptive (how much automation exists?) rather than prescriptive (do X to reach the next level). Future work would need to define distinct maturity levels like CMM does.
When to Use
- Assessing a new ML team’s readiness
- Identifying bottlenecks in an existing workflow
- Prioritizing process improvement investments
- Benchmarking against other teams
Related: 05-atom—nine-stage-ml-workflow, 03-atom—experience-shifts-challenge-perception