IND Senior Software Engineer

Model Monitoring & Observability

• Collaborate with Data Scientists to onboard models into monitoring systems (e.g., MHM database) and maintain configuration files for drift and performance checks.
• Support retraining workflows when drift or degradation is detected.

• Implement monitoring frameworks to track data drift, training-serving skew, and model performance metrics in production environments.
• Utilize observability tools like Arize to set up monitors for accuracy, precision, recall, and other KPIs.
• Configure automated alerts for anomalies in input/output distributions and performance degradation.

Data Quality Assurance

• Perform QC checks on incoming data to validate integrity and completeness before model scoring.
• Develop pipelines for continuous validation of data sources and ensure compliance with quality standards.

Performance Evaluation

• Monitor KPIs such as accuracy, precision, recall, and fairness across demographic slices.
• Conduct root-cause analysis for performance degradation and recommend retraining strategies.

Infrastructure & Automation

• Build and maintain CI/CD pipelines for deploying monitoring solutions and model updates.
• Leverage cloud technologies for scalable monitoring and orchestration.

Documentation & Reporting

• Maintain detailed logs of monitoring activities, thresholds, and alerts.
  • Provide periodic reports on model health, including drift metrics and performance trends.