Principal Site Reliability Engineer, Machine Learning at Cambridge Mobile Telematics in Cambridge, MA

Job description

• CMT is looking for a Principal Site Reliability Engineer I, Machine Learning to help us change the world. CMT has helped protect over 65 million drivers and prevent over 126,000 crashes worldwide. We build AI to solve some of the most difficult challenges in mobility — understanding and reducing risk, detecting crashes, and getting people life-saving help. The problems are hard. The impact is real. No matter your role, your work will matter at CMT.
• CMT is looking for a collaborative, customer-committed, and creative SRE team member who wants to join us in making roads safer by making drivers better! This is a hybrid role based in Cambridge, MA, with two days per week onsite.

Responsibilities

• Use independent judgment and discretion to own SLOs, error budgets, and the operational health of Ray clusters running on AWS EKS and Databricks workloads on AWS EC2 across multiple accounts and regions
• Maintain the observability of uptime, availability, and scalability of EKS Ray and Databricks workloads using CloudWatch and Datadog, including defining alerting that maps to SLOs
• Operate and tune EKS Ray workloads at scale including autoscaling, GPU scheduling, and automated failure recovery
• Manage Databricks on AWS including workspace administration, cluster policies, Unity Catalog, job orchestration, and IAM Roles and Policies
• Maintain ongoing cost visibility, cost optimization, and capacity planning across EC2 and EKS workloads, including through the use of On Demand Capacity Reservations and Spot lifecycle
• Perform ongoing maintenance of the underlying EC2 and EKS infrastructure, including regular security updates and operating system upgrades
• Codify everything as infrastructure-as-code using Terraform and CI/CD pipelines, enabling updates through Pull Requests with approval workflows, while also automating maintenance tasks to reduce toil
• Lead incident response for Data Science and Machine Learning platform outages, run blameless postmortems, and drive systemic remediation, including participating in an on-call rotation
• Complete any additional tasks as they arise

Requirements

• Bachelor’s degree or equivalent years of experience and/or certification in a related field
• 7+ years working in Site Reliability Engineering or Information Technology
• Design and document systems, including writing and reviewing code, to automate away problems within your team’s domain
• Intermediate to expert experience deploying and maintaining AWS services such as EC2, ECS, EKS, SQS, Lambda, Dynamo, RDS/Aurora, S3, and IAM
• Intermediate to expert experience monitoring services and applications using tools such as CloudWatch Metrics, CloudWatch Logs, and Datadog, including defining and configuring alerts and SLO reports
• Intermediate to expert experience maintaining the uptime and scalability of AWS compute services used for Machine Learning and Data Science workloads, specifically EC2 and EKS
• Intermediate to expert coding skills in at least one programming language; we work primarily in Python
• Intermediate to expert experience using Infrastructure as Code platforms and CI/CD pipelines, specifically Terraform, to manage AWS infrastructure and services
• In-depth knowledge & experience with Linux operating systems (Amazon Linux, Ubuntu) on EC2 and Docker / Kubernetes
• Experience with leading projects in system design, architecture changes, and technology selection
• Ability to work a hybrid schedule from our Cambridge, MA office two days per week

Skills

Required

• Infrastructure as Code: Terraform
• CI/CD pipelines
• Docker / Kubernetes
• System design and architecture
• SLOs and error budgets
• Incident response and on-call

Benefits

• Work on a mission with real impact: crashes prevented, injuries avoided, lives protected around the world
• Join an industry leader — 65 million drivers protected, powering 140+ programs across 25 countries
• Recognized innovator in mobility AI, earning top honors including the TIME Industry Leader in AI, a Gold Edison Award, and the Artificial Intelligence Excellence Award for AI for Social Good. CMT is also Great Place to Work Certified
• Be part of the team inventing the future of mobility and road safety
• Move fast, own outcomes, do work that matters
• High ownership, small teams, and direct access to leadership — no layers between your work and its impact
• Unlimited PTO, flexible scheduling, competitive salary, annual performance bonus, RSUs, and full benefits including medical, dental, vision, and 401k match
• Summer Fridays provide team members with half days to recharge
• Join one of our employee resource groups: Black, AAPI, LGBTQIA+, Women, Book Club, and Health & Wellness
• Comprehensive wellness, education, and employee assistance programs
• Fair and competitive salary based on skills and experience, and annual performance bonus
• Equity may be awarded in the form of Restricted Stock Units (RSUs)
• Medical, Dental, Vision and Life Insurance, matching 401k, short-term & long-term disability and parental leave
• Unlimited Paid Time Off including vacation, sick days & public holidays
• Flexible scheduling and work from home policy depending on role and responsibilities
• The base salary range for this position is: $142,000 to $177,600. This range is specifically for Cambridge, MA