Data Scientist/ Engineer (Hybrid -Boston) at Shift in US - Boston

Job description

• Shift delivers AI agents that transform insurers' most critical work. By combining deep industry expertise and unmatched data resources, Shift provides proven results that have earned the trust of hundreds of the world's leading insurers. Our insurance-grade AI is accurate, explainable, and secure—empowering human experts to move with unmatched speed, total confidence, and a renewed focus on the people they serve.
• Our culture is built on innovation, trust, and a drive to transform the insurance industry through our SaaS platform. We come from more than 50 different countries and cultures and together we are creating the future of insurance.
• Learn more at www.shift-technology.com

Responsibilities

• Your role will be to actively contribute to the US- Insurance roadmap and clients, and working on various data types such as structured data, free text, documents and images.
• Build and productionize data pipelines (structured, text, documents, images) optimized for LLMs and multi-modal models.
• Design, develop and deploy LLM-based solutions (RAG, embeddings, instruction tuning) for subrogation, claims handling, document understanding, and related use cases.
• Experiment with the latest in Agentic AI technologies (Langchain/Langgraph, OpenAI Agent SDK, MCP, A2A) and develop MVP for the next generation of autonomous subrogation solutions.
• Develop "Chain-of-Thought" and "ReAct" prompting strategies to ensure the agent can justify its liability percentages based on the Comparative Negligence laws of different jurisdictions.
• Create custom "tools" for the agent, allowing it to query internal databases, call external weather APIs, or calculate impact force based on telemetry data.
• Establish rigorous evaluation frameworks (LLM-as-a-judge) to ensure the agent’s decisions are unbiased, legally sound, and explainable.
• Ensure responsible-AI practices: privacy, hallucination mitigation, explainability and compliance.
• Lead client workshops, present prototypes, gather feedback and help define roadmap priorities.

Requirements

• We are looking for candidates with diverse skills to help us build excellent technology solutions for our clients and be proficient in the following skills:
• Expert proficiency in production-level object-oriented programming (OOP) for building scalable and reliable systems.
• Proven hands-on experience with Large Language Models (LLMs) and generative AI techniques (including RAG, embeddings, prompt engineering, and model tuning), leveraging frameworks such as OpenAI/Anthropic or open-source variants.
• Solid foundation in ML fundamentals with practical experience in the full machine learning lifecycle, including model evaluation, monitoring, versioning, and deployment in production environments.
• Experience designing and implementing robust data pipelines for document, OCR, and multi-modal data workflows.
• Agentic Frameworks: Experience with integrating frameworks like LangChain/LangGraph, OpenAI Agent SDK, CrewAI, A2A/MCP with Databricks or Azure-hosted models (e.g., DBRX, OpenAI GPT-5.3, Anthropic Claude, Google Gemini).
• Demonstrated ability to effectively engage with clients, translate complex business needs into clear, actionable technical solutions, and manage stakeholder expectations.
• Databricks Ecosystem: Deep expertise in Mosaic AI (formerly MosaicML), Unity Catalog, and Delta Lake. Good understanding of Spark data architecture is a plus.
• Experience using MLflow for the full lifecycle: from experiment tracking and prompt engineering in the AI Playground to model evaluation.

Skills

Preferred

• Understanding of Spark data architecture

Benefits

• To support our permanent, full time employees at every stage of their careers and lives, we provide a competitive total rewards and benefits package. Here are the global benefits we’d like to highlight:
• Flexible remote and hybrid working options
• Competitive Salary and a variable component tied to personal and company performance
• Multiple Learning and Development opportunities, including Focus Fridays, a half-day each month to focus on learning and personal growth
• Generous PTO and paid holidays
• Mental health benefits
• 2 MAD Days per year (Make A Difference Days for paid volunteering)
• Additional benefits may be offered by country, based on your eligibility - ask your recruiter for more information. Intern and Apprentice positions may receive some of these benefits - ask your recruiter for more details.
• The range listed is for base compensation. Your actual base salary will vary based on factors including location and individual qualifications objectively assessed during the interview process.
• In addition to base salary, your total rewards package will include additional components such as incentive pay and benefits. If you're interviewing for this role, speak with your Talent Acquisition Partner to learn more about the specific details for this position.
• Base Salary Pay Range
• $100,000 - $110,000 USD