Principal Data Scientist at Commonwealth Bank in Sydney CBD Area

Job description

• AI & General Model Validation (AIGMV) provides independent technical challenge and review of the Group's most important ML and AI models, across business and support units.
• See yourself in our team

Responsibilities

• As a Principal Data Scientist within the team, you will lead independent review, challenge and validation of ML, statistical and AI models across the Group, including GenAI and agentic systems. You will assess whether they are fit for purpose and operate as intended, and whether the risks they introduce are understood and appropriately controlled. This will involve:
• Scope and design validation of complex ML and AI systems, split the work into a structured plan, and lead other data scientists through it
• Form independent risk opinions and provide credible challenge to senior business, engineering and governance stakeholders
• Set and improve the team's AI validation practices, tooling and standards, and mentor other data scientists

Requirements

• Extensive experience in developing, evaluating or validating ML/AI, GenAI or statistical models, or equivalent depth
• A courageous mindset and independent judgement. You enjoy self-directed learning, and have the curiosity and discipline to stay ahead of fast-changing AI techniques and risks
• Close attention to detail, and the judgement to see which problems matter
• Deep working knowledge of GenAI and agentic architecture, enough to validate and challenge orchestrators, sub-agents, RAG pipelines, tool use, guardrails, multi-agent routing, memory and persona handling, and LLM-as-a-Judge
• Hands-on experience designing, building and evaluating AI and Generative AI solutions. Fine-tuning transformer models is a plus.
• Advanced Python, with strong GitHub, version control and CI/CD experience. SQL, R or TypeScript is a plus.
• Ability to read and navigate unfamiliar codebases, reproduce evaluation pipelines, write test harnesses and inspect trace data
• Hands-on experience with AI coding agents, including Claude Code and Codex, and the ability to build and maintain skills, sub-agents, hooks and plugins that automate workflows.
• Working knowledge of traditional machine learning models, including regression, random forests, XGBoost, and related evaluation methods, and the ability to validate and challenge them.
• Demonstrated experience as the technical lead on projects, leading other data scientists through the work rather than as a people manager, including turning a complex system into a structured test plan
• Strong communication skills, with the ability to explain complex technical concepts to mixed audiences, including senior stakeholders and governance forums.
• Practical experience with PyTorch and AWS SageMaker, and with GenAI tooling such as LangChain, LangGraph, OpenAI Agents SDK, Pydantic and Langfuse/HoneyHive
• Degree in Statistics, Data Science, Computer Science, Actuarial Science, or a related field, or equivalent industry experience

Skills

Required

• Advanced Python
• Strong communication skills
• Close attention to detail

Preferred

• Fine-tuning transformer models
• SQL
• R
• TypeScript