Senior Lead Software Engineer - Java & Databricks at JPMorgan Chase & Co. in Bengaluru, Karnataka, India

Job description

• Be an integral part of an agile team that's constantly pushing the envelope to enhance, build, and deliver top-notch technology products.
• As a Senior Lead Software Engineer at JPMorganChase within the Digital Communications Compliance team, you are an integral part of an agile team that works to enhance, build, and deliver trusted market-leading technology products in a secure, stable, and scalable way. Drive significant business impact through your capabilities and contributions, and apply deep technical expertise and problem-solving methodologies to tackle a diverse array of challenges that span multiple technologies and applications.

Responsibilities

• Design and develop scalable, fault-tolerant microservices and APIs that support rule-based and ML-based detection pipelines
• Model and implement supervision and reviewer workflows using state machines
• Build streaming and batch data pipelines that ingest, index, and enrich communications content and alerts
• Design data models on Databricks for surveillance, search, retention, and life cycle management at scale
• Develop and optimize large-scale batch and streaming data pipelines using Databricks, Apache Spark, and Delta Lake
• Build robust unit, integration, and performance tests following Test-Driven Development. Leverage Databricks Workflows, notebooks, and CI/CD pipelines to automate data processing and deployments. Troubleshoot and tune Databricks jobs, clusters, and data pipelines for efficiency and performance
• Drive adoption of Databricks best practices for data engineering, software engineering, observability, and platform operations
• Drives adoption and governance of approved AI-assisted engineering practices across teams to improve code quality, delivery speed, and operational outcomes (e.g., AI-assisted code review/refactoring, test acceleration, release readiness, incident/root-cause analysis), while establishing measurable validation standards (secure coding, peer review, automated testing) and promoting reuse of proven patterns and automation within the SDLC/TLM toolchain.
• Applies knowledge of tools within the Software Development Life Cycle toolchain, including approved AI-assisted development and automation capabilities, to improve the value realized by automation at scale.
• Partner with product management and compliance SMEs to monitor and improve alert accuracy and reliability. Proactively identify hidden problems and patterns in communications data to improve detection quality

Requirements

• Formal training or certification on software engineering concepts and 5+ years applied experience
• Hands-on practical experience delivering system design, application development, testing, and operational stability
• Expertise in building resilient, scalable, enterprise-grade cloud-native products, with strong exposure in compliance for the financial industry
• Hands-on Databricks experience including development of production workloads using Spark, Delta Lake, and Databricks Workflows. Expert Java/Kotlin and Python programmer with experience building headless, externally consumable APIs. Experience building cloud-native microservices for streaming and batch architectures using Spark and/or Flink
• Proficient with AWS services including EC2, ECS, EKS, EMR, S3, and Glacier. Hands-on with Semantic search, Kafka, and PostgreSQL. Experience integrating and operationalizing ML/LLM models and pipelines in production. Experience with observability and monitoring tools such as Prometheus, Grafana, and OpenTelemetry
• Experience building CI/CD pipelines using ArgoCD, Helm, Terraform, Jenkins, and GitHub Actions
• Prior experience in Test-Driven Development, delivering products with well-defined SLI/SLO/SLAs
• Strong ownership mentality with excellent communication skills and a collaborative mindset. Experience mentoring engineers and providing technical leadership at a senior level
• Exposure to Unity Catalog, and Databricks Workflows
• Exposure to building and optimizing large-scale batch and streaming data pipelines in cloud-native environments
• Understanding of distributed systems, data engineering best practices, and operationalizing data and ML workloads in production

Skills

Required

• Java
• Kotlin
• Python
• Databricks
• Apache Spark
• Delta Lake
• Databricks Workflows
• AWS (EC2, ECS, EKS, EMR, S3, Glacier)
• Kafka
• PostgreSQL
• Semantic search
• ML/LLM models

Preferred

• Unity Catalog
• Distributed systems
• Data engineering best practices