Azure Databricks Data Engineer at TALENT Software Services in Louisville, KY

Job description

• Location: Louisville, KY
• Duration: 6 months
• Experience Required: 8–10 years
• Role: Databricks / Azure Data Engineer

Responsibilities

• Work on Agile development projects and participate in sprint delivery.
• Design and implement data integration solutions in Microsoft Azure.
• Develop and maintain data pipelines using Azure Data Factory.
• Ingest data from various source systems into Azure.
• Perform data transformations using Azure Databricks and Delta Lake.
• Implement and maintain workflows using Prefect.
• Perform data analysis and troubleshooting for data-related issues.
• Assist in identifying and resolving data quality and pipeline issues.
• Collaborate with development teams to support CI/CD processes using Azure DevOps.
• Act as an individual contributor and problem solver.
• Participate in Agile ceremonies, sprint planning, and delivery activities.

Requirements

• Experience developing and implementing data pipelines using Azure Data Factory (ADF).
• Experience ingesting data from multiple sources using Azure ADF.
• Experience transforming data using Azure Databricks and Delta Lake.
• Stories
• Bugs
• Issue management
• Experience implementing Prefect.
• Strong hands-on experience with Databricks.
• Experience working with Microsoft Azure integration solutions.
• Experience working in Agile development environments.
• Experience with sprint planning and sprint delivery.
• Strong analytical and problem-solving skills.
• Ability to work independently as an individual contributor.

Skills

Required

• Azure Data Factory (ADF)
• Azure Databricks
• Delta Lake
• Azure DevOps
• Prefect
• Microsoft Azure integration
• Agile development
• Sprint planning
• Data analysis
• Problem-solving