Technology Architect | Analytics - Packages | Python - Big Data at IMCS Group in Charlotte, NC

Job description

• Job Details:

Responsibilities

• Should be able to work with other onsite team members and client
• Should be able to manage offshore resources also.
• Should be able to manage complex software application development and support projects

Requirements

• Python, Pyspark, API, Microservices CI/CD, GCP, ETL
• At least 10 years of experience in IT industry.
• We are seeking a Python-based platform engineer to design and build a containerized API layer that abstracts and governs interactions with Apache Spark through a well-defined API contract. This role focuses on building platform capabilities, not simply consuming existing data tools—enabling consistent, secure, and scalable access to Spark-based data pipelines.
• The ideal candidate has strong experience developing production-grade APIs in Python that interface with data frameworks or pipeline orchestration systems, packaging services using containers (Docker/Kubernetes), and operating them as reusable platform services. A proven background in CI/CD automation using GitHub Actions is required, along with solid software engineering practices around testing, versioning, and deployment.
• Should have good hands-on experience in GCP skills. Should be able to interact, coordinate with the client.
• Should be able to manage offshore resources also.
• Should be able to work with other onsite team members and client.
• This role is suited for engineers who have built data or compute platforms, service layers, or internal frameworks—translating complex Spark capabilities into stable, contract-driven APIs for broad enterprise use.
• Python,
• Pyspark
• • Experience building control planes
• • Design patterns
• o Modular containerized services
• o Asynchronous processing
• o Job-based work queue execution
• o Immutability and idempotency
• o Metadata management
• o Policy enforcement
• o Semantic modeling
• o Observability and telemetry
• o Side effect capability injection
• GCP
• Testing
• • Functional testing
• • Behavioral testing
• Languages
• • Python
• Frameworks and Platforms
• • FastAPI
• • Kubernetes
• • Pytest
• >10 years
• Certifications Needed :No
• ETL

Skills

Required

• Python
• Pyspark
• API
• Microservices CI/CD
• GCP
• ETL