Big Data Engineer at Select Minds LLC in MI-Lansing-48933 Hybrid - US

Job description

• We are seeking an experienced Big Data Engineer to design, develop, and optimize enterprise-scale big data solutions. The ideal candidate will have strong hands-on expertise in the Hadoop ecosystem, Spark, Scala, Hive, Apache NiFi, and Kafka, along with experience building scalable data pipelines and supporting modern CI/CD practices. Candidates should have worked directly with clients, participated in requirement gathering sessions, and delivered end-to-end Big Data implementations.

Responsibilities

• * Design, develop, and optimize large-scale data processing pipelines using Spark, Scala, and Hive.
• * Build and maintain data ingestion workflows using Apache NiFi.
• * Develop and support real-time data streaming solutions using Apache Kafka.
• * Monitor application logs and system health using Kibana, Elasticsearch, and Logstash.
• * Implement and support CI/CD pipelines using Git, Jenkins, and Nexus.
• * Ensure data quality, integrity, and governance across Big Data platforms.
• * Participate in architecture discussions and recommend scalable design solutions.
• * Troubleshoot and resolve complex production issues across the Big Data stack.
• * Collaborate with cross-functional teams in an Agile environment.
• * Mentor junior engineers and contribute to technical best practices.

Requirements

• * 8+ years of experience in Big Data technologies.
• * Strong hands-on experience with Spark, Scala, Hive (HQL), and the Hadoop ecosystem.
• * Experience implementing Apache NiFi and Apache Kafka.
• * Proficiency in Shell Scripting or Python.
• * Experience with PostgreSQL.
• * Familiarity with scheduling tools such as Oozie and Control-M.
• * Working knowledge of Git, Jenkins, Nexus, Maven, Ansible, and SonarQube.
• * Strong understanding of CI/CD practices.
• * Experience in client-facing roles, including requirement gathering workshops.
• * Minimum of two end-to-end Big Data implementation projects.
• * Ability to work independently and contribute effectively within Agile teams.

Skills

Required

• Spark
• Scala
• Hive (HQL)
• Hadoop ecosystem
• Apache NiFi
• Apache Kafka
• Shell Scripting or Python
• PostgreSQL
• Oozie
• Control-M
• Git
• Jenkins

Benefits

• ONSITE
• Competitive salary
• Opportunity for advancement
• Flexible work from home options available.
• Compensation: $110,000.00 - $125,000.00 per year