Big Data Engineer

Software Engineer Big Data with 4+ yrs to 15 Years of experience

4+ years of professional experience in Big data platform. Responsible for producing excellent quality of code, adhering to expected coding standards and industry best practices.

Key Responsibilities:

• Design, develop and optimize big data processing pipelines using Spark/Scala and Hive (HQL).
• Implement and manage data ingestion workflows using Apache NiFi.
• Develop data streaming solutions leveraging Apache Kafka.
• Monitor and analyse logs and metrics using Kibana, Elasticsearch, and Logstash.
• Collaborate with DevOps teams to implement Continuous Integration and Continuous Deployment (CI/CD) pipelines using Git, Jenkins, and Nexus.
• Ensure data quality, consistency, and governance across big data platforms.
• Participate in architecture discussions and recommend best practices for data processing and pipeline design.
...