Big Data Cloud Engineer (HDFS, YARN, Kafka, Linux, AWS, Shell scripting)

Key Responsibilities

• Administer, maintain, and support large-scale Hadoop and Cloudera (CDH/CDP) environments across distributed enterprise infrastructures.
• Design, deploy, and manage Hadoop cluster architecture, ensuring high availability and scalability in production environments.
• Manage and optimize Big Data ecosystem components such as HDFS, YARN, Spark, Kafka, Hive, Impala, HBase, Oozie, and Zookeeper.
• Perform cluster monitoring, troubleshooting, and performance tuning to ensure system stability and efficiency.
• Support both real-time and batch data processing platforms using Spark, Kafka Streams, Hive, and HBase.
• Configure and maintain ecosystem tools including Hue, Airflow, NiFi, Sqoop, Datameer, Splunk, and Arcadia.
• Implement and manage security frameworks such as Kerberos, Ranger, Sentry, SSL, and IAM policies.
• Manage AWS cloud services including EC2, S3, EMR, IAM, and VPC for Big Data w...