We are looking for AWS & Cloudera Administrator with 3-5 years of experience. A
Professional with expertise and passion for building large scale distributed systems and to
setup, configure and maintain the Hadoop clusters including components such as Hive, Impala,
Kafka, Storm, Kudu, Pig, Spark, HDFS, HBase, Oozie, Sqoop, Flume, Zookeeper, etc.
Desired Knowledge and Skills:
Good knowledge of capacity planning phase i.e. for the first time or for scheduled
addition of nodes.
Being involved in the hardware requirement of the nodes, network architecture, and
planning.
Experienced in the configuration management for the nodes
Experienced in Testing and benchmarking the new nodes in Dev, Staging and
Production clusters.
Experienced in cluster management & monitoring via Cloudera Manager / Ambari or
conventional tools like Ganglia or Nagios
Hand-on experience in LINUX based system for software installation, patch upgrades
and shell scripting
Should take care of the standard System Admin work like creating new users in
Hadoop, handling permissions, performing upgrades
Should take care of the Day to Day Cluster issues like finding out which jobs are
taking more time if users say that jobs are stuck to find out the reason
Should take care of the daily supporting of users in their admin needs. This would
involve checking for heavy jobs and if some jobs are taking more time as per the SLA.
Good knowledge in the security administration during installation and for this Kerberos
knowledge is required and also Apache Knox, Apache Sentry basics will be helpful
Handling backups to the metadata of the cluster and other Eco-system metadata
Being involved in data migration between clusters if needed.
Good knowledge on AWS environment and Service knowledge such as AWS EC2, S3,
VPC, VPN, Amazon API Gateway, AWS ECS, Elastic Container Service, AWS RDS,
AWS Route 53 shall be preferred.
Good knowledge on AWS VPC Deployment such as Create VPC, Subnets, Internet
Gateway, VPC peering, NAT Instance, VPN server Instance (Sophos), Create Route
Tables, Network ACL, Cloud watch Alarms shall be preferred.
Other Skills / Responsibilities:
Translating functional and technical requirement into detailed architecture and design
Coding and testing complex system components
Participating in code and design reviews to maintain out high development standards
Able to perform technical root cause analysis and outline corrective action for given
problems
Excellent verbal and written communication skills, demonstrating the ability to
communicate technology concepts to non-technology personnel
Maintain high team morale.
Adhere to company and project standards and guidelines.
Promptly escalate issues, which affect product delivery and quality that are beyond scope of influence and report same to reporting manager.
Pay high attention to detail in all work.
Quality, on time delivery of agreed deliverables.
Educational Qualification:
Bachelor's degree in Computer Science or related field.
