At a Glance
- Tasks: Join our Ninja Team to migrate big data from Hadoop to AWS.
- Company: HCLTech, a fast-growing global tech company with diverse talent.
- Benefits: Flexible virtual-first work environment and comprehensive employee benefits.
- Other info: Diverse workplace with opportunities for growth and global collaboration.
- Why this job: Be part of a transformational journey in cutting-edge data technology.
- Qualifications: Experience with Hadoop, AWS, and big data technologies required.
The predicted salary is between 63000 - 77000 $ per year.
We are seeking a Data Engineer (Hadoop+Teradata+AWS) - Scale Migration Specialist to join our Data Team to drive the successful migration of on-premise Hadoop big data infrastructure to AWS, ensuring scalability, performance, and data integrity throughout the process.
The Data Engineer will be a key player in our Ninja Team, responsible for designing, developing, and implementing data processing and migration strategies. The role involves working with various big data technologies and AWS services to facilitate a seamless transition from on-premise Hadoop to a cloud-based environment. This includes data migration, optimization, and integration tasks to ensure robust and scalable data solutions.
What we're looking for:
- Bachelor's or Master's degree in Engineering, Computer Science, or a related field.
- 6-10 years of experience with the following technologies: Spark, Oracle 11g, Hive, HDFS, Hadoop, AWS services.
- 4-6 years of experience specifically with data migration and big data technologies.
- Expertise in Apache Spark and AWS EMR.
- Strong experience with AWS services such as S3, Glue, and Lake Formation.
- Proficiency in HDFS, Hive, and data migration strategies.
- Solid understanding of data processing and optimization techniques.
- Experience with other AWS services such as Redshift and Snowflake.
- Familiarity with Hadoop ecosystem tools like Sqoop, Zookeeper, and Yarn.
- Knowledge of data governance and security best practices.
- AWS Certified Data Analytics – Specialty or AWS Certified Solutions Architect – Associate/Professional.
- Cloudera Certified Professional (CCP) in Data Engineering (preferred).
- Experienced in big data, Hadoop, Data warehousing concepts, creating pipelines in HDFS and Teradata framework using RDV Meta, Data Pipeline Studio and Alation tools.
- Experienced in working with AWS Neo ingestion framework and data validation.
- Expertise in creating the data model and source to target mapping design for processing data and loading to AWS S3 through ingestion framework.
- Expertise in writing and debugging complex SQLs in Hive, Spark, Teradata and Oracle databases.
Why Us:
- We are one of the fastest-growing large tech companies in the world, with offices in 50+ countries across the globe and 219,000 employees.
- Our company is extremely diverse with 165 nationalities represented.
- We offer the opportunity to work with colleagues across the globe.
- We offer a virtual-first work environment, promoting a good work-life integration and real flexibility.
- We offer comprehensive benefits for all employees.
- We are a certified great place to work and a top employer in 17 countries, offering a positive work environment that values employee recognition and respect.
Equality & Opportunity for All:
Representing 165 nationalities across the globe, we pride ourselves on being an equal opportunity employer, committed to providing equal employment opportunities to all applicants and employees regardless of race, religion, sex, colour, age, national origin, pregnancy, sexual orientation, physical disability or genetic information, military or veteran status, Aboriginal and Torres Strait Islander people or any other protected classification, in accordance with federal, state, and/or local law.
Data Engineer (Hadoop+Teradata+AWS) in Sydney employer: HCLTech
HCLTech Australia is an exceptional employer that fosters a dynamic and inclusive work culture, where NOC Engineers can thrive in a collaborative environment. With a strong focus on employee growth, we offer extensive training and development opportunities, ensuring you stay at the forefront of SD-WAN, Cisco, and Juniper technologies. Located in Australia, our 24x7 Network Operations Centre provides a unique chance to work with cutting-edge technology while enjoying a supportive team atmosphere and flexible shift rotations.
