Overview
Job Description for Lead Data Engineer in Sydney
Responsibilities
- Leads data engineering initiatives with a focus on data warehousing, data analysis, visualization and data modeling.
- Designs and implements data solutions and pipelines using Python, Scala and SQL.
- Works with cloud platforms (Azure and Databricks) to build and maintain data infrastructure.
- Leverages Big Data technologies (Cloudera: HDFS, Spark, Hive, Kafka) and Apache Airflow for data workflows.
- Applies DevOps practices with Azure DevOps (Git, Repos, Pipelines), Git, Terraform and Docker.
- Supports reporting needs using SQL Server Reporting Services and Power BI.
Qualifications
- 10+ years of experience in software engineering, cloud computing, Databricks and Big Data.
- Data Engineering, Data Warehousing, Solution Design, Data Analysis, Visualization and Data Modeling.
- Programming: Python, Scala, SQL.
- Cloud: Azure and Databricks.
- Big Data: Cloudera (HDFS, Spark, Hive, Kafka), Apache Airflow.
- DevOps: Azure DevOps (Git, Repos, Pipelines), Git, Terraform, Docker.
- Reporting: SQL Server Reporting Services, Power BI.
- Certified DevOps Engineer Expert, Azure Cloud, Azure Data Lake.
Lead Data Engineer employer: Pyramid Global Technologies
Pyramid Global Technologies is an exceptional employer, offering a dynamic work culture in the heart of Sydney that fosters innovation and collaboration. Employees benefit from comprehensive training programs and growth opportunities within the cybersecurity domain, ensuring they stay at the forefront of industry advancements. With a commitment to work-life balance and a supportive team environment, this role provides a meaningful chance to contribute to critical projects while developing your career.
Contact Details:
Pyramid Global Technologies Recruitment Team
