We are seeking a skilled GCP Data Engineer to design, develop, and optimize scalable data pipelines and cloud-based data solutions on Google Cloud Platform (GCP). The ideal candidate should have strong experience in data engineering, ETL development, big data technologies, and cloud-native services within GCP.
Key Responsibilities
- Design, build, and maintain scalable ETL/ELT data pipelines using GCP services.
- Develop and manage data ingestion, transformation, and integration processes from multiple data sources.
- Implement and optimize data solutions using BigQuery, Dataflow, Dataproc, Cloud Storage, Pub/Sub, and Composer.
- Build and maintain data warehouses, data lakes, and data marts on GCP.
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements.
- Ensure data quality, governance, security, and compliance standards.
- Monitor, troubleshoot, and optimize data pipeline performance and costs.
- Automate deployment processes using CI/CD tools and Infrastructure as Code (Terraform preferred).
- Support real-time and batch data processing requirements.
Required Skills & Qualifications
- Bachelor's degree in Computer Science, Information Technology, or related field.
- 4+ years of experience in Data Engineering.
- Strong experience with Google Cloud Platform (GCP) services:
- BigQuery
- Dataflow
- Pub/Sub
- Cloud Composer (Airflow)
- Cloud Functions
- Strong SQL and Python programming skills.
- Experience building ETL/ELT pipelines.
- Hands-on experience with Apache Spark, Hadoop, or similar big data technologies.
- Knowledge of data modeling, data warehousing, and dimensional modeling.
- Experience with version control systems such as Git.
- Understanding of CI/CD and DevOps practices.
Preferred Qualifications
- GCP Professional Data Engineer Certification.
- Experience with Terraform and Infrastructure as Code (IaC).
- Familiarity with Kafka, Airflow, or distributed data processing frameworks.
- Experience working in Agile/Scrum environments.
- Exposure to machine learning data pipelines is a plus.
Key Technologies
- Cloud: GCP
- Data Warehouse: BigQuery
- Programming: Python, SQL
GCP Data Engineer in Sydney employer: N2S.Global
Join a dynamic team that values innovation and collaboration, where your expertise in Golang will be instrumental in building scalable backend services. Our vibrant work culture fosters continuous learning and growth, offering ample opportunities for professional development while working with cutting-edge technologies in a supportive environment. Located in a thriving tech hub, we provide a unique chance to engage with like-minded professionals and contribute to impactful projects that make a difference.
