- Design, build, and optimise batch and streaming data pipelines using approved frameworks and tooling.
- Ensure pipelines are resilient, performant, and cost optimised.
- Implement monitoring, alerting, and automated recovery processes.
- Apply data quality checks and reconciliation processes to ensure accuracy and completeness.
- Maintain metadata, lineage, and documentation to support governance requirements.
- Adhere to role-based access control, data privacy, and security standards.
- Contribute to the development and maintenance of ingestion frameworks, transformation layers, and data models.
- Implement and maintain CI/CD pipelines for automated testing and deployment.
- Optimise data storage and retrieval strategies to meet performance needs.
- Work closely with Technical Leads, Data Architects, Analysts, and business stakeholders to deliver fit-for-purpose solutions.
- Participate in code reviews and knowledge-sharing sessions to improve team capability.
- Explore and adopt new tools, frameworks, and techniques to enhance engineering capability.
- Identify opportunities to automate and streamline workflows.
Key Skills & Experience
- Proficient in SQL for data wrangling, transformation, and analysis
- Experience building ETL/ELT pipelines in cloud or distributed environments
- Familiarity with cloud platforms and services
- Knowledge of data modeling concepts
- Experience with data orchestration tools
- Familiar with version control and CI/CD pipelines
- Understanding of data quality, validation, and observability practices
- Works independently with minimal supervision on well-scoped tasks
- Collaborates effectively with stakeholders
- Communicates clearly and documents work for others to maintain or build upon
- Open to code reviews, feedback, and continuous learning
- Able to troubleshoot and resolve data pipeline failures or performance issues
- 2-4 years in a data engineering or closely related role
- Demonstrated ability to improve code quality, reliability, or performance
#J-18808-Ljbffr
GCP Data Engineer in Sydney employer: Nityo Infotech
Join a dynamic team where innovation meets collaboration! As a Salesforce Developer, you'll thrive in a supportive work culture that values professional growth and offers ample opportunities for skill enhancement. Located in a vibrant area, our company provides competitive benefits and a unique chance to work on cutting-edge projects that make a real impact.
