Senior Data Engineer (Microsoft Fabric, PySpark, SQL)

Build a centralised, trusted data warehouse for internal technology insights using Microsoft Fabric. Own complex engineering work across PySpark, SQL, medallion architecture and source-system integration. Apply AI-powered engineering thoughtfully across coding, testing, deployment and documentation.

Be part of the team driving enterprise-wide transformation at CommBank. In the End User Experience team, you'll help bring data from internal technology sources into a centralized Microsoft Fabric warehouse, creating a controlled and reusable foundation for reporting, analytics and data sharing across the organisation.

You'll join a collaborative, Sydney-based team working across data engineering, analytics and business stakeholders. Your focus will be to design, build and improve scalable data solutions that turn complex source data into trusted, analytics-ready products.

• Design and build data ingestion and transformation workloads in Microsoft Fabric using PySpark and advanced SQL.
• Apply medallion architecture principles across Bronze, Silver and Gold layers to create governed, reusable data products.
• Connect to and ingest data from APIs, web services, relational databases, files, cloud storage and event-based sources.
• Design analytics-friendly data models, including facts, dimensions and curated datasets.
• Implement data quality checks, reconciliation controls, validation, monitoring and robust error handling.
• Translate business and technical requirements into data models, business rules and practical engineering solutions.
• Use source control, automated testing, continuous integration and delivery practices, and clear technical documentation.
• Collaborate with engineers, analysts, BI developers, architects and stakeholders in an Agile delivery environment.

We're interested in hearing from people who:

• Bring strong hands-on data engineering experience and can work confidently across design, development, troubleshooting and optimisation.
• Have strong hands-on experience with Microsoft Fabric, PySpark and advanced SQL.
• Have extensive experience applying medallion architecture to enterprise data solutions.
• Can develop and troubleshoot scalable ingestion, transformation and API-based integration patterns across multiple environments.
• Understand data modelling, data lineage, metadata management, data quality and reconciliation controls.
• Use generative AI and coding-assistant tools responsibly, including model selection, prompt and context design, cost awareness, output evaluation, security and human review.
• Are familiar with Git, automated testing, continuous integration and delivery, and release management for data engineering solutions.
• Can gather requirements and translate them into durable data models, rules and engineering outcomes.
• Experience with T-SQL and dynamic SQL is desirable.

At CommBank, we're committed to creating an accessible, inclusive and respectful workplace. If you require support or adjustments, please let us know. We welcome applications from people of all backgrounds and we're particularly committed to making a positive difference for Aboriginal and/or Torres Strait Islander Peoples.