At a Glance
- Tasks: Design and develop scalable data pipelines using Databricks and AWS.
- Company: Leading tech firm in Brisbane focused on data engineering solutions.
- Benefits: Competitive salary, career growth, and hands-on experience with cutting-edge technologies.
- Other info: Opportunity to work on innovative projects in a collaborative environment.
- Why this job: Join a dynamic team and make an impact in the data engineering field.
- Qualifications: 7+ years of experience in data engineering, AWS, and Databricks.
The predicted salary is between 123750 - 151250 $ per year.
We are seeking an experienced Senior Data Engineer with strong hands‑on expertise in Databricks and AWS to design, develop, optimize, and support enterprise‑grade data engineering solutions. The ideal candidate will have extensive experience in building scalable data pipelines, implementing modern data transformation frameworks, and maintaining reliable cloud‑based data platforms.
The role combines platform engineering with production Data Business As Usual (BAU) support, requiring strong capabilities in monitoring data pipelines, resolving production incidents, maintaining data quality, and ensuring compliance with operational SLAs. A solid understanding of ITIL and IT Service Management (ITSM) practices is essential for maintaining platform stability and driving continuous improvement across production data environments.
Technical Competencies & Requirements
- Data BAU, ITIL & ITSM (Critical): Strong hands‑on experience in Data BAU operations, supporting production data pipelines, monitoring data freshness, and resolving data quality alerts. Thorough understanding of ITIL frameworks and ITSM processes, including Incident, Problem, Change, and Release Management. Experience with ITSM tools such as ServiceNow or Jira Service Management for SLA tracking, incident management, escalation handling, and Root Cause Analysis (RCA) documentation.
- AWS & Databricks Ecosystem: Hands‑on experience with AWS cloud services, particularly Amazon S3. Advanced proficiency in Databricks development using PySpark, including Unity Catalog, Delta Live Tables (DLT)/Lakeflow Declarative Pipelines, Serverless Compute, SQL Warehouses, dashboards, and alerting. Strong understanding of performance optimization techniques, including Liquid Clustering, Z‑Ordering, and Spark query optimization. Practical experience managing deployments and workloads using Databricks Asset Bundles (DABs).
- Data Ecosystem & Infrastructure: Advanced experience with dbt (dbt Core or dbt Cloud) for developing and managing version‑controlled data transformations. Hands‑on experience building and supporting Kafka‑based streaming pipelines for real‑time and near‑real‑time data integration. Proven experience with Terraform for Infrastructure as Code (IaC), including provisioning and managing AWS and Databricks resources. Strong experience with GitLab for source control and implementing automated GitLab CI/CD pipelines. Good working knowledge of Unix/Linux environments, including scripting, troubleshooting, and operational support.
Preferred Qualifications
- Experience working in the Airline or Aviation domain will be highly regarded.
- Databricks certifications will be an added advantage.
Data Engineer in Brisbane employer: Silverlink Technologies
Join a leading technology team in Sydney as a Data Center Control System Engineer, where you will play a crucial role in ensuring the seamless operation of critical data centre facilities. Our company fosters a collaborative work culture that prioritises employee growth through continuous learning opportunities and hands-on experience with cutting-edge technologies. Enjoy competitive benefits and the unique advantage of working in a vibrant city known for its innovation and diversity.
