At a Glance
- Tasks: Design and optimise high throughput data pipelines using Azure Databricks, Spark, and SQL.
- Company: Join a leading tech firm in Sydney focused on innovative data solutions.
- Benefits: Enjoy a competitive salary, flexible work options, and opportunities for professional growth.
- Other info: Dynamic team environment with excellent career advancement opportunities.
- Why this job: Be at the forefront of data engineering and make a real impact in the tech industry.
- Qualifications: Experience with Scala, Spark, SQL, and Python; strong communication skills are a plus.
The predicted salary is between 123750 - 151250 $ per year.
Location: Sydney
Mode: Permanent
Primary or Mandatory skills:
- Scala
- Spark
- SQL
- Python
Good to have skills:
- Databricks
- SQL
- Python
- Data warehousing concepts
Detailed Job description (Role & Responsibility, Skill and Experience range):
We are looking for a hands-on Developer Programmer to design, build, and optimize high throughput data pipelines on Azure Databricks using Spark (Scala) and SQL.
Design & develop scalable ETL/ELT pipelines in Azure Databricks using Spark (Scala) and SQL, following medallion architecture (Bronze/Silver/Gold) and Delta Lake best practices.
Proven expertise in designing real-time and near real-time integrations.
Operationalize workloads: jobs & workflows scheduling, CI/CD with Git (e.g., Azure DevOps), secrets management, and observability.
Expertise in cloud platforms and related technologies, specifically: Data Platforms (e.g. Databricks) and big data technologies.
Real-time data processing frameworks: Spark Streaming, Apache/Confluent Kafka.
Proficiency in SQL, Python, and data integration tools.
Familiarity with Master Data Management solutions.
Excellent communication and stakeholder management skills.
Data Engineer in Sydney employer: CareCone Group
CareCone Group is an exceptional employer located in the vibrant city of Sydney, offering a dynamic work culture that fosters innovation and collaboration. Employees benefit from ongoing professional development opportunities, competitive remuneration, and a supportive environment that values work-life balance. Joining our team means being part of a forward-thinking organisation dedicated to delivering high-quality IT solutions while ensuring compliance with Australian working rights.
