Data Engineer-GCP

• 2 days ago Be among the first 25 applicants
• Direct message the job poster from ALOIS Australia
• Talent Acquisition Specialist at ALOIS Solutions
• Role: Data Engineer with GCP
• Role Type: Contract

Experience level - 7+ years total in Data Engineering; 4+ years on GCP with production systems.

• Top 3 Must‐Have Skillsets
• GCP Big Data Engineering (Big Query + Dataform + Pub/Sub)
• Expert in designing and optimising Big Query schemas, partitioning/clustering, cost/performance tuning, query optimisation, and policy tag integration.
• Building streaming and batch pipelines using Apache Beam/Dataflow and Pub/Sub with exactly-once semantics, backpressure handling, and replay strategies.
• Strong experience with Dataform (or similar) for SQL-based transformations, dependency graphs, unit tests, and multi-environment deployments.
• Production-grade Python for ETL/ELT, distributed processing, robust error handling, and testable modular design.
• Designing resilient Airflow DAGs on Cloud Composer: dependency management, retries, SLAs, sensors, service accounts, and secrets.
• Monitoring, alerting, and Cloud Logging/Stackdriver integration for end-to-end pipeline observability.
• Data Security & Governance on GCP
• Hands‐on with Dataplex (asset management, data quality, lineage), Big Query policy tags, Cloud IAM (least privilege, fine‐grained access), KMS (key rotation, envelope encryption), and audit trails via Cloud Logging.
• Practical experience implementing PII controls (data masking, tokenisation, attribute‐based access control) and privacy‐by‐design in pipelines.
• Cloud Run & APIs: Building stateless microservices for data access/serving layers, implementing REST/g RPC endpoints, authentication/authorisation, rate limiting.
• Data Modelling: Telecom‐centric event models (e. g., CDRs, network telemetry, session/flow data), star/snowflake schemas, and lakehouse best practices.
• Performance Engineering: Big Query slot management, materialised views, BI Engine, partition pruning, cache strategies.
• Secure Source Manager (CI/CD): Pipeline‐as‐code, automated tests, artifact versioning, environment promotion, canary releases, and Git Ops patterns.
• Data Quality & Testing: Great Expectations/Deequ‐like checks, schema contracts, anomaly detection, and automated data validations in CI/CD.
• Streaming Patterns: Exactly‐once delivery, idempotent sinks, watermarking, late data handling, windowing strategies.
• Observability & SRE Practices: Metrics, logs, traces, runbooks, SLIs/SLOs for data platforms, major incident response to support Dev Ops.
• Cost Governance: Big Query cost controls, slot commitments/reservations, workload management, storage lifecycle policies.
• Domain Knowledge (Mobile Networks): Familiarity with 3G/4G/5G network data, OSS/BSS integrations, network KPIs, and typical analytics use cases.

Experience Level

• 7+ years total in data engineering; 4+ years on GCP with production systems.

• Evidence of impact

• Led end-to-end delivery of large‐scale pipelines (batch + streaming) with strict PII governance.
• Owned performance/cost optimisation initiatives in Big Query/Dataflow at scale.
• Implemented CI/CD for data workflows (Secure Source Manager) including automated tests and environment promotion.
• Drove operational excellence (SLAs, incident management, RTO/RPO awareness, DR patterns).
• Soft skills: Technical leadership, code reviews, mentoring, clear documentation, cross‐functional collaboration with Network/Analytics teams, and a bias for automation & reliability.
• Seniority level
• Mid‐Senior level
• Employment type
• Contract
• Job function
• Consulting
• Industries
• IT Services and IT Consulting