Data Platform Lead
Liam Tran
HR Administrator
Efterfrågade kompetenser
Om tjänsten
Role Summary
Our client is seeking a Data Platform Lead to guide the development and operation of a secure, scalable data platform spanning raw-data ingestion through actionable insights. The assignment focuses on integrating data, analytics, and AI/ML capabilities into services and business processes in close collaboration with data and platform engineering teams.
The platform operates in an AWS-focused hybrid-cloud environment. The role owns the Terraform codebase and the architecture of the Databricks lakehouse and its supporting infrastructure, with responsibility for technical integrity, efficient resource utilization, lifecycle management, and disaster recovery practices. The Data Platform Lead will also help define best-practice architectures for end-to-end data solutions, support quality and performance, enable scalable delivery, and guide strategic technology choices.
Key Responsibilities
- Lead the architecture and continued evolution of a large-scale, secure cloud data platform.
- Own and maintain the Terraform codebase used to provision and manage platform infrastructure.
- Define the architecture of the Databricks lakehouse and its underlying cloud infrastructure.
- Ensure platform-level technical integrity, efficient resource utilization, lifecycle management, resilience, and disaster recovery.
- Collaborate with platform engineers to manage access, audit platform usage, improve maintainability, and support federated data teams.
- Design batch, high-speed API, and event-streaming integrations for high-concurrency telematics applications.
- Establish a unified access-control framework across the data platform.
- Translate business privacy policies into practical data-masking and security controls.
- Define naming standards and implement automated data-contract validation without unnecessarily constraining domain teams.
- Guide architectural and technology decisions while promoting quality, performance, and scalable delivery.
Technical Stack & Requirements
- Cloud environment: AWS-focused hybrid-cloud infrastructure.
- Data platform: Databricks lakehouse architecture and large-scale cloud data-platform design.
- Data processing: Apache Spark, Python, and SQL.
- Infrastructure as code: Hands-on ownership and development of Terraform-based infrastructure.
- Integration patterns: Batch processing, low-latency API integrations, and event-streaming architectures.
- Security: Access control, usage auditing, data masking, privacy controls, and secure serving layers.
- Platform engineering: Resource optimization, infrastructure lifecycle management, maintainability, and disaster recovery.
- Data governance: Naming conventions and automated data-contract checks for federated teams.
Qualifications & Experience
Required qualifications include solid experience designing and deploying large-scale cloud data platforms, strong knowledge of modern data-architecture principles, and hands-on expertise with Databricks, Apache Spark, Python, and SQL. Candidates must also have experience designing secure cloud infrastructure, data-serving layers, and infrastructure-as-code solutions. The role requires the ability to define end-to-end technical architectures collaboratively, evaluate technology choices, and balance centralized platform controls with the autonomy of domain-oriented teams.
Work Logistics & Location
This is a consulting assignment supporting an AWS-focused hybrid-cloud data environment. The physical work location and remote or on-site working arrangement are not specified.