Assignment Description:
We are looking for a Senior Data Engineer to join our dynamic team.
What you will work on:
- Design, build, and optimize end-to-end data pipelines integrating multiple data sources.
- Develop and maintain data models, ensuring data quality, scalability, and performance.
- Work with Databricks, Unity Catalog, Apache Spark, and Delta Lake to implement advanced data engineering solutions.
- Collaborate with data scientists, software engineers, and project managers to deliver robust data integration and analytical capabilities.
- Utilize Azure Data Factory and Azure Data Lake for data ingestion, transformation, and storage.
- Implement CI/CD pipelines, version control (Git), and DevOps practices for automation and deployment.
- Apply data engineering best practices to ensure governance, security, and efficiency.
- Monitor and optimize cloud usage to improve system performance and cost-effectiveness.
- Contribute to continuous improvement by sharing insights, mentoring peers, and refining development standards.
What you bring:
- Minimum of 5 years of experience in data engineering or software development.
- Strong proficiency in SQL and Python for data manipulation and automation.
- Hands-on experience with Databricks, Unity Catalog, Apache Spark, and Delta Lake.
- Proven track record in building and maintaining ETL/ELT pipelines across diverse data sources.
- Deep understanding of Azure cloud services, including Azure Data Lake and Azure Data Factory, along with cost management expertise.
- Experience implementing CI/CD pipelines, version control (Git), and DevOps methodologies.
- Familiarity with data warehousing, dimensional modeling, and data lakehouse architectures.
- Strong knowledge of data engineering best practices with the ability to advocate and apply them effectively.
- Excellent analytical, problem-solving, and communication skills.
- Ability to work independently while collaborating effectively in cross-functional teams.
- Experience with Power BI is considered a strong advantage.
If this sounds interesting, please send your CV to cv@veritaz.se
