Assignment Description:
We are looking for a Senior Data Engineer to join our dynamic team.
What You Will Work On:
- Analyze and document existing manual data collection processes, identifying opportunities for optimization and automation.
- Design and implement a robust, scalable, and efficient data engineering and processing pipeline that meets the needs of the organization.
- Select and deploy appropriate tools, technologies, and frameworks for data ingestion, transformation, storage, and processing.
- Ensure the quality, integrity, and consistency of data throughout the pipeline, incorporating validation, error handling, and monitoring mechanisms.
- Ensure that all data engineering practices comply with relevant data protection regulations (e.g., GDPR, CCPA) and internal data governance policies.
- Implement data access controls, encryption, and anonymization techniques to maintain data security and privacy.
- Identify and prioritize the components and tasks necessary for transitioning from manual processes to an automated pipeline.
- Develop comprehensive project plans, including scope, timelines, resource requirements, and risk assessments.
- Lead and manage data engineering projects to ensure on-time delivery within scope, maintaining high quality and compliance standards.
- Collaborate closely with data analysts, data scientists, software engineers, and business stakeholders to ensure alignment and successful project outcomes.
- Establish and maintain data quality metrics and KPIs, and implement automated data quality checks and validation processes.
- Work with data stewards and business users to define and enforce data quality standards across the organization.
What You Bring:
- Educational Background: Bachelor’s degree in Computer Science, Engineering, Data Science, or a related field; a Master’s degree is preferred.
- Experience: 5+ years of experience in data engineering with a strong focus on data pipeline design, implementation, and optimization.
- Technical Proficiency:
- Expertise in programming languages such as Python.
- Experience with data processing frameworks like Apache Spark and Apache Kafka.
- Strong knowledge of SQL and NoSQL database systems and data warehousing solutions.
- Proficiency with cloud platforms (e.g., AWS, Azure, Google Cloud) and related data services.
- Data Compliance and Governance: Solid understanding of data governance, data privacy regulations, and compliance requirements.
- Project Management Skills: Proven ability to manage and deliver complex data engineering projects on time and within scope.
- Analytical and Problem-Solving Skills: Strong capacity to analyze, diagnose, and solve technical and process challenges.
- Data Quality Expertise: Familiarity with data quality tools and best practices for maintaining high standards.
If this sounds interesting, please send your CV to cv@veritaz.se
