Ingestion Engineer

Job Description:

As an Ingestion Engineer in the DDF team, your key responsibilities are as follows:

1. Technical Proficiency:

• Collaborate in hands-on development using Databricks, Python, PySpark, ADF, and other relevant technologies to ingest data for downstream consumption.
• Assist in engineering and managing data models and pipelines within a cloud environment, utilizing technologies like Databricks, Spark, Delta Lake, and SQL.
• Contribute to the maintenance and enhancement of our progressive tech stack, which includes Python, PySpark, Logic Apps, Azure Functions, ADLS, Django, and ReactJs.
• Support the implementation of DevOps and CI/CD methodologies to foster agile collaboration and contribute to building robust data solutions.
• Develop code that adheres to high-quality standards, promoting a scalable and maintainable data platform.

• Collaborate with the team to learn and apply the best practices in data engineering.
• Actively participate in engineering projects, gaining experience in developing high-quality, scalable, and sustainable data solutions.
• Stay updated with emerging technologies and trends in data engineering, contributing to the team's knowledge base by sharing insights and ideas.
• Assist in the development of data solutions within the Pet Nutrition data platform, working on challenging aspects under the guidance of senior team members.
• Contribute to the management of data from various divisions to generate valuable data assets related to pets and pet owners.
• Support the maintenance of a semantic and intelligent data layer to contribute to the comprehensive leadership of the data solution within the environment.

• Collaborate closely with analysts, data scientists, and team members to understand their requirements and assist in translating them into actionable data solutions.
• Maintain effective communication with the Data Ingestion Lead, actively participating in team discussions and sharing ideas to improve platform excellence.