Design, Implement and Operationalize distributed, scalable, and reliable data flows that ingest, process, store, and access data at scale in batch / real-time. Develop distributed applications on-prem as well as on Cloud that scale to serve analytics, rules, web-applications, models, and Visualizations for end-users. Partner with stakeholders to develop and analyze features at scale. Provide SME level interface for team members to optimize their workflows, streamline operationalization and reduce time-to-market. Contribute to metadata management, Data modelling and documentation. Contribute to adoption of CI/CD and Data Ops practices. Define and Implement metrics to monitor data flow performance as well as optimize when necessary. Evaluate state-of-art data platforms and technologies and prototype solutions. Take part in building the organisation, including hiring and performance management. Take part in defining goals and priorities in your scope, ensuring alignment with business objectives. Collaborate with stakeholders to understand priorities and ensure proper delivery and quality of delivery. Take part on defining strategic direction for MLOps initiatives and roadmap. Communicate progress, challenges, and milestones to senior leadership. Ensure compliance with data security, privacy, and regulatory requirements. Foster a collaborative and inclusive team environment to drive innovation and high performance. Experienced professional with 5+ years of experience working towards design, architecture, development, and operationalization of data flows across Hadoop eco-system, Spark (Databricks or otherwise), Snowflake and Cloud platform(s) Experience working on cloud platforms - Azure (Databricks, Snowflake), AWS, and their respective offerings. Experience in developing large scale Distributed data-driven applications leveraging technologies defined above Experience and understanding across key SQL and NoSQL datastores - HDFS, S3, Snowflake, MongoDB, Splunk as well as In-memory datastores. Experience and understanding of machine learning and advanced analytics models. Proven understanding of the overall Data and Model deployment lifecycle and processing pipelines including orchestration, workflow scheduling tools, monitoring, optimization Programming Languages - Expertise in Python, Advanced SQL, and Shell (Scripting) Expertise in Data analytics and Data wrangling through complex and optimized Python / Spark / SQL Ability to work in Fast paced and Dynamic environment. Good written and verbal communication skills Strong understanding of the importance of data governance, metadata management, and data lineage around feature engineering. Our values - putting clients first, doing the right thing, leading with exceptional ideas, committing to diversity and inclusion, and giving back - aren't just beliefs, they guide the decisions we make every day to do what's best for our clients, communities and more than 80,000 employees in 1,200 offices across 42 countries. Our teams are relentless collaborators and creative thinkers, fueled by their diverse backgrounds and experiences. We are proud to support our employees and their families at every point along their work-life journey, offering some of the most attractive and comprehensive employee benefits and perks in the industry. There's also ample opportunity to move about the business for those who show passion and grit in their work. If this role is deemed a Certified role and may require the role holder to hold mandatory regulatory qualifications or the minimum qualifications to meet internal company benchmarks. Flexible work statement Interested in flexible working opportunities? Speak to our recruitment team to find out more. We work to provide a supportive and inclusive environment where all individuals can maximize their full potential. Our skilled and creative workforce is comprised of individuals drawn from a broad cross section of the global communities in which we operate and who reflect a variety of backgrounds, talents, perspectives, and experiences. Our strong commitment to a culture of inclusion is evident through our constant focus on recruiting, developing, and advancing individuals based on their skills and talents.
Want more jobs like this?
Get Software Engineering jobs delivered to your inbox every week.
