Senior Data Engineer

We are looking for a Sr Data Engineer with in-depth expertise in AWS, Databricks, and modern data architecture and data modeling to help build the next generation of our data foundation, including data platforms such as Advisor360, CRM integrations, ETF/MF data pipelines, and other enterprise analytics assets.

This is a senior, highly autonomous role. You will operate as a technical consultant and data architecture thinker, working closely with stakeholders to understand ambiguous business needs, conduct independent investigations, and design solutions that are scalable, reliable, and aligned to both business and technology goals.

Ideal candidates have 5+ years of data engineering experience and thrive in environments where they drive clarity, define standards, and elevate engineering best practices across the team.

Core Responsibilities:

Data Engineering & Architecture

• Design, build, and optimize scalable, secure, and repeatable data pipelines using AWS (S3, Glue, Lambda, Step Functions, Redshift, IAM) and Databricks (PySpark, Lakeflow, Delta Lake, Unity Catalog).
• Serve as the technical leader for data ingestion pipelines, modeling new datasets such as Advisor360, CRM, ETF, and Mutual Fund platforms.
• Apply strong data modeling (dimensional, canonical, and domain-driven) principles to support analytics, reporting, and AI/ML use cases.
• Ensure alignment with enterprise data architecture standards, promoting reusability, governance, and long-term maintainability.

• With limited guidance, independently perform deep investigations, identify data issues, and propose solutions that balance performance, cost, risk, and business needs.
• Engage business stakeholders to gather ambiguous requirements, ask the right questions, and translate them into clear technical designs.
• Provide thought leadership and recommend technical patterns, frameworks, and toolsets.

• Implement robust data quality frameworks, monitoring, and alerting to ensure high trust in business-critical data assets.
• Troubleshoot data inconsistencies and ensure proper logging, testing, and recovery mechanisms across pipelines.
• Lead regression testing, software upgrades, and production deployments with strong change control discipline.

• Lead all phases of solution development-from design to deployment and operationalization.
• Mentor and guide other engineers in coding standards, architecture patterns, Databricks best practices, and AWS platform usage.
• Partner with Data Architecture, Analytics, Product, and Business teams to deliver solutions that improve decision-making.
• Provide training sessions and documentation to uplift the data engineering maturity across the organization.

• Participate in strategic initiatives such as AI readiness, data unification efforts, metadata strategy, and enterprise integration roadmaps.
• Drive continuous improvement in engineering frameworks, onboarding workflows, and platform capability.

• 5+ years of experience in data engineering, data architecture, or large-scale distributed data systems.
• Expert-level experience with cloud platforms such as AWS, GCP, or Azure, leveraging services for data storage, ingestion, pipeline orchestration, database or lake house management, data transformation.
• Strong background in data modeling (dimensional, canonical, data vault, or domain-driven).
• Proven ability to work independently with minimal direction and deliver high-quality solutions in ambiguous environments.
• Demonstrated experience translating complex business problems into scalable technical solutions.
• Strong SQL and Python skills, with emphasis on ETL/ELT pipeline development.
• Experience with CI/CD, GitHub, DevOps workflows, and automated testing.

• Experience in asset management, wealth management, or financial services (ETF, Mutual Funds, CRM, Advisor analytics).
• Experience with enterprise data quality tools and metadata management concepts.
• Familiarity with modern semantic layers, dbt, or domain-oriented data mesh concepts.
• Undergraduate degree or equivalent professional experience in Computer Science, Engineering, Information Systems, or related field.