Lead Software Engineer (Data Engineer)

The Identity Verification group uses machine learning models, large proprietary data sets, and high volume / low latency services to help our customers reduce transaction, account takeover, and financial risks. This position is within our Data Platform team, focused on organizing, transforming, and responsibly governing the data used by our machine learning models and Data Science teams.

Identity Graph & Ingestion team is looking for a Lead Software Engineer who thrives on designing, coding, and maintaining large data processing systems, primarily in the Spark framework using Scala. Our ideal candidate will understand the challenges of working with data at scale and have a firm knowledge of algorithms and processing complexity.

Job Description
Work as part of a Scrum team that is geographically diverse (US west coast and central Europe)
Design, implement, and maintain our big data processing systems using Apache Spark and related technologies which fulfills the needs of Data Science across the org
Create design documents and supporting diagrams. Incorporate feedback and negotiate trade-offs with product, security, data privacy, and other stakeholders
Increase the capabilities and efficiency of our data processing pipeline. Lead Retrospectives and Root Cause Analysis to enable continual improvement
Identify technical and process gaps, provide estimates for business solutions, define the initial scope/requirements of the project, and manage the scope during the lifecycle of the project
Participate and contribute to team's agile process and decisions. Drive prioritization decisions and trade-offs in working with product partners
Develop documentation templates, lead and enforce development of documentation throughout the development lifecycle of the solution.
Conduct technical interviews for hiring engineering staff and raising the performance bar
Mentor software engineers and hold system overview sessions
Advance company engineering culture by identifying and promoting improved technologies and processes.

Required Skills
A strong background in computer science, with a deep understanding of data structures, algorithms, and software engineering principles.
Proficiency in Spark, SQL, and Scala is required.
Hands-on experience with Databricks and Airflow is required.
Familiarity with cloud platforms (preferably AWS) and infrastructure as it relates to data engineering and ETL processes.
Experience working with REST APIs and integrating external data services.
Exposure to machine learning and data science workflows.
Excellent understanding of Agile methodologies, with the ability to drive iterative delivery and cross-team collaboration.
Strong communicator with the ability to explain complex concepts to both technical and non-technical audiences, and to influence stakeholders across product, engineering, and acquisition teams.
Curiosity and willingness to grow beyond your initial domain
Understanding of engineering tradeoffs and designing within constraints
* Bachelor's degree in Computer Science, Software Engineering, or a related field or equivalent practical experience.

Mastercard is a merit-based, inclusive, equal opportunity employer that considers applicants without regard to gender, gender identity, sexual orientation, race, ethnicity, disabled or veteran status, or any other characteristic protected by law. We hire the most qualified candidate for the role. In the US or Canada, if you require accommodations or assistance to complete the online application process or during the recruitment process, please contact reasonable_accommodation@mastercard.com and identify the type of accommodation or assistance you are requesting. Do not include any medical or health information in this email. The Reasonable Accommodations team will respond to your email promptly.

Corporate Security Responsibility

All activities involving access to Mastercard assets, information, and networks comes with an inherent risk to the organization and, therefore, it is expected that every person working for, or on behalf of, Mastercard is responsible for information security and must:

• Abide by Mastercard's security policies and practices;
• Ensure the confidentiality and integrity of the information being accessed;
• Report any suspected information security violation or breach, and
• Complete all periodic mandatory security trainings in accordance with Mastercard's guidelines.

Pay Ranges
Vancouver, Canada: $127,000 - $203,000 CAD