Senior Data Engineer Gen AI

Location Designation:Hybrid

When you join New York Life, you're joining a company that values career development, collaboration, innovation, and inclusiveness. We want employees to feel proud about being part of a company that is committed to doing the right thing. You'll have the opportunity to grow your career while developing personally and professionally through various resources and programs. New York Life is a relationship-based company and appreciates how both virtual and in-person interactions support our culture.

Location: Clinton, NJ or New York City office

This role is hybrid (3 days in the office with Mondays & Fridays working remote)

Company Summary

At the heart of New York Life is a commitment to be there for our customers when they need us-whether today or decades into the future. We have delivered on that promise for over 175 years by investing wisely, growing a portfolio of strategic businesses, and remaining true to our mission as a mutual company, accountable only to our customers, not to outside investors. For our customers, that means having the confidence that comes with knowing they can build a better future for themselves and those they love.

• Engages in collaborative relationships across architecture, product management, data, and product delivery teams to design, implement solutions based on defined business challenges and outcomes
• Participateonscrum teams working in short sprints and ensureon time deliverywith high quality
• Build and maintaindata engineering solutions on cloud platforms using AWS or Azure services
• Design, develop, and implement scalable data pipelines and ETL/ELT processes using Python, PySpark and API integrations
• Implement data quality and validation checks to ensure accuracy and consistency of data
• Collaborate with data scientists and analysts to understand data requirements and translate them into scalable, high performant data pipeline solutions.
• Develop and maintain data models and schemas to support data integration and analysis
• Monitor and troubleshoot data pipeline performance, identifying and resolving bottlenecks and issues
• Stay updated with the latest technologies and trends in big data, data engineering, data science, and REST API development, and provide recommendations for process improvements
• Use AWS Cloud technologies to support data needs for expansion of Machine Learning/Data Science capabilities, applications/mobile apps/systems, BI/analytics, and cross-functional teams
• Create methods and routines to transition data from on-premise systems to the AWS Cloud
• Store data in Cloud platform and develop transformational logic based on business rules
• Build the infrastructure required for optimal ETL/ELT (extraction, transformation, and loading) of data from a wide variety of data sources using SQL and AWS big data technologies
• Develop and debug code in PySpark/Spark/Python. Strong SQL and query optimization techniques
• Maintaining high degree of knowledge and expertise in cloud-based technologies contributing to go forward best practices, standards, and patterns
• Active participant in relevant communities of practice to exchange ideas and knowledge

• Experience with database technologies forstructured and unstructureddata both forstorage and optimal retrieval
• Experience in data management of Content and Content data stores
• Experience with AWS Cloud tools like Amazon Relational Database Service (AWS RDS), Redshift, AWS Database Migration Service (AWS DMS), AWS Glue, Amazon Simple Storage Service (AWS S3), AWS Lambda triggers & functions, etc.
• Strong, demonstratable hands-on experience with data engineering and data pipelines
• Proficiency with languages such asPython, PySpark, Javaetc.
• Experience with developingCloud-native applications and very good understanding of Cloud architectureand best practices
• Experience with relevant code repository and project tools such asGitHub, JIRA, Confluence
• Working experience with Continuous Integration, Continuous DeploymentCI/CD
• Experience with highly scalableData stores, Data Lake, Data Warehouse, Lakehouse, and unstructured datasets
• Experience with Data integration, data processing, data streaming, message queuing techniques
• Detail oriented individual with strong analytical, problem-solving, and organizational skills
• Ability to clearly communicatetoboth technical and business teams

• Solid understanding of the insurance, wealth management or financial services industry
• Knowledge of scripting tools on multiple platforms preferred
• Extensive hands-on development experience withCloud (AWS, Azure) and Storage servicesare desirable
• Experience in developing solutions leveraging LLMs a plus
• Knowledge of machine learning pipelines and frameworks likeMLFlowa plus
• Strong hands-on experience and extensive development experience with API integrations a plus

• Bachelor's degree in Computer Science or an Engineering discipline
• Certification in data management (structure and unstructured), and/or cloud architecture or engineering a plus
• Certification in delivery methodologies such as Scaled Agile a plus

• 8+ years of experience in data engineering, machine learning, and cloud tech developing production grade software components and solutions

• Trusted and respected team member putting objectives of the team over individuals.
• Comfortable, experienced, and accomplished at working with business stakeholders, and able to debate and challenge in a professional and diplomatic way.
• Highly Collaborative and supportive of business and of its ideals and strategies.
• Highly innovative with aptitude for foresight, systems thinking and design thinking.
• More interested in achieving targeted business outcomes and what is best for the organization than in personal preferences, or in vested personal preferences of other business and IT leaders.
• Composed in the face of opposition to technology principles, governance, and standards.
• Practical in approach to problem solving and decision making, with a bias towards simplifying processes.