Software Engineer Lead - Databricks, Scala, Sparks, Azure

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together.

About the Role:

We are seeking a highly skilled and experienced Senior Software Engineer with expertise in Databricks, Scala, Spark, and Azure to join our dynamic team. The ideal candidate will have a strong background in designing and developing large-scale data processing solutions and integrating them with cloud platforms. Experience with databases and SQL is required. Exposure to application development using .Net and React is a plus.

Primary Responsibilities:

• Design, develop, optimize scalable data pipelines using Databricks (Spark/Scala) on Azure
• Collaborate with cross-functional teams to deliver robust end-to-end data solutions
• Write efficient SQL queries for ETL processes, data validation, and reporting
• Optimize performance of complex data processing workflows
• Implement best practices for code quality, testing, deployment, and documentation
• Participate in code reviews and mentor junior engineers
• Work closely with stakeholders to understand business requirements and translate them into technical solutions
• Integrate data solutions with other applications; exposure to .Net APIs or React-based UIs is a plus
• Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so

• Bachelor's degree in computer science or related field
• 6+ years of hands-on experience in software engineering/data engineering roles
• Solid experience with Databricks platform for big data analytics
• Hands-on experience working on Azure cloud services (e.g., Data Lake Storage, Azure Data Factory)
• Experience working within Agile/Scrum methodologies
• Proficient in Scala programming language; solid understanding of Spark framework (RDD/Data Frame/SQL)
• Solid background in Database design & development; excellent SQL skills (T-SQL/PLSQL/PostgreSQL etc.)
• Understanding of CI/CD pipelines for automated deployments on cloud platforms

• Exposure to application development using .Net (C#/ASP.NET) and/or ReactJS frontend frameworks
• Knowledge of RESTful API design/integration concepts
• Familiarity with DevOps tools like Azure DevOps/Jenkins/GitHub Actions

• Excellent analytical and problem-solving abilities
• Solid communication skills - able to convey technical concepts clearly to non-tech stakeholders
• Self-motivated team player who can work independently when required