Senior Data Engineers

CCC Intelligent Solutions Inc. (CCC) is a leading cloud platform for the multi-trillion-dollar insurance economy, creating intelligent experiences for insurers, repairers, automakers, part suppliers, and more. At CCC, we're making life just work by empowering more than 35,000 businesses with industry-leading technology to get drivers back on the road and to health quickly and seamlessly. We're pushing boundaries with innovative AI solutions that simplify and enhance the claims and repair journey. Through purposeful innovation and the strength of its connections, CCC technologies empower the people and industry relied upon to keep lives moving forward when it matters most. Learn more about CCC at www.cccis.com.

The Role

Key Responsibilities:

Senior Data Engineers for Chicago, IL location. Develop large scale end to end data pipeline applications, covering multiple data sources spread across data center and AWS cloud. Use developed software applications to locate and analyze source data; create data flows to extract, profile, and store ingested data; define and build data cleansing and imputation; map to a common data model; transform to satisfy business rules and statistical computations; and validate data content. Produce software data building blocks, data models, and data flows, such as dimensional data, data feeds, dashboard reporting, and data science research and exploration. Produce automated software tests of data flow components and for data content quality. Automate orchestration and error handling for use by production operation teams. Provide technical expertise to diagnose errors from production support teams. Guide junior team members in performance tuning applications in distributed computing environments. Perform root cause analysis on all data and processes and identify opportunities for improvement. Develop metadata-driven and fully parameterized data processing tools. Mentor junior engineers. Technical Environment: Programming with Python and PySpark; Massively Parallel Processing (MPP) and distributed resource management with YARN in AWS EMR; columnar storage formats (Parquet, ORC) and Apache Hudi; ETL pipeline orchestration using Apache Airflow; Building and optimizing data pipelines with Apache Kafka; Advanced SQL for profiling and validation; Presto; PostgreSQL; Unix scripting and root cause analysis of data and processes; CI/CD tools (Git, GitHub Actions, Jenkins); Agile methodologies and metadata-driven development; AWS tools - EMR, S3, RDS, Redshift, CloudWatch, Bedrock, MWAA; Machine learning; predictive modeling using DataRobot.

#LI-DNI

#NOINDEED

Requirements:

Master's degree in Computer Science, Computer Engineering, Management Information Systems, Business Analytics or related field plus 3 years of experience with software development/data processing or analysis required. Required skills: Programming with Python and PySpark; Massively Parallel Processing (MPP) and distributed resource management with YARN in AWS EMR; columnar storage formats (Parquet, ORC) and Apache Hudi; ETL pipeline orchestration using Apache Airflow; Building and optimizing data pipelines with Apache Kafka; Advanced SQL for profiling and validation; Presto; PostgreSQL; Unix scripting and root cause analysis of data and processes; CI/CD tools (Git, GitHub Actions, Jenkins); Agile methodologies and metadata-driven development; AWS tools - EMR, S3, RDS, Redshift, CloudWatch, Bedrock, MWAA; Machine learning; predictive modeling using DataRobot. Some telecommuting permitted. $130,707/yr - $139,000/yr.

• 401K Match
• Paid time off
• Annual Incentive Plan Performance Bonus
• Comprehensive health insurance
• Adoption Assistance
• Tuition Reimbursement
• Wellness Programs
• Stock Purchase Plan options
• Employee Resource Groups