Big Data Engineer
Where good people build rewarding careers.
Think that working in the insurance field can't be exciting, rewarding and challenging? Think again. You'll help us reinvent protection and retirement to improve customers' lives. We'll help you make an impact with our training and mentoring offerings. Here, you'll have the opportunity to expand and apply your skills in ways you never thought possible. And you'll have fun doing it. Join a company of individuals with hopes, plans and passions, all using and developing our talents for good, at work and in life.
Job Family Summary
Big Data Engineering incorporates techniques across many disciplines - including mathematics/statistics, computer programming, data engineering and ETL, software development, and high performance computing - with traditional business expertise with the goal of extracting meaning from data to optimize future business decisions. Individuals in this field should be an expert/fluent in several of these disciplines and sufficiently proficient in others to effectively design, build, and deliver end to end predictive analytics products to optimize future decisions. Individual demonstrates sufficient analytic agility to quickly develop new skills across these disciplines as those disciplines evolve. The Big Data Engineer job family is accountable for end to end engineering of data solutions which includes designing and building systems for data storage and analytics that enable Allstate analysts to make better decisions to achieve Allstate's goals
This role is responsible for the delivery of Big Data solutions to make decisions. This includes the development of new systems for analyzing data; the coding & development of advanced analytics solutions to make/optimize business decisions and processes; integrating new tools to improve descriptive, predictive, and prescriptive analytics; and discovery of new technical challenges that can be solved with existing and emerging Big Data hardware and software solutions. This role will also begin project management of moderately complex projects. This role contributes to the structured and unstructured Big Data / Data Science tools of Allstate from traditional to emerging analytics technologies and methods. The role is responsible for assisting in the selection and development of other team members
- Executes complex and occasionally highly complex functional work tracks for the team. Drives the execution of operational/technical objectives for data analytic outputs and business solutions.
- Partners with ATSV teams on Big Data efforts.
- Partners closely with peers in the department to ensure holistic Big Data solutions for our data science community and analytic users.
- With coaching, can identify new areas of data, research and big data technology that can solve business problems
- Leverages and uses Big Data best practices / lessons learned to develop technical solutions used for descriptive analytics, ETL, predictive modeling, and prescriptive "real time decisions" analytics
- Influences within the department on the effectiveness of Big Data systems to solve their business problems
- Is recognized as an expert in Big Data and is influential within the department.
- Develops technical solutions using Big Data techniques in data & analytics processes.
- Supports Innovation; regularly provides new ideas to help people, process, and technology that interact with analytic ecosystem.
- With coaching, develops and builds frameworks/prototypes that integrate big data and advanced analytics to make business decisions.
- With coaching, implements new areas of Big Data technologies, (ingestion, processing, distribution) and research delivery methods that can solve business problems.
- Understands the Big Data related problems and requirements to identify the optimal technical approach.
- Works with key team members to ensure efforts within owned tracks of work will meet their needs.
- Drives multiple tracks of moderately complex and occasionally complex work within the research group.
- Identifies and develops Big Data sources & techniques to solve business problems.
- Co-mingles data sources to lead work on secure data and problems across departments to drive improved business & technical results through designing, building, and partnering to implement models.
- Manages various Big Data analytic tool development projects with midsize teams.
- Executes on Big Data requests to improve the accuracy, security, quality, completeness, speed of data, and decisions made from Big Data analysis.
- Uses, learns, teaches, and supports a wide variety of Big Data and Data Science tools to achieve results (i.e., R, ETL Tools, Hadoop, and others).
- Uses, learns, teaches, and supports a wide variety of programming languages on Big Data and Data Science work (i.e. Java, C#, Python, and Perl)
- The development and execution of a communication strategy that keeps all relevant stakeholders informed and provides an opportunity to influence the direction of the work
- Trains and develops more junior engineers.
- 3 - 4 Years of experience or equivalent skills & ability
- Master's or PhD preferred in a quantitative or scientific field such as computer science, computer engineering or equivocal experience
- Experience in using software development to drive data science & analytic efforts
- Experience with database & ETL technologies
- Experience with various data types (e.g. Relational, Unstructured, Hierarchical, Linked "Graph" Data)
- Experience in developing, managing, and manipulating large, complex datasets
- Understanding of security risks and vulnerabilities pertaining to open source systems leveraging tools and techniques to minimize risk. Where appropriate, provide recommendations and justifications to ensure speed of access while minimizing risk for scientists and developers.
- Experience and solid understanding of Bigdata ecosystems such as Hadoop, Spark, Kafka
- Proven ability to code and develop prototypes in languages such as Python, Scala, Java, C, R, SQL
- Ability to communicate and present advanced technical topics to general audiences including teams in different time zones.
- Leading project teams of various skills levels
- Understanding of predictive modeling techniques a plus
Good Work. Good Life. Good Hands®.
As a Fortune 100 company and industry leader, we provide a competitive salary - but that's just the beginning. Our Total Rewards package also offers benefits like tuition assistance, medical and dental insurance, as well as a robust pension and 401(k). Plus, you'll have access to a wide variety of programs to help you balance your work and personal life -- including a generous paid time off policy.
Learn more about life at Allstate. Connect with us on Twitter, Facebook, Instagram and LinkedIn or watch a video.
Allstate generally does not sponsor individuals for employment-based visas for this position.
Effective July 1, 2014, under Indiana House Enrolled Act (HEA) 1242, it is against public policy of the State of Indiana and a discriminatory practice for an employer to discriminate against a prospective employee on the basis of status as a veteran by refusing to employ an applicant on the basis that they are a veteran of the armed forces of the United States, a member of the Indiana National Guard or a member of a reserve component.
For jobs in San Francisco, please click "here" for information regarding the San Francisco Fair Chance Ordinance.
For jobs in Los Angeles, please click "here" for information regarding the Los Angeles Fair Chance Initiative for Hiring Ordinance.
It is the policy of Allstate to employ the best qualified individuals available for all jobs without regard to race, color, religion, sex, age, national origin, sexual orientation, gender identity/gender expression, disability, and citizenship status as a veteran with a disability or veteran of the Vietnam Era.
Meet Some of Allstate's Employees
Director Of Technology Strategy
Josh is constantly looking for opportunities for Allstate to use technology and computer software in innovative ways that continue to build on the company’s core strengths.
Back to top