Senior Data Engineer
Dow Jones is a leading provider of financial and publishing data, providing customers with historical and near real-time information.
Ingesting, translating, and exporting all that data is a big data challenge requiring the use of the latest cloud and data tools. Data Engineering at Dow Jones is responsible for processing and delivering of tens of terabytes of data to internal and external customers via batch and streaming interfaces.
As a data engineer you will work with a small team to deliver data products to business customers, leveraging the vast corpus of data using the latest big data technologies in the cloud. A data engineer has the unique experience of working across multiple programming languages and technologies to implement the right solution for the customer
- Design and implement large scale distributed solutions in AWS and GCP clouds.
- Implement solutions which utilize Big Data cloud tools like Google Dataproc, Dataflow, and BigQuery.
- Design, build, and debug Hadoop, Hive, and Spark based data pipelines.
- Create APIs which expose collected and analyzed data to partners using NodeJS and Python.
- Design, build, and debug Python based pipelines using the Luigi framework.
- Process real-time streaming data using the ELK stack, Apache Storm, and Spark Streaming.
- Create new web visualizations using common open-source libraries like D3
- BS degree in Computer Science or equivalent work experience
- Experience with Java, Python, or Scala and a willingness to learn the others.
- Experience with SQL languages
- Experience with distributed version control like Git or Mecurial
- Excellent problem solver and communicator
- Self-motivated and ability to take initiative with creative problem solving
- Understanding of software craftsmanship qualities and principles
- Experience with Scala and/or Spark
- Experience with Kafka, Amazon Kinesis, or Google Pub/sub
- Experience building applications at scale using distributed databases, NoSQL, and MapReduce tools.
- Experience creating and consuming APIs for integration with external customers
- Familiarity with Amazon Web Services or Google Cloud Platform
- Familiarity with natural language processing and data modeling.
Dow Jones is a global provider of news and business information, delivering content to consumers and organizations around the world across multiple formats, including print, digital, mobile and live events. Dow Jones has produced unrivaled quality content for more than 125 years and today has one of the world’s largest news gathering operations globally. It produces leading publications and products including the flagship Wall Street Journal, America’s largest newspaper by paid circulation; Factiva, Barron’s, MarketWatch, Financial News, DJX, Dow Jones Risk & Compliance, Dow Jones Newswires, and Dow Jones VentureSource.
Equal Opportunity Employer:
All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, protected veteran status, or disability status. EEO/AA/M/F/Disabled/Vets
Requisition ID 2016-29118
Job Locations USA – MN-MINNEAPOLIS
Job Function IT Development
Business Area TECHNOLOGY – PROD & ENG
Job Area1 Technology
Job Area2 ..
Meet Some of Dow Jones's Employees
Subscriber Services Manager
Sendi works in Subscription Department at Dow Jones, dealing with fraudulent complaints and providing customer service support for WSJ, WSJ+, Barron's, and its affiliated sub-agencies.
Back to top