Senior Data Engineer

Job Description:

Dow Jones is a leading provider of financial and publishing data, providing customers with historical and near real-time information.

Ingesting, translating, and exporting all that data is a big data challenge requiring the use of the latest cloud and data tools. Data Engineering at Dow Jones is responsible for processing and delivering of tens of terabytes of data to internal and external customers via batch and streaming interfaces.

As a data engineer you will work with a small team to deliver data products to business customers, leveraging the vast corpus of data using the latest big data technologies in the cloud. A data engineer has the unique experience of working across multiple programming languages and technologies to implement the right solution for the customer

Responsibilities:

  • Design and implement large scale distributed solutions in AWS and GCP clouds.
  • Implement solutions which utilize Big Data cloud tools like Google Dataproc, Dataflow, and BigQuery.
  • Design, build, and debug Hadoop, Hive, and Spark based data pipelines.
  • Create APIs which expose collected and analyzed data to partners using NodeJS and Python.
  • Design, build, and debug Python based pipelines using the Luigi framework.
  • Process real-time streaming data using the ELK stack, Apache Storm, and Spark Streaming.
  • Create new web visualizations using common open-source libraries like D3

Required Skills:

  • BS degree in Computer Science or equivalent work experience
  • Experience with Java, Python, or Scala and a willingness to learn the others.
  • Experience with SQL languages
  • Experience with distributed version control like Git or Mecurial
  • Excellent problem solver and communicator
  • Self-motivated and ability to take initiative with creative problem solving
  • Understanding of software craftsmanship qualities and principles

Preferred Skills:

  • Experience with Scala and/or Spark
  • Experience with Kafka, Amazon Kinesis, or Google Pub/sub
  • Experience building applications at scale using distributed databases, NoSQL, and MapReduce tools.
  • Experience creating and consuming APIs for integration with external customers
  • Experience with Javascript
  • Familiarity with Amazon Web Services or Google Cloud Platform
  • Familiarity with natural language processing and data modeling.

Company Overview:

Dow Jones is a global provider of news and business information, delivering content to consumers and organizations around the world across multiple formats, including print, digital, mobile and live events. Dow Jones has produced unrivaled quality content for more than 125 years and today has one of the world’s largest news gathering operations globally. It produces leading publications and products including the flagship Wall Street Journal, America’s largest newspaper by paid circulation; Factiva, Barron’s, MarketWatch, Financial News, DJX, Dow Jones Risk & Compliance, Dow Jones Newswires, and Dow Jones VentureSource.

Equal Opportunity Employer:

All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, protected veteran status, or disability status. EEO/AA/M/F/Disabled/Vets

Requisition ID 2016-29118

Job Locations USA – MN-MINNEAPOLIS

Job Function IT Development

Business Area TECHNOLOGY – PROD & ENG

Job Area1 Technology

Job Area2 ..


Meet Some of Dow Jones's Employees

Sendi D.

Subscriber Services Manager

Sendi works in Subscription Department at Dow Jones, dealing with fraudulent complaints and providing customer service support for WSJ, WSJ+, Barron's, and its affiliated sub-agencies.

Yvette M.

Deputy Video Technology Director

From live productions to trans-coding to troubleshooting, Yvette manages the WSJ Video Department, implementing the latest technologies to deliver news and events seamlessly to multiple media platforms worldwide.


Back to top