Hadoop Administrator

Job Summary:

The Data Infrastructure Team at Shutterstock is responsible for the health and performance of our entire data transport pipeline (events/logs, metrics, etc) and the long term storage and availability of that data via Hadoop and other tools. We're looking for an exceptional engineer with a passion of enabling data for consumption across our organization. We believe in providing access that encourages data interaction, exploration and analysis.

 

Responsibilities:

  • Day to day operational support of our Hadoop installations in Dev, QA, Product.
  • Collaborate with Infrastructure teams around data center activities.
  • Responsible for upgrades, installations, systems management of Hadoop & related utilities.
  • Develop, manage and implement backup and recovery process Evaluate new technologies as they come out.
  • Work closely and collaboratively in an Agile environment with our developers and product teams to analyze issues and find new insights into our business and operations.

 

Basic Qualifications:

  • Familiarity with building hosts (kickstart, PXE boot), and configuration management system (Pupppet, Chef).
  • Experience with upgrading / expanding / deploying Hadoop clusters (Apache Hadoop, Hortonworks, MapR, or Cloudera).
  • Development and Operational Experience within Hadoop ecosystem (MR, HDFS, YARN, Hive, Sqoop, Oozie, etc).
  • Proficiency in Python, Java, or Scala Proficiency in SQL, Hive or another SQL-on-Hadoop tool.
  • Focus on delivering a quality experience to customers Must be self starter and motivated.

 

Bonus Qualifications:

  • Experience with ETL process / software [Pentaho, Scalding, Cascading, Luigi, Oozie, etc].
  • Experience with Cassandra, Vertica Experience with a stream processing technology (Spark, Storm, Samza, Flink, etc).
  • Experience in predictive analytics Open source contributions.

 

About Shutterstock:

Shutterstock, Inc. (NYSE: SSTK), is a leading global provider of high-quality licensed photographsvectorsillustrations, videos and music to businesses, marketing agencies and media organizations around the world. Working with its growing community of over 100,000 contributors, Shutterstock adds hundreds of thousands of images each week, and currently has more than 80 million images and nearly 4 million video clips available.

Headquartered in New York City, with offices in Amsterdam, Berlin, Chicago, Dallas, Denver, London, Los Angeles, Montreal, Paris, San Francisco, and Silicon Valley, Shutterstock has customers in more than 150 countries. The company also owns Bigstock, a value-oriented stock media agency; Offset, a high-end image collection; PremiumBeat a curated royalty-free music library; Rex Features, a premier source of editorial images for the world's media; and WebDAM, a cloud-based digital asset management service for businesses.

For more information, please visit www.shutterstock.com and follow Shutterstock on Twitter and on Facebook.

Equal Opportunity Employer, M/F/D/V

 

Back to top