Big Data Platform Engineer
Your Opportunity Our Global Data Technologies organization is currently seeking an experienced Big Data professional to showcase your technical expertise and passion. You will be working with a brilliant team of Big Data technologists who bring energy, focus and fresh ideas that support our mission to provide value by seeing the world "Through Clients' Eyes".
As a Big Data Engineer you will positively impact the design and development of our Big Data solutions, while aligning with the needs of our business partners. You will use your Hadoop technology experience to implement and support new solutions within the MapR environment. Join this bright team of Big Data minds and share in our success.
What you're good at
- Designing, Developing, Configuring, Deploying, Optimizing, Documenting, and Maintaining:
- Custom ingest jobs
- Enhance and allocate data in a MapR ecosystem
- File, source and record level data integrity and quality checks
- Logging and Error Handling
- Job & workflow scheduling, including ESP jobs
- Python scripting, Bash Scripting
- Hive Tables and Views
- Role-Based security across all data storage locations
- MapR Folder, Processing and Storage Components
- Data comparison across data storage locations: from files, to Hive, to traditional relational databases
- Working as an engineering team member in an agile team environment, including supporting story creation and refinement, creation of technical documentation, maintain Jira and support scrum and project meetings and status reporting
- Leveraging existing frameworks and standards, while contributing ideas and resolving issues with current framework owners.
- Professionally influencing and negotiating with other technical leaders to define and implement the optimal solution with consideration for technical and project constraints
- Interacting professionally with business partners and key contacts
- Creating solutions with a production end state in mind
- Reviewing code and provide peer feedback relative to best practices
- Working closely with architects to deliver appropriate technical solutions
- Working with data scientists and solution analysts to understand the business problems, pro-actively design intuitive data structures and create the best-suited data structures for modeling and analysis
- Translating advanced analytics results into production and maintain updates/refresh.
- Integrating additional and new sources of data that can enhance analytic capability, continually identifying and fixing issues that enforce a robust analytic process
- Providing customer service analytic support and behavioral insight and engagement
What you have
BS or MS in computer science
- 2+ years of hands on experience with Hadoop
- 5+ years software engineering experience
- 5+ years expertise with UNIX, JAVA, Python, BASH
- Cloud experience with either AWS or GCP on Data processing and storage solutions.
- 2+ years hands on experience with Big data technologies ( SPARK, HBASE, FLUME, HIVE, SQOOP, MAPR, DRILL,KAFKA, PIG, STORM MAPR Streaming technologies, machine learning libraries)
- 2+ years hands on experience with query optimization and tuning on Hive/Drill technology for heterogeneous large data sets.
- Experience with SPLUNK
- MapR experience desired
- Solid grasp of computer science fundamentals including data structures and algorithms.
- Proven ability to evaluate and apply new technologies in a short time.
- Proven understanding of the full software development lifecycle including testing, continuous integration, and deployment.
- Data and Analytics tools and building data structures to support advanced analytic and research functions.
- Expertise with both structured and unstructured data in a Big Data ecosystem
- Experience on database development, management and ETL in a big data ecosystem
- Experience with analytic scripting R, Python and strong with SQL.
Nice to have :
- Experience communicating with senior level business leaders and stakeholders
- Ability to interact and communicate successfully with business partners and technology teams.
- Personality that engages peers and promotes collaborative teamwork
- Extremely strong problem-solving skills
- Knowledge of various data science techniques and experience implementing models developed with these techniques into a production environment
- Experience in building data science or data analysis tools
- Machine Learning background
Back to top