Job Description
- Great opportunity to work on large scale big data project
- Opportunity to showcase and take ownership on your work
- Agile, fast-paced environment
As an experienced Big Data Engineer, your mission is to help lead the Data Science team towards creating next-level solutions that improve the way business is run. Your deep knowledge of design, analytics, development, coding, testing and application programming will help the team raise their game, as well as satisfying both business and functional requirements.
Key Requirements:
- Optimize the Big Data systems to best carry out enterprise analytical projects
- Deploy enterprise-level analytical models on cloud
- Build up libraries of transformations, utilities, and code to best operate the Big Data system
- Architect and build Big data pipelines to support deployment of analytical models
- Integrate Machine Learning models developed by the Data Science team into end-to-end big data pipelines
- Keep up on the latest Data technologies
- Mentor juniors on the newer Big Data technologies, offer advice and guidance to best utilize new Big Data tools and optimize jobs/code
- Bachelor's Degree or higher in Engineering, Computer Science or Information Technology
- Minimum 5 years of solid commercial exposure to Big Data environment – Hadoop, Spark, Scala
- Stream Processing - Storm, Hbase, Nifi, Spark Streaming, etc.
- Batch Processing - Talend, Informatica, or Hive/SQL
- Cloud platform – AWS, GCP, Azure / any private cloud
- Programming language – Python, Scala, Bash, Java etc.
- Extensive experience with horizontally scalable and highly available system design and implementation, with focus on performance and resiliency
- Excellent communications – both written and verbal
- Strong stakeholder engagements