Big Data Engineer

Job Description

Big Data Engineer (Java/Scala, Python, Spark) San Francisco Bay Area

Must have:

• University degree in Computer Science or similar
  


• Strong OOP and Design patterns knowledge
  


• 5+ years of commercial development experience including Java or Scala, and Python
  


• 2+ years experience with Spark and Hive including understanding on how these technologies work under the hood
  


• Strong experience in distributed Big Data processing (batch/offline, Terabytes and more)
  


• Experience with AWS
  


• Rigor in high code quality, unit testing, and other engineering best practices
  


• Strong communication, collaboration and interpersonal skills
  


• Good English (oral & written)
  


• Ability to learn quickly
  


• Experience working in Agile environment
  

Main Tasks and Responsibilities:

• Obtains tasks from the project lead or Team Lead (TL), prepares functional and design specifications, approves them with all stakeholders.
  


• Ensures that assigned area/areas are delivered within set deadlines and required quality objectives.
  


• Provides estimations, agrees task duration with the manager and contributes to project plan of assigned area.
  


• Analyzes scope of alternative solutions and makes decision about area implementation based on his/her experience and technical expertise.
  


• Leads functional and architectural design of assigned areas. Makes sure design decisions on the project meet architectural and design requirements.
  


• Addresses area-level risks, provides and implements mitigation plan.
  


• Reports about area readiness/quality, and raises red flags in crisis situations which are beyond his/her AOR.
  


• Responsible for resolving crisis situations within his/her AOR.
  


• Initiates and conducts code reviews, creates code standards, conventions and guidelines.
  


• Suggests technical and functional improvements to add value to the product;
  


• Constantly improves his/her professional level.
  


• Collaborates with other teams.