Sign In
 [New User? Sign Up]
Mobile Version

Data Engineer - Lead

Bengaluru, KA
Job Code:
  • IT
Applying for this job will take you to an external site
  • Shortlist
  • Email Friend
  • Print

Job Details

Design data warehouse using Cloud Storage for data analysis using Hive/Spark etc.Write ETL pipelines in Hive and Spark for loading data into a Cloud based Data Warehouse.Write agents in Bash/Python/Java to publish data from different distributed data sourcesDesign and manage real-time data ingestion using Apache Kafka and RoR based web-service.Work with product and engineering teams on different data driven products and help drive/implement the Data flows for the same.Perform ad-hoc data analysis to test various hypothesis using SQL/Scala/PythonMentor a team of engineers while maintaining a hands-on role.MUST HAVEHands on experience in BigData space (Kafka, Spark, Hadoop Stack like M/R, Hive, HBase, Sqoop, etc..NoSQL stores like HBase etc)Designing, integrating and documenting technical components for seamless dataextraction and analysis on big dataplatformPassionate for continuous learning, experimenting, applying and contributing towards cutting edge open source technologies and software paradigmsEnsuring best practices that can be adopted in BigData stack and share across teams and BUs.Provides operational excellence through root cause analysis and continuous improvement in technologies and processes.Programming experience with Java & Python. Experience managing software projects.Bachelors Degree in Computer Science, or related discipline with minimum 6 years experience.NICE TO HAVEPrior experience in actively contributing to open source projects like Hadoop/Spark/Hive/HBaseExperience in applied machine learning.Academic research experience in the area of distributed systems and query processing.Advanced Degree (MS or PhD in Computer Science).
Degree: ME/ M.Tech./ MS (Engg/ Sciences) | Ph.D. (Doctorate)

Additional Degree: BE/ B.Tech (Engineering)

Experience: 6-11


Database Administration (DBA) | Project Leader/ Project Manager | Software Engineer | System Administration
Applying for this job will take you to an external site


© Copyright 2015 Al Nisr Publishing LLC - powered by Gulf News