RequirementsBS degree in computer science, computer engineering or equivalent5–6 years of experience delivering enterprise software solutionsProficient in Spark, Scala, Python, AWS Cloud technologies3+ years of experience across multiple Hadoop / Spark technologies such as Hadoop, MapReduce, HDFS, HBase, Hive, Flume, Sqoop, Kafka, ScalaFlair for data, schema, data model, how to bring efficiency in big data related life cycleTechnical requirements and can translate them into technical implementationsExperience with Agile Development methodologiesExperience with data ingestion and transformationSolid understanding of secure application development methodologiesExperience with Airflow and PythonUnderstanding of automated QA needs related to Big dataObject-oriented design and experience approaching a problem from different angles, analyzing the pros and cons of different solutions skillsExperience communicating updates and resolutions to customers and other partnersResponsibilitiesUse your software engineering skills, including Java, Spark, Python, Scala, to Analyze different, complex systems and collaboratively design new products and servicesIntegrate new data sources and toolsImplement scalable and reliable distributed data replication strategiesMentor and provide direction in architecture and design to onsite/offshore developersCollaborate with other teams to design and deploy data tools that support both operations and product use casesPerform analysis of large data sets using components from the Hadoop ecosystemManage product features from the development, testing through to production deploymentEvaluate big data technologies and prototype solutions to improve our data processing architectureAs a Big Data Engineer, you will be reporting to the Director, Software Development
