Sr. Data Engineer - Cloud

Sr. Data Engineer - Cloud

Vacancy expired!

job summary:

This role is for a senior data engineer that will join a team responsible for managing a growing cloud-based data ecosystem consisting of a metadata driven data lake and databases that support real time analytics, extracts, and reporting.

Required:

• The right candidate will have a solid background in data engineering and should have a few years of experience on a major cloud platform such as Azure.
• Databricks
• Apache Spark
• Scala programming
• Azure
• Data engineering experience - 5 years
• Cloud platform experience - 2 years
• Version Control (Git or equivalent) - 2 years

Assets:

• Data Integration Tools (Spark/Databricks or equivalent) 2 years
• Version Control (Git or equivalent) 2 years
• Scripting (Linux/Unix Shell scripting or equivalent) 2 years
• Netezza experience

location: Schaumburg, Illinois

job type: Contract

salary: $52.50 - 65.50 per hour

work hours: 8am to 5pm

education: No Degree Required



responsibilities:

• Building and maintaining a data processing framework on Azure using Databricks
• Writing code in Apache Spark/Scala
• Working with existing Databricks Delta Lake tables to optimize for CDC performance using techniques
• Working with existing Databricks Notebooks to optimize or address performance concerns
• Create new Databricks Notebooks or stand-alone Apache Spark/Scala code as needed
• Willingness to learn existing on-premise data management tools as required, such as Ab Initio

The essential functions of this role include:

• working in a smoke free environment

qualifications:

• Experience level: Experienced
• Minimum 3 years of experience
• Education: No Degree Required

skills:

• Azure
• Databricks
• Apache Spark
• Scala

Equal Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status.