Senior Data Engineer - ETL, Bigdata and Azure

Senior Data Engineer - ETL, Bigdata and Azure

07 Mar 2024
Virginia, Richmond, 23173 Richmond USA

Senior Data Engineer - ETL, Bigdata and Azure

Vacancy expired!

Senior Data Engineer - ETL, Bigdata and Azure

Richmond, VA

Long term Contract

Looking for the candidates who can work on our W2.

1. 8+years in total, ETL/Data Warehouse background (3-5 years) with more recent Big Data/Data Engineering experience (5-8 years)
2. 3 years of Azure experience (writing new pipelines in data bricks, setting up arm templates, data factory)
3. 5+ years of development/coding (mostly Python, but also maybe Spark/Spark SQL)

Must have very good communcation skills.
This is a part of the Enterprise Data Lake team who is focused on delivery and new ingestion gonig into data lakes, creating new piplines with data bricks and Python.
Databricks WITHIN AZURE and Azure Data Factory are going to be non-negotiable.

Work collaboratively and creatively with data architects, engineers, and developers to develop scalable and maintainable data pipelines using the latest Microsoft Azure technologies and leading industry practices
Develop patterns and build data pipelines that ingest enterprise-wide data into the Enterprise Data Lake and make it available for consummation for both data analytics and data science
Partner with a variety of internal stakeholders across the organization to deliver creative, cutting-edge, high-quality engineering solutions that meet business requirements in a fast-paced, ever-changing business environment.

• 8+ years overall data engineering experience including guiding ideation, design, and decisions in a collaborative team setting
• Data pipeline development with Azure Data Factory and Azure Databricks
• Focus on complete solutions including unit/function/integration testing, documentation, pipeline monitoring and support, and team teach-outs
• Experience working with streaming data and technologies, such as Azure Event Hubs

Qualifications/Requirements
• Experience with Azure cloud data engineering solutions.
• Experience developing scalable and supportable Enterprise Data Lake patterns
• Experienced in working with Big Data frameworks, preferably Apache Spark
• Experienced in software development experience in either Python, Scala, C# or Java
• Experienced with transactional/relational data and SQL
• Experience working on Agile teams implementing DevOps practices
• Years of experience typically 8+ years

Skills
• Strong people skills including the ability to interact with employees at all levels
• Excellent communication skills, adapting to various audience types
• Strong on ideas for system optimization, support, and monitoring
• Passionate about innovation and loves solving complex problems in a highly- collaborative, fast-paced team environment
• Advanced knowledge of team software development concepts and techniques and the ability to communicate those concepts to junior team members including patterns in the areas of security, scalability, and usability
• Experience in creating and developing software and effectively communicating the rationale behind the designs
• Demonstrated ability to compare-and-contrast alternative approaches to meet objectives while assessing risk both quantitatively and qualitatively
• Possess strong organizational and time management skills

Preferred Qualifications:
• 8+ years overall data engineering experience including guiding ideation, design, and decisions in a collaborative team setting
• Data pipeline development with Azure Data Factory and Azure Databricks
• Focus on complete solutions including unit/function/integration testing, documentation, pipeline monitoring and support, and team teach-outs
• Experience working with streaming data and technologies, such as Azure Event Hubs

Related jobs

Job Details

  • ID
    JC10736287
  • State
  • City
  • Job type
    Contract
  • Salary
    Depends on Experience
  • Hiring Company
    KSN Technologies, Inc.
  • Date
    2021-03-03
  • Deadline
    2021-05-02
  • Category

Jocancy Online Job Portal by jobSearchi.