Data Engineer : Spark + Python + Java + AWS

Data Engineer : Spark + Python + Java + AWS

03 Jul 2022
Texas, Richardson, 75085 Richardson USA

Data Engineer : Spark + Python + Java + AWS

Vacancy expired!

Data Engineer : Python + AWS or Java + AWS Note day 1 onsite and later 3 months hybrid mode. However, if you come across any candidates exceptional and want day 1 hybrid only Pls share the below pre screening answers for each submittal.

  • Questions
  • Answer
    1. What version of Spark did they work with? What programming language did they use (should be Python, Scala, or Java)?

    2. Do they have experience pulling data from REST APIs with Python?

    3. Do they have experience with AWS Glue? What did they use it for and where did they load data (Redshift, S3, Snowflake, etc.)?



    Willing to work Onsite Initial 8weeks onsite later hybrid mode


    Sr.Data Engineer: CTH & FTE

    Position Responsibilities
    Partner with business stakeholders to gather requirements and translate them into technical specifications and process documentation for IT counterparts (on-prem and offshore)
    Highly proficient in the architecture and development of an event driven data warehouse; streaming, batch, data modeling, and storage
    Advanced database knowledge; creating/optimizing SQL queries, stored procedures, functions, partitioning data, indexing, and reading execution plans
    Skilled experience in writing and troubleshooting Python/PySpark scripts to generate extracts, cleanse, conform and deliver data for consumption
    Expert level of understanding and implementing ETL architecture; data profiling, process flow, metric logging and error handling
    Support continuous improvement by investigating and presenting alternatives to processes and technologies to an architectural review board
    Develop and ensure adherence to published system architectural decisions and development standards
    Multi-task across several ongoing projects and daily duties of varying priorities as required
    Interact with global technical teams to communicate business requirements and collaboratively build data solutions
    The duties listed above are the essential functions, or fundamental duties within the job classification. The essential functions of individual positions within the classification may differ. May assign reasonably related additional duties to individual employees consistent with standard departmental policy.

    Requirements
    6-8 years of development experience
    Bachelor's degree in Computer Science, MIS or related field (industry experience substitutable)
    Expert level in data warehouse design/architecture, dimensional data modeling and ETL process development
    Advanced level development in SQL/NoSQL scripting and complex stored procedures (Snowflake, SQL Server, DynomoDB, NEO4J a plus)
    Extremely proficient in Python, PySpark, and Java
    AWS Expertise Kinesis, Glue (Spark), EMR, S3, Lambda, and Athena
    Streaming Services Confluent Kafka and Kinesis (or equivalent)
    Hands on experience in designing and developing applications using Java Spring Framework (Spring Boot, Spring Cloud, Spring Data etc)

    Related jobs

    Job Details

    • ID
      JC43800547
    • State
    • City
    • Job type
      Contract
    • Salary
      $Open
    • Hiring Company
      Cloudious LLC
    • Date
      2022-07-03
    • Deadline
      2022-09-01
    • Category

    Jocancy Online Job Portal by jobSearchi.