ETL/PySpark

ETL/PySpark

Vacancy expired!

Job Description:
Key responsibilities

• Preparing HLD and LLD document with ETL methodology transformation rules for data pipelines.
• Extensive experience on Spark, PySpark and Hive SQL Scripts.
• Knowledge on Sqoop and Impala
• Involves in build and unit test of data pipeline ETL's.
• Provide support and bug fixes during trial validation runs and UAT.
• Performs ETL code packaging, sequencing and validations of same
• SIT and UAT support.
• Conduct code reviews and provide improvement guidelines
• Extensive experience on Spark, PySpark and Hive SQL Scripts.
• Coding and development, unit test the ETL code changed/developed.
• Provide support and bug fixes during trial validation runs and UAT.
• Performs ETL code packaging, sequencing and validations of same.
• Involves peer review of ETL code artifacts.
• Support for Production Deployment
• Participate in Unit and System Testing and approve deliverable