Azure Cloud Data Engineer

Azure Cloud Data Engineer

03 Nov 2023
Texas, Houston, 77024 Houston USA

Azure Cloud Data Engineer

job summary:

Overview:

3+ years of expertise in building scalable distributed data processing solutions with Azure Data Lake Storage and Azure Synapse Data Warehouse, understanding of the functionalities - Identity Management, Security, Data Governance, DevOps, and Operations on Azure Platform.

Expert in managing Setup Dataiku DSS install, configuration and optimization Manage Python Recipes via Dataiku, docker container Troubleshoot, Python jobs failure issues Optimization of existing ML jobs Optimize Spark for better performance Run R/Python jobs via docker container and optimize it for better performance.


Essential Duties

  • Work as an azure cloud data engineer, an individual contributor, and a team player
  • Analyze, design, and determine the coding, programming, and integration activities required based on specific objectives
  • Dataiku DSS platform installation, configuration, and optimization
  • Manage data pipeline jobs failure issues
  • Optimization of existing ML jobs Optimize Spark for better performance
  • Python jobs via docker container and optimize it for better performance
  • Develop processes, techniques, and tools to analyze and monitor platform performance
  • strong working knowledge in Azure Data Warehouse and Data lakes
  • Experience in Azure cloud ecosystem
  • Experience in managing databases and objects
  • Experience Azure Data Lake Storage with Azure Synapse and Azure data factory
  • Develop Azure Data Factory/Synapse pipelines and experience with Azure SQL Pools, Apache Spark
  • Identify, troubleshoot and resolve issues related to slow / failed jobs
  • Support ingestion pipelines from a data engineering standpoint
  • Experience with SQL (Queries, Functions, and Stored Procedures) and Python languages
  • Participation in the design of data models for reports
  • Experience with Azure Functions and API integration management


Non-Essential Duties: Performs other duties as assigned.

  • Negotiate and influence changes outside of the team that continuously shape and improve the Data strategy.
  • Participate or run critical engagements, consistently delivering quality services. Drive high-quality work products within expected timeframes and on budget.
  • Collaborate with the advanced analytics team.


Skills and Abilities:


Ability to:

  • Dataiku DSS admin tasks
  • Dataiku platform installation and configuration
  • Docker containerization and optimization
  • Azure Synapse/Data Factory
  • Azure Data Lake Storage /ADLS Gen2
  • Oracle PL/Sql, SQL (Queries, Functions, Stored Procedures)
  • Databricks


Knowledge:

  • Powershell
  • Python
  • Pyspark
  • Azure DevOps


Minimum Education and Certification:

  • Bachelor's degree


Minimum Experience:

  • 5+ years of IT and Data Warehouse experience


Leadership:

  • Customer-focused professional who is motivated to drive the creation of great data engineering platforms
  • Contribute to the continued evolution of Corporate Analytics Platform
  • Responsible for managing a growing cloud-based data ecosystem



location: HOUSTON, Texas

job type: Permanent

salary: $100,000 - 120,000 per year

work hours: 9am to 6pm

education: Bachelors



responsibilities:

Essential Duties



  • Work as an azure cloud data engineer, an individual contributor, and a team player
  • Analyze, design, and determine the coding, programming, and integration activities required based on specific objectives
  • Dataiku DSS platform installation, configuration, and optimization
  • Manage data pipeline jobs failure issues
  • Optimization of existing ML jobs Optimize Spark for better performance
  • Python jobs via docker container and optimize it for better performance
  • Develop processes, techniques, and tools to analyze and monitor platform performance
  • Strong working knowledge in Azure Data Warehouse and Data lakes
  • Experience in Azure cloud ecosystem
  • Experience in managing databases and objects
  • Experience Azure Data Lake Storage with Azure Synapse and Azure data factory
  • Develop Azure Data Factory/Synapse pipelines and experience with Azure SQL Pools, Apache Spark
  • Identify, troubleshoot and resolve issues related to slow / failed jobs
  • Support ingestion pipelines from a data engineering standpoint
  • Experience with SQL (Queries, Functions, and Stored Procedures) and Python languages
  • Participation in the design of data models for reports
  • Experience with Azure Functions and API integration management




Non-Essential Duties: Performs other duties as assigned.



  • Negotiate and influence changes outside of the team that continuously shape and improve the Data strategy.
  • Participate or run critical engagements, consistently delivering quality services. Drive high-quality work products within expected timeframes and on budget.
  • Collaborate with the advanced analytics team.





qualifications:


  • Experience level: Experienced
  • Minimum 3 years of experience
  • Education: Bachelors (required)


skills:
  • Data Warehouse
  • Data Analysis
  • Cloud
  • Python
  • SQL
  • Azure
  • Powershell
  • Pyspark
  • Azure DevOps
  • Dataiku




  • Equal Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status.

    At Randstad, we welcome people of all abilities and want to ensure that our hiring and interview process meets the needs of all applicants. If you require a reasonable accommodation to make your application or interview experience a great one, please contact HRsupport@randstadusa.com.

    Pay offered to a successful candidate will be based on several factors including the candidate's education, work experience, work location, specific job duties, certifications, etc. In addition, Randstad offers a comprehensive benefits package, including health, an incentive and recognition program, and 401K contribution (all benefits are based on eligibility).



    For certain assignments, Covid-19 vaccination and/or testing may be required by Randstad's client or applicable federal mandate, subject to approved medical or religious accommodations. Carefully review the job posting for details on vaccine/testing requirements or ask your Randstad representative for more information.

    Related jobs

    Job Details

    • ID
      JC46915531
    • State
    • City
    • Job type
      Permanent
    • Salary
      USD100,000 - USD120,000
    • Hiring Company
      Randstad Technologies
    • Date
      2022-11-03
    • Deadline
      2023-01-01
    • Category

    Jocancy Online Job Portal by jobSearchi.