3+ years of expertise in building scalable distributed data processing solutions with Azure Data Lake Storage and Azure Synapse Data Warehouse, understanding of the functionalities - Identity Management, Security, Data Governance, DevOps, and Operations on Azure Platform.
Expert in managing Setup Dataiku DSS install, configuration and optimization Manage Python Recipes via Dataiku, docker container Troubleshoot, Python jobs failure issues Optimization of existing ML jobs Optimize Spark for better performance Run R/Python jobs via docker container and optimize it for better performance.
Essential Duties
Work as an azure cloud data engineer, an individual contributor, and a team player
Analyze, design, and determine the coding, programming, and integration activities required based on specific objectives
Dataiku DSS platform installation, configuration, and optimization
Manage data pipeline jobs failure issues
Optimization of existing ML jobs Optimize Spark for better performance
Python jobs via docker container and optimize it for better performance
Develop processes, techniques, and tools to analyze and monitor platform performance
strong working knowledge in Azure Data Warehouse and Data lakes
Experience in Azure cloud ecosystem
Experience in managing databases and objects
Experience Azure Data Lake Storage with Azure Synapse and Azure data factory
Develop Azure Data Factory/Synapse pipelines and experience with Azure SQL Pools, Apache Spark
Identify, troubleshoot and resolve issues related to slow / failed jobs
Support ingestion pipelines from a data engineering standpoint
Experience with SQL (Queries, Functions, and Stored Procedures) and Python languages
Participation in the design of data models for reports
Experience with Azure Functions and API integration management
Non-Essential Duties: Performs other duties as assigned.
Negotiate and influence changes outside of the team that continuously shape and improve the Data strategy.
Participate or run critical engagements, consistently delivering quality services. Drive high-quality work products within expected timeframes and on budget.
Customer-focused professional who is motivated to drive the creation of great data engineering platforms
Contribute to the continued evolution of Corporate Analytics Platform
Responsible for managing a growing cloud-based data ecosystem
location: HOUSTON, Texas
job type: Permanent
salary: $100,000 - 120,000 per year
work hours: 9am to 6pm
education: Bachelors
responsibilities:
Essential Duties
Work as an azure cloud data engineer, an individual contributor, and a team player
Analyze, design, and determine the coding, programming, and integration activities required based on specific objectives
Dataiku DSS platform installation, configuration, and optimization
Manage data pipeline jobs failure issues
Optimization of existing ML jobs Optimize Spark for better performance
Python jobs via docker container and optimize it for better performance
Develop processes, techniques, and tools to analyze and monitor platform performance
Strong working knowledge in Azure Data Warehouse and Data lakes
Experience in Azure cloud ecosystem
Experience in managing databases and objects
Experience Azure Data Lake Storage with Azure Synapse and Azure data factory
Develop Azure Data Factory/Synapse pipelines and experience with Azure SQL Pools, Apache Spark
Identify, troubleshoot and resolve issues related to slow / failed jobs
Support ingestion pipelines from a data engineering standpoint
Experience with SQL (Queries, Functions, and Stored Procedures) and Python languages
Participation in the design of data models for reports
Experience with Azure Functions and API integration management
Non-Essential Duties: Performs other duties as assigned.
Negotiate and influence changes outside of the team that continuously shape and improve the Data strategy.
Participate or run critical engagements, consistently delivering quality services. Drive high-quality work products within expected timeframes and on budget.
Collaborate with the advanced analytics team.
qualifications:
Experience level: Experienced
Minimum 3 years of experience
Education: Bachelors (required)
skills:
Data Warehouse
Data Analysis
Cloud
Python
SQL
Azure
Powershell
Pyspark
Azure DevOps
Dataiku
Equal Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status.
At Randstad, we welcome people of all abilities and want to ensure that our hiring and interview process meets the needs of all applicants. If you require a reasonable accommodation to make your application or interview experience a great one, please contact HRsupport@randstadusa.com.
Pay offered to a successful candidate will be based on several factors including the candidate's education, work experience, work location, specific job duties, certifications, etc. In addition, Randstad offers a comprehensive benefits package, including health, an incentive and recognition program, and 401K contribution (all benefits are based on eligibility).
For certain assignments, Covid-19 vaccination and/or testing may be required by Randstad's client or applicable federal mandate, subject to approved medical or religious accommodations. Carefully review the job posting for details on vaccine/testing requirements or ask your Randstad representative for more information.
