Data Engineer I
Location: Milpitas California 95035
Duration: 6 month contract
NO B2B NO C2C must be able to be on W-2
Duties: Keeps abreast of industry trends and implementing them as needed
Strong Python development experience who has demonstrated track record of delivering complex projects on a large-scale Environment preferably size of TB of data daily
Data Eng -I : Expertise in architecting messaging systems, distributed data stores and NoSQL technologies (Some of them we work on: Kafka, Spark, Hive, Google Pub Sub, Cloud composer, BigQuery, Google Cloud Storage)
Must have Big Data Technology professional experience of 2+ years or more with hands-on development experience in Hadoop eco-system, Kubernetes, and cloud technologies Must have solid of experience in building tools and application on a scalable environment Must have professional experience working with building ETL pipelines and data cleansing processes Hands-on experience with any cloud based infra, Preferred GCP. Experience must include Data Proc, Data flow, pub/Sub and BigQuery to name a few.
(AWS experience is also accepted with similar to above listed cloud services and products)
Must have 1+ experience on building terraform code and Airflow for orchestrating the ETL data pipeline Performance tuning - table partitioning and indexing and sharding, process threading Willing to work on a fast-paced environment and be adoptable to change Must be a team player and have experience working with cross functional and be willing to work in a collaborative environment Detail-oriented with strong analytical and problem-solving skills Good and effective Communications skills (both verbal & written) Strategic thinker with a strong technical aptitude and a passion for execution
Familiarity with Machine learning and deep learning algorithm. Candidate must have experience with working with data scientists to scale their models and deploy on Cloud based environment.
Semiconductor industry experience
Have experience in migrating on-premises Data Lake and Hadoop to GCP Experience with designing and scaling ML models on distributed system or cloud environment and who has worked closely with data scientist