We are looking for a Lead Data Engineer to join our Data Domain!What is this role about?In this newly created, critical role, you will be responsible for the design, development and support of the Data Products Cuscal creates for both internal and client facing use cases. The role accomplishes this by:Sourcing, transforming, securing and monitoring data for the creation of products using Cuscal’s data lake and analytics systems.Engaging in discovery activities with the Data Product Owner and product, engineering and analytics personnel from both internal and client teams.Working in partnership with our Data Practice to deliver solutions that meet our organisational standards and best practices.Here's more of what you’ll work on:Design Solutions: Develop and maintain scalable pipelines. Ensure systems are designed for extensibility, are robust, and can handle large volumes of data efficiently.Data Integration: Lead efforts in integrating data from various sources, ensuring data quality and consistency. Work on data transformation, processing, and loading to support analytics and business intelligence needs.Performance Optimisation: Identify and resolve performance bottlenecks within the Databricks environment, including optimizing the data ingestion processes and ensuring efficient data querying.Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet those needs. Provide guidance and mentorship to junior data engineers.Innovation and Improvement: Continuously explore new data technologies and methodologies. Implement best practices for data engineering in partnership with our Data practice.Documentation: Maintain comprehensive documentation of data workflows, architectures, and best practices. Ensure that all data processes are well-documented and reproducible.Security and Compliance: Ensure data security and compliance with industry standards and regulations. Implement data governance practices and monitor adherence to policies.
