Seeking a highly technical "hands on" engineering leader to guide the evolution of company’s data platform, which ingests over ten thousands datasets, stores over a petabyte of spatial, relational and raster data, and serves over five billion API calls per month. This is a very “hands-on” technical engineering leadership role. This role will be approx 25-30% technical and hands-on. The rest of the time (65-70%) will be working with senior leadership to translate the company’s “big data” platform vision and strategy and you will also be a mentor for team members, an evangelist of the "data platform" for other engineering teams, and a translator between engineering and the business. This will include facilitating and participating in design sessions, code reviews and sprint ceremonies, as well as giving presentations on company’s data platform for technical and non-technical audiences.
The Vice President, Engineering - Data Platform has four key functions: (1) execution of company’s data platform strategy, transitioning from traditional RDBMS clusters and API monoliths to modern data lakes, distributed data processing, real-time data streams, automated ingest pipelines and microservices on top of horizontally scalable cloud infrastructure; (2) unification of company’s data ingestion, storage, compute and API services, breaking down silos and converging industry-leading brands and data assets onto a single platform; (3) creation of new ingestion, storage, compute and service capabilities, leveraging big data technologies, machine learning, natural language processing, optical character recognition, automation and orchestration technologies, cloud scaling, etc.; and (4) management of company’s data engineering processes to improve velocity, quality, and efficiency, and to expand the number and types of datasets within the company’s ecosystem. An ideal candidate is a hands-on, transformational technology leader with experience building distributed data storage and processing systems, managing large scale data ingestion operations, and creating high performance engineering organizations. This individual will lead the transformation and evolution of company’s data platform and play a key role in the convergence of company’s industry-leading but independent brands into a single informatics platform at the center of the commercial real estate ecosystem What you will do and achieve:
Lead the creation of a next generation data platform to ingest tens of thousands of datasets, support petabyte-scale storage and compute, and deliver billions of real-time queries per month, while maintaining cost effectiveness and implementing appropriate data safeguards.
Manage an organization of over a half dozen software and data engineering teams responsible for company’s data ingestion, storage, compute and API services.
Identify opportunities to improve the performance and scale of APIs, the velocity and efficiency of data ingestion, the connectivity and linking of datasets, and the extraction of natural language and imagery sources.
Work with senior leadership to translate company’s platform vision and strategy into an actionable roadmap, maintain KPIs to track process, and deliver on-time and on-budget.
Collaborate with the application engineering, product management, project management, data science and market-facing teams to align the data platform with business needs.
Define standards and practices around automation, system reliability, data architecture, process management, containerization, infrastructure-as-code, auto-scaling, data security, etc.
Serve as a mentor for team members, an evangelist of the data platform for other engineering teams, and a translator between engineering and the business. This will include facilitating and participating in design sessions, code reviews and sprint ceremonies, as well as giving presentations on company’s data platform for technical and non-technical audiences.
Investigate and resolve technical and non-technical issues, including leading and participating within incident management processes and root cause analyses.
Contribute to company’s technology strategy as a member of its architectural leadership team.
5 or more years of experience managing software engineering teams
3 or more years of experience with big data systems
3 or more years of experience with cloud architecture
10 or more years of experience in software engineering
B.S. in Computer Science (or equivalent)
Knowledge & Skills
Big data architecture and systems, including distributed data processing systems (such as Spark or Dask), distributed data storage systems (such as Parquet or HDFS), low-latency data lake query architectures (such as Alluxio) and real-time streaming systems (such as Kafka)
Data engineering techniques for big data, including data automation frameworks (such as Airflow or Prefect), metadata management (such as Amundsen) and process management strategies
Relational, graph, document, key-value and fuzzy data systems, as well as accompanying data models, architectures, and principles
Data lake design strategies for metadata, ontology, governance, authorization, etc.
API design principles and practices, including microservice architecture, RESTful endpoint design, GraphQL graph design, OpenAPI service definition, and API testing methodologies.
Infrastructure management and automation, such as Kubernetes, Terraform and Chef
Test automation for data quality, data flow, and API endpoints
Cloud infrastructure management, ideally with past experience in AWS, including both technical aspects, such as solutions architecture, and non-technical aspects, such as financial planning
Modern practices around agile development, release management, continuous integration, system reliability, cloud architecture, authN/Z and data security
Organizational leadership, including managing roadmaps, budgets and human resources, as well as defining processes, developing KPIs and managing knowledge
Fundamentals of computer science and software engineering
Execute on a data platform strategy in collaboration with team members, architects, product managers and other groups across the business
Manage high-performance software and data engineering teams, including defining tasks, reviewing designs, facilitating sprint ceremonies, managing releases, hiring to expand the team’s competencies, and guiding professional development to grow the team from within
Clearly communicate decision points, opportunities, and outcomes to senior leadership
Exercise discretion and independent judgment on all projects and responsibilities
Contribute as a technology leader, including hands on when necessary, to meet team objectives
Mentor team members on technical and non-technical topics
Stay up to date on emerging technologies, standards, and protocols
TO APPLY: email resume to Lisa@taskmanagement. com