Data Engineering at the City is a unique opportunity for meaningful, exciting work and professional development using state-of-the art technologies and software development best practices. This is a position on a diverse team of engineers, analysts, and GIS specialists, contributing to developing, maintaining, and optimizing our data pipelines and integration strategies, assisting departments and agencies in innovating, and enhancing the City’s service to its residents.The data engineer/integration specialist will develop an intimate understanding of the City’s diverse data and contribute to improving the City’s data engineering infrastructure, pipelines, models, and integrations. The team uses a blend of open source, custom developed and off-the-shelf tools, including Python, Bash, SQL, DBT, GIS, Docker, Terraform, Apache Airflow, Jenkins, Postgres, PostGIS, AWS, GitHub, Github actions, MuleSoft as an iPaaS and SAS API providers like ArcGIS Online and CARTO.Some prominent services the team is responsible for include DataBridge, OIT’s enterprise data warehouse, AIS, the City’s custom address information systems which supports public address search and internal geocoding, and Databridge-Airflow, a custom workflow generator and orchestrator for departmental integrations (https://metadata.phila.gov/) and open data publishing (https://opendataphilly.org/). Some examples of OIT’s commitment to serving the public through the effective and innovative integration of City open data include:https://Atlas.phila.govhttps://property.phila.govhttps://Openmaps.phila.govhttps://streetsmartphl.phila.gov/www.phila.gov/solarmapwww.phila.gov/stressmapWe are looking for a candidate who is not only technically skilled but also passionate about leveraging data to improve City operations and community life. This team is fundamental to the innovation of City services that directly affect residents. If you are eager to contribute to a team that values innovation, efficiency, service-oriented work and data-driven decision-making, we encourage you to apply.Essential FunctionsWorking with business partners of varying technical ability to understand how their data is produced, stored, and updated, designing enhancements to optimize enterprise integrations and public data accessibility.Design and develop new and expand and refine existing data pipelines using Apache Airflow, custom Python-based software, and other tools to connect diverse systems of record, centralized databases, and SaaS environments, improving data platform components, data governance, master data management, data standardization, enrichment and quality assurance.Play a key role in enterprise integration and open data initiatives, developing and supporting critical systems like the City’s DataBridge, Databridge-Airflow and AIS.Write clear, descriptive technical documentation for systems/applications, partner with data stewards of various technical levels and communicate technical details to non-technical stakeholders.
