Design & Architect Data Infrastructure — Design and oversee deployment of data architecture and pipelines to capture, manage, and store structured and unstructured data from internal and external sources; establish processes and data flows using cloud, local databases, and applicable storage formsBuild & Automate Data Transformation — Develop technical tools using ML and data-engineering techniques to cleanse, organize, and transform data; implement automated processes to maintain data structure integrity and quality standards on an ongoing basisDefine Agentic Evaluation Metrics & Ground Truth — Partner with product and AI teams to define evaluation metrics for agentic workflows (task completeness, tool use, workflow success); establish ground truth labeling standards and validation criteria; design evaluation datasets that reflect real-world agent executionBuild Agentic Evaluation Pipelines — Design and implement automated evaluation infrastructure that measures AI agent performance using Now LLM Service models and execution logs; create dashboards, reporting, versioning, and reproducibility of evaluation datasets and resultsEstablish Standards & Continuous Improvement — Create design standards and quality assurance processes for data systems; define quality gates and validation frameworks; analyze workflow performance and recommend optimizations to accommodate evolving CRM AI requirementsLead Cross-Functional Collaboration — Collaborate with product, engineering, and data science teams; mentor junior engineers on data engineering and evaluation design; generalize evaluation patterns and metrics across Agentic AI products
