PLATO (Platform Engineering and AI Technology Organization) at ServiceNow is a customer-focused innovative group building intelligent software using a variety of technology stacks to enable end-to-end, industry-leading work experiences for our customers. We are a group of people deeply invested in the success of our customers that happen to have expertise and knowledge in advanced technologies and software engineering best practices. We are data driven, structured, committed and we enjoy what we are doing. We prioritize robustness, performance and user experience over the technology stack and tools.What you get to do in this role:The AI Evaluation & Observability team, part of the Platform and AI organization, is focused on building a scalable, reliable platform for supporting evaluating of Gen AI applications supported on the ServiceNow platform.A consistent and effective evaluation solution helps build confidence and trust in the solution, improves clarity on how the application will respond to a range of inputs and insights into how to iterate on the feature.As a Director for AI Evaluation team, you will lead the teams responsible for building and productionizing our metrics, associated datasets and drive decision making for internal and customer facing use cases. This includes everything from the initial exploration to fine-tuning to rigorous evaluation and deployment. Your leadership will be critical in developing state-of-the-art techniques that push the boundaries of what our platform can do.This is a fast-growing, high-impact role where you will:Lead and scale a team of talented researchers, applied researchers and machine learning engineers.Define the strategy and roadmap for model evaluation, ensuring we deliver high-quality, performant, and reliable applications.Drive innovation in various ML techniques, including developing auto prompt optimizations, automated dataset curation among others.Establish best practices for model evaluation, including creating robust benchmarks, metrics, and frameworks to ensure model quality and integrity.Collaborate with cross-functional teams across product, engineering, and research to translate business needs into technical requirements and deliver impactful solutions.Stay ahead of the curve by researching and implementing the latest advancements in large language models and AI research.
