Are you passionate about building a platform that leverages crowd? Do you love working with an energetic and cross-functional group of people? Does pioneering innovative ways to understand and interact with LLMs sound exciting to you? Are you passionate about quality of the products and want to contribute to product improvement driven by insights from crowd and LLM? If so, Crowd Intelligence Platform team might be an excellent place for you to grow your career.The Crowd Intelligence Platform team within Bing Fundamentals is the central team that evaluates the quality of products within Microsoft AI leveraging crowdsourcing and LLM’s. We have an ambition to scale to all products in Microsoft, providing necessary infrastructure to schedule & distribute a large number of product scenarios for validation. This system leverages the power of crowdsourcing to auto generate a user perceived quality score representative of the health of functionalities offered by the product, to enable teams to be in control of their feature quality and take pride in their product. The service is expected to run 10s of thousands of scenarios every day for both webpages and apps starting with deployment of the right version of operating system/hosting app with appropriate settings and connecting with relevant judges across multiple crowdsourced platforms. In addition, the service will include capabilities to capture the validation process for future debuggability, built in intelligence to target high fidelity judges, ability to dynamically update the pricing for these tasks and support real time routing across platforms to optimize for cost. To name a few products this service is expected to support - Bing, Maps, Edge, Windows Feeds.We are seeking a Senior Software Engineer to join our Crowd Intelligence Platform team in Microsoft AI. A suitable candidate is expected to have experience of designing and delivering reliable & scalable distributed services. Understanding of crowdsourced platforms and or prompt engineering experience will be a plus. The role requires quick prototyping, narrowing down design options to deliver on the vision in collaboration with a strong team of software engineers.Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.Responsibilities
Work with PM team to create MVP scope.
Design the system at scale for scenarios and canvases to be supported.
Design Engineering systems that leverage LLMs in a performant manner.
Collaborate closely with partner teams in Crowdsourcing and other dependencies.
Implement some core components and manage deployment, live site
Iterate the service capabilities as new requirements emerge with service rolled out.
Act as a Tech lead for the area.
QualificationsRequired Qualifications:
Bachelor's Degree in Computer Science, or related technical discipline AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C, C#, Java, JavaScript, or Python
OR equivalent experience.
3+ years of experience of designing, coding, debugging and delivering software projects
Experience with designing and building reliable, distributed services
Preferred Qualifications:
Master's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C, C#, Java, JavaScript, or Python
OR Bachelor's Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C, C#, Java, JavaScript, or Python
OR equivalent experience.
Demonstrated problem solving and prioritization skills
Experience with Search products, Telemetry, Metrics.
Experience of using LLMs and ML in engineering systems will be preferred.
Knowledge of Crowdsourced platforms.
Software Engineering IC4 - The typical base pay range for this role across the U.S. is USD $117,200 - $229,200 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $153,600 - $250,200 per year.Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-payMicrosoft will accept applications and processes offers for these roles on an ongoing basis.Microsoft is an equal opportunity employer. Consistent with applicable law, all qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations (https://careers.microsoft.com/v2/global/en/accessibility.html) .
