Conversational AI Specialist, Director

Prompt Engineering: Develop and refine context-aware prompts tailored to different use cases, domains, and user intents with a view of improving the accuracy, relevance and efficiency of AI generated chatbot responses. Ensure these prompt solutions are scalable, sustainable, effectively integrates with technical infrastructure, and successfully meet the specific needs of the business. Testing and Evaluation: Develop and maintain robust frameworks for testing and evaluating Generative AI responses for coherence, bias, factual accuracy and user satisfaction. Create and manage comprehensive test sets and human evaluations to ensure LLM Model outputs meet high standards. Troubleshooting and Optimization: Identify and troubleshoot issues and anomalies in the LLM model outputs and provide feedback and recommendations for solution improvement and optimization. Stakeholder Management: Establish and maintain effective communication channels with internal stakeholders such as tech partners and business product owners. Manage expectations and address any concerns or issues that arise. Collaboration: Collaborate with cross-functional teams, including internal stakeholders and vendor partners. Risk Management: Identify potential risk and issues that may impact project delivery and develop mitigation strategies to address them proactively. Monitor project performance and implement corrective actions as necessary to ensure successful outcomes. Performance Monitoring and Reporting: Track project performance metrics and KPIs to measure progress against objective and identify areas for improvement. Experience with Conversational AI solutioning / product development / experimentation or Innovation. Experience with prompt engineering for using Commercial or Open Source LLMs such as GPT, Claude, Gemini, LLAMA. Experience with conversational AI evaluation techniques such as F1, BLEU, ROUGE, and human feedback analysis. Strong understanding of artificial intelligence concepts and technologies. Excellent communication, interpersonal and leadership skills. Proven experience in program or project management Ability to manage multiple projects simultaneously in a fast-paced environment. Solid analytical and problem-solving abilities with a keen attention to detail At our foundation are five core values - putting clients first, doing the right thing, leading with exceptional ideas, committing to diversity and inclusion, and giving back - that guide our more than 80,000 employees in 1,200 offices across 42 countries. We Firm is differentiated by the caliber of our diverse team. While our company culture and commitment to inclusion define our legacy and shape our future, helping to strengthen our business and bring value to clients around the world. Learn more about how we put this commitment to action: morganstanley.com/diversity. We are proud to support our employees and their families at every point along their work-life journey, offering some of the most attractive and comprehensive employee benefits and perks in the industry. We're committed to bringing passion and customer focus to the business. Expected base pay rates for the role will be between $90,000 and $155,000 per year at the commencement of employment. Consequently, our recruiting efforts reflect our desire to attract and retain the best and brightest from all talent pools. We want to be the first choice for prospective employees. It is the policy of the Firm to ensure equal employment opportunity without discrimination or harassment on the basis of race, color, religion, creed, age, sex, sex stereotype, gender, gender identity or expression, transgender, sexual orientation, national origin, citizenship, disability, marital and civil partnership/union status, pregnancy, veteran or military service status, genetic information, or any other characteristic protected by law.