At Moody's, we unite the brightest minds to turn today’s risks into tomorrow’s opportunities. We do this by striving to create an inclusive environment where everyone feels welcome to be who they are—with the freedom to exchange ideas, think innovatively, and listen to each other and customers in meaningful ways. Moody’s is transforming how the world sees risk. As a global leader in ratings and integrated risk assessment, we’re advancing AI to move from insight to action—enabling intelligence that not only understands complexity but responds to it. We decode risk to unlock opportunity, helping our clients navigate uncertainty with clarity, speed, and confidence.

If you are excited about this opportunity but do not meet every single requirement, please apply! You still may be a great fit for this role or other open roles. We are seeking candidates who model our values: invest in every relationship, lead with curiosity, champion diverse perspectives, turn inputs into actions, and uphold trust through integrity. 

Skills and Competencies

• Ph.D. in Computer Science, Machine Learning, Natural Language Processing, Statistics, or a related quantitative field; or Master’s degree with 2-3 years of experience in machine learning evaluation or a related area
• Strong foundations in statistical methods, experimental design, and hypothesis testing
• Experience evaluating machine learning or NLP models, including designing experiments and interpreting results
• Familiarity with LLM evaluation benchmarks and methodologies
• Strong programming skills in Python or R
• Excellent communication skills in English (both written and verbal)

Preferred:

• Experience evaluating LLMs or generative AI systems
• Experience with production machine learning systems
• Exposure to cloud platforms such as AWS, GCP, or Azure
• Publications or demonstrated work in model evaluation, benchmarking, or related areas

Education

• Ph.D. in Computer Science, Machine Learning, Natural Language Processing, Statistics, or a related quantitative field; or Master’s degree with 2-3 years of experience in machine learning evaluation or a related area

Responsibilities

• Evaluate and validate large language models for production-grade analytical and decision-support systems
• Design and implement evaluation frameworks for assessing LLM performance in credit analytics and decision-support contexts
• Develop metrics and benchmarks to measure model robustness, reliability, consistency, and output quality
• Analyze model behavior across diverse inputs, identifying failure modes, edge cases, and areas for improvement
• Collaborate with model development and deployment teams to integrate validation processes into the model lifecycle
• Conduct systematic assessments of model stability over time and across updates
• Evaluate model outputs for bias, fairness, and economic relevance to credit risk applications
• Develop and maintain documentation for evaluation methodologies, findings, and recommendations
• Contribute to the advancement of best practices for LLM evaluation within the Credit COE

About the Team
Our Credit Center of Excellence (COE) team is responsible for maintaining and enhancing our industry-leading credit analytics and predictive modelling capabilities. We work closely with various teams including product management, commercial strategy, and go-to-market leaders to ensure the delivery of high-quality credit risk assessments and solutions. By joining our team, you will be part of exciting work in credit analytics with a global team spread across all US time zones, GMT, and GMT+1.

For US-based roles only: the anticipated hiring base salary range for this position is $116,500.00 - $169,000.00, depending on factors such as experience, education, level, skills, and location. This range is based on a full-time position. In addition to base salary, this role is eligible for incentive compensation. Moody’s also offers a competitive benefits package, including not but limited to medical, dental, vision, parental leave, paid time off, a 401(k) plan with employee and company contribution opportunities, life, disability, and accident insurance, a discounted employee stock purchase plan, and tuition reimbursement.

Moody’s is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, sex, gender, age, religion or creed, national origin, ancestry, citizenship, marital or familial status, sexual orientation, gender identity, gender expression, genetic information, physical or mental disability, military or veteran status, or any other characteristic protected by law. Moody’s also provides reasonable accommodation to qualified individuals with disabilities or based on a sincerely held religious belief in accordance with applicable laws. If you need to inquire about a reasonable accommodation, or need assistance with completing the application process, please email accommodations@moodys.com. This contact information is for accommodation requests only, and cannot be used to inquire about the status of applications

For San Francisco positions, qualified applicants with criminal histories will be considered for employment consistent with the requirements of the San Francisco Fair Chance Ordinance.

This position may be considered a promotional opportunity, pursuant to the Colorado Equal Pay for Equal Work Act.

Click here to view our full EEO policy statement. Click here for more information on your EEO rights under the law. Click here to view our Pay Transparency Nondiscrimination statement. Click here to view our Notice to New York City Applicants.
Candidates for Moody's Corporation may be asked to disclose securities holdings pursuant to Moody’s Policy for Securities Trading and the requirements of the position. Employment is contingent upon compliance with the Policy, including remediation of positions in those holdings as necessary.