Profile Description

We’re seeking someone to join our Enterprise Technology team as a Copilot Integration and QA Automation Engineer, in Workforce Technology & Services to to join our team as a Copilot Integration and QA Automation Engineer, with a primary focus on integrating Copilot and Agent capabilities and implementing QA Automation and observability practices across these experiences. This role will focus on Extending Microsoft Copilot capabilities across Microsoft 365 using Connectors and Agents, leveraging tools such as Microsoft Copilot Studio to deliver innovative solutions. Ensuring the stability, accuracy, and reliability of Copilot and Agent experiences through automated testing, continuous evaluation, benchmarking and observability mechanisms.

In the Technology division, we leverage innovation to build the connections and capabilities that power our Firm, enabling our clients and colleagues to redefine markets and shape the future of our communities.

This is Director position that manages and optimizes the technical environment and end-user experience across various workplace technologies for end-users, ensuring seamless operations and user satisfaction across the organization.

Since 1935, Morgan Stanley is known as a global leader in financial services, always evolving and innovating to better serve our clients and our communities in more than 40 countries around the world.

What you’ll do in the role:

• M365 Copilot: Demonstrate foundational knowledge of Generative AI and configure Copilot features across Microsoft 365 applications.
• Copilot & Extensibility: Strong understanding of the Copilot extensibility model, including MCP/Connectors, Copilot APIs, Actions, and Agents.
• Agent & Connector Development: Design, build, and manage integrations using Microsoft Graph and MCP Connectors with external and on‑premises data sources, and develop agents using Agent Builder, Copilot Studio, and the Power Platform.
• Technical Support: Serve as an escalation point for Connector and Agent integration issues, providing advanced troubleshooting and resolution.
• Cross-Team Collaboration: Collaborate with teams to onboard solutions into Microsoft 365.
  QA Automation Engineer: Implement automated testing, continuous evaluation, benchmarking and observability for Copilot, Agents and LLM-powered applications.
• Build reusable test harnesses and evaluation libraries to support continuous validation across teams.
  Diagnose LLM quality issues and lead root cause analysis, including stability monitoring and performance drift detection.
• Apply strong problem-solving and analytical skills to diagnose issues and deliver effective solutions.
• Stakeholders and cross-functional teams communication.
• Continuously learn and adopt new technologies to ensure successful delivery in a dynamic, fast-paced environment.

What you’ll bring to the role:

• A minimum of 5+ years of IT experience with a minimum of 2 years' experience delivering Microsoft 365 solutions.
• A degree in computer sciences or another related area.
• Experience in M365 and Copilot deployments.
• Track record of design and delivering Copilot Connectors, Copilot Agents and Integrations.
• Excellent communication and interpersonal skills.
• Ability to work independently and as part of a team.
• Prior experience of working in financial services or consulting firm preferred.
• Experience in QA automation for continuous evaluation, benchmarking and observability for Copilot, Agents and LLM-powered applications.

Technical skills

• Strong foundational knowledge of Microsoft 365 services, with hands-on experience Microsoft 365 Copilot.
• Strong understanding of Gen AI fundamentals.
• Strong knowledge of Connectors for extending Copilot and integrating external data sources using Connectors
• Expertise in implementing Agents using Microsoft Copilot Studio and Power Platform.
• Proven experience in building and deploying Copilot Agents and integrations in enterprise environments.
• Working knowledge of Copilot Integration APIs (e.g., Chat API, Retrieval API) or willingness to learn and develop expertise in this area.
• Knowledge of MCP servers, OAuth-based integration, and relevant architectures.
• Experience in QA Automation for continuous evaluation, stability, hallucination detection, and accuracy testing for Copilot, Agents and LLM-powered applications.
• Experience with Python, PowerShell and automation/integration work.
• Familiarity with multi-model benchmarking across Claude, Copilot, Grok, Gemini, and GPT or willingness to learn and develop expertise in this area.
• Familiarity with Code based, Human-in-the-Loop and Model-as-Judge approaches for evaluation workflows or willingness to learn and develop expertise in this area.

Business skills:

• Excellent communications skills.
• Experience working in globally distributed teams and liaising with engineering and vendor support team.
• Ability to multi-task at a high level.
• Strong problem solving and analytical skills.
• Stakeholder management.
• Collaborate with IT and governance teams to ensure agents comply with firmwide AI policies, including usage monitoring, security, and lifecycle management.

Personal profile:

• Strong sense of ownership
• Ability to deal with pressure and ambiguity
• Strong interpersonal skills
• Ability to take initiative and work independently
• Results-oriented
• Ability to influence and persuade
• High attention to detail
• Must be highly reliable, trustworthy, honest, and commitment oriented

WHAT YOU CAN EXPECT FROM MORGAN STANLEY:

At Morgan Stanley, we raise, manage and allocate capital for our clients – helping them reach their goals. We do it in a way that’s differentiated – and we’ve done that for 90 years.  Our values - putting clients first, doing the right thing, leading with exceptional ideas, committing to diversity and inclusion, and giving back - aren’t just beliefs, they guide the decisions we make every day to do what's best for our clients, communities and more than 80,000 employees in 1,200 offices across 42 countries. At Morgan Stanley, you’ll find an opportunity to work alongside the best and the brightest, in an environment where you are supported and empowered. Our teams are relentless collaborators and creative thinkers, fueled by their diverse backgrounds and experiences. We are proud to support our employees and their families at every point along their work-life journey, offering some of the most attractive and comprehensive employee benefits and perks in the industry. There’s also ample opportunity to move about the business for those who show passion and grit in their work.

To learn more about our offices across the globe, please copy and paste https://www.morganstanley.com/about-us/global-offices​ into your browser.

Morgan Stanley is an equal opportunities employer. We work to provide a supportive and inclusive environment where all individuals can maximize their full potential. Our skilled and creative workforce is comprised of individuals drawn from a broad cross section of the global communities in which we operate and who reflect a variety of backgrounds, talents, perspectives, and experiences. Our strong commitment to a culture of inclusion is evident through our constant focus on recruiting, developing, and advancing individuals based on their skills and talents.