I'm Interested

Job Details

Meta Reality Lab’s Codec Avatar Research team is building technology to enable immersive, photorealistic social presence. Codec Avatars are real-time live-drivable representations that match the appearance of their users. As part of the Lab’s Instant Codec Avatar group, you’ll work to scale up Codec Avatar technology by modeling the diversity of human appearance and applying that model to the process of rapidly generating new avatars.This role is focused on our Large Scale Experimentation efforts, which both support our new Research Supercluster compute resource and uses that resource to run large-scale machine learning experiments that advance the state-of-the-art in Codec Avatar technology. In this role, you will lead a team of software engineers, research engineers, and research scientists to plan and deliver software systems needed to support large scale model training over thousands of GPUs. These systems ingest, store, and serve some of the largest ML training datasets in the world, and coordinate complex workflows composed from a mixture of traditional graphics and ML algorithms. You’ll also plan, design and execute research experiments using those workflows to advance our understanding of how appearance modeling scales over large populations.

Codec Avatars Large Scale Experimentation Lead Responsibilities:
  • Develop and debug machine learning workflows on a large multi-node cluster
  • Automation of data ingress into cluster
  • Implement compute allocation policy for the cluster
  • Define and implement strategy for compute environment management and deployment
  • Development of data read/access layer using proprietary framework
  • Define and communicate cluster software requirements, based on research needs
  • Enabling adoption of the cluster by additional research cases
  • Definition, design and implementation of automated testing
  • Point of contact for hardware & software questions regarding cluster capabilities
  • Reporting on progress, presenting technical risks, challenges and status to executive management
  • Partner with Data Collection and Asset Generation teams to specify and ingest assets required for large scale training
  • Partner with Codec Avatars Universal Avatar Research team to support large scale experimentation based on Python workflows
  • Partner with Research SuperCluster production engineering team to support reliable operation
  • Partner with Research SuperCluster storage engineering team to support development of features required for Codec Avatars datasets
  • Partner with security, privacy, and policy teams to ensure workflow compliance with company policy

Minimum Qualifications:
  • Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience.
  • Experience providing technical leadership for teams of 5 or more engineers
  • Experience with multi-node ML training workflows and frameworks
  • Experience developing and debugging distributed systems
  • Experience operating in a self-directed environment with multiple stakeholders across multiple teams
  • Proven communication skills, including experience driving decision making
  • Experience working with cross functional teams including hardware, software, network, legal, privacy and security
  • Proven Python experience
  • Proven Linux/shell scripting development experience
  • Experience developing and supporting reliable multi-stage data pipelines
  • Proven quantitative reasoning skills, analyzing trade-offs of different hardware and software solutions

Preferred Qualifications:
  • Experience providing technical leadership for teams of 12 or more engineers
  • Masters or higher degree in Computer Science or related technical field, or equivalent experience
  • 8+ years of experience in ML or distributed systems
  • Experience developing or applying computer graphics algorithms
  • Experience developing or applying computer vision algorithms
  • 5+ years of experience developing workflows for large scale AI training
  • Understanding of deep neural network training
  • Experience with securing sensitive data (encryption, access control, audit logging)
  • Experience with HPC (High Performance Computing)
  • Experience with scheduling systems such as Slurm or Kubernetes
  • Experience with large scale object storage services (S3 or similar)
  • Experience in research or converting research to products
  • Experience using git
  • Experience using Conda
  • SQL databases experience
  • Modern C++ development experience

About Meta:
Meta builds technologies that help people connect, find communities, and grow businesses. When Facebook launched in 2004, it changed the way people connect. Apps like Messenger, Instagram and WhatsApp further empowered billions around the world. Now, Meta is moving beyond 2D screens toward immersive experiences like augmented and virtual reality to help build the next evolution in social technology. People who choose to build their careers by building with us at Meta help shape a future that will take us beyond what digital connection makes possible today—beyond the constraints of screens, the limits of distance, and even the rules of physics.

Meta is proud to be an Equal Employment Opportunity and Affirmative Action employer. We do not discriminate based upon race, religion, color, national origin, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender, gender identity, gender expression, transgender status, sexual stereotypes, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics. We also consider qualified applicants with criminal histories, consistent with applicable federal, state and local law. Meta participates in the E-Verify program in certain locations, as required by law. Please note that Meta may leverage artificial intelligence and machine learning technologies in connection with applications for employment.

Meta is committed to providing reasonable accommodations for candidates with disabilities in our recruiting process. If you need any assistance or accommodations due to a disability, please let us know at accommodations-ext@fb.com.
We're connecting diverse talent to big career moves. Meeting people who boost your career is hard - yet networking is key to growth and economic empowerment. We’re here to support you - within your current workplace or somewhere new. Upskill, join daily virtual events, apply to roles (it’s free!).
Are you hiring? Join our platform for diversifiying your team
Codec Avatars Large Scale Experimentation Lead
I'm Interested