Machine Learning: Multimodal Foundation Models

Job description

Job responsibilities

• Build Native Multimodal Policies: Develop architectures where vision, language, and more modalities share a unified representation.
• Improve Cross-Modal Reasoning: Research and implement methods to ensure the model doesn't just "associate" modalities but actually reasons through them (e.g., grounding visual physics in kinematic constraints).
• Own the Training Loop End-to-End: Design, run, debug, and iterate on large-scale training experiments; diagnosing failure modes, improving data mixtures, and tightening evaluation to drive measurable gains.
• Ship and Iterate on Real Systems: Integrate models into real robotic stacks, build on robot code to deploy your models, and optimize performance for edge inference.

Job requirements

• Very strong coding skills in Python, C++, or Rust.
• Production MLLM Experience: Track record of training and deploying large-scale multimodal models.
• Pretraining & RL Mastery: Deep intuition for LLM-style pretraining, post-training, and Reinforcement Learning at scale.
• Infrastructure Fluency: Comfortable managing and optimizing large-scale experiments on massive GPU clusters.