Staff, Robotics ML/Data Engineer

Job description

Job responsibilities

• Force Analysis & Hidden State Inference: Design cross-modal validation systems that verify video, proprioception, force/haptic signals, and language annotations agree with each other, e.g., reprojecting robot state…
• Kinematic Retargeting & Alignment: orchestrating hand-tracking, segmentation, depth estimation, 3D reconstruction, and pose-tracking modules; retargeting human demonstrations into robot trajectories;
• Advanced Data Augmentation: Implement robust data augmentation strategies (spatial transformations, temporal scaling, synthetic viewpoints, and sensor noise injection) to expand expert trajectories and improve the…
• Teleoperation Synchronization: unified state–action representations across differing embodiments, coordinate frames, rotation conventions, gripper/hand parameterizations, and sampling rates, with per-dimension validity…
• Close the loop with data consumers : build the tooling that lets researchers query, visualize, and audit datasets (clip browsers, trajectory viewers, annotation review UIs), and turn model-failure analyses into new…
• Multimodal Data Pipelines: Architect end-to-end ingestion pipelines that take raw, unstructured recordings (egocentric video, teleoperation sessions, third-party open datasets) and produce indexed, queryable,…

Job requirements

• Education: M.S., or Ph.D. in Computer Science, Data Engineering, Machine Learning, Robotics, or a related field.
• Programming & ML Frameworks: Deep expertise in Python and extensive experience with PyTorch, specifically in handling custom dataloaders for multimodal datasets.
• Force & Time-Series Data Processing: Experience analyzing and processing complex time-series data from force-torque (F/T) sensors, load cells, or tactile arrays, ensuring pristine alignment with visual frames.
• Video Processing Expertise: Mastery of video processing pipelines and libraries (OpenCV, FFmpeg, Decord) and managing the I/O bottlenecks of terabyte-scale video datasets.
• Solid working knowledge of 3D geometry and robotics data: coordinate frames and transforms, rotation representations, camera intrinsics/extrinsics, forward/inverse kinematics, URDF.
• Data Augmentation: Proven ability to implement programmatic and generative data augmentation techniques for computer vision and time-series data.
• Experience with NVIDIA’s robotic software stack (Open X-Embodiment, DROID, AgiBot World, EgoDex, or similar).
• Familiarity with the modern perception toolbox as a user: segmentation (SAM-family), monocular depth, hand/body pose estimation (MANO/SMPL), 6-DoF object pose tracking, point tracking—you don't need to train these…