Staff Engineer, Test Automation - San Mateo (R5792)

Job description

Job responsibilities

• Own the technical strategy, architecture, and roadmap for automated testing, verification, and MLOps quality across the Hivemind ecosystem.
• Design and maintain scalable test frameworks for autonomy software, backend services, APIs, operator-facing applications, and distributed hardware environments.
• Lead functional, integration, regression, system, performance, reliability, and end-to-end testing across simulation, edge-compute, software-in-the-loop, and hardware-in-the-loop environments.
• Build automated ML validation pipelines covering data quality, training reproducibility, model accuracy, robustness, regression, latency, resource utilization, and system integration.
• Establish CI/CD and continuous training workflows that provide versioning and traceability for datasets, models, configurations, evaluation results, and deployment artifacts.
• Develop scenario-based validation for autonomy models, including edge cases, degraded sensing or communications, distribution shifts, and representative mission conditions.
• Create observability, analytics, and failure-triage capabilities for software behavior, model and data drift, inference health, test results, and production performance.
• Build Python automation that improves test execution, parallelization, reporting, environment setup, experiment comparison, and developer productivity.

Job requirements

• Typically 8+ years of relevant experience in software engineering, test infrastructure, developer tooling, MLOps, systems integration, or systems verification, or an equivalent combination of experience and…
• 5+ years of experience building scalable automation frameworks or developer tooling in Python.
• Demonstrated success designing test, CI/CD, or MLOps infrastructure used across multiple engineering teams.
• Experience validating machine learning systems across the data, training, evaluation, packaging, deployment, and monitoring lifecycle.
• Understanding of ML quality risks such as data leakage, training-serving skew, nondeterminism, distribution shift, drift, model regression, and statistical acceptance criteria.
• Experience defining model-performance baselines, automated evaluation suites, release thresholds, and candidate-to-production comparison workflows.
• Experience testing GPU-accelerated infrastructure and workloads, including GPU scheduling, allocation, utilization, and resource contention in Kubernetes environments.
• Experience with performance benchmarking, profiling, and observability for GPU workloads, including identifying compute, memory, storage, networking, and data-loading bottlenecks.