About the Role
We're looking for an engineer to own how we know whether our robots are actually getting better. As we move from research to production, we need evaluation that's rigorous enough to trust and fast enough to keep up. You'll build the harnesses, task suites, and regression infrastructure that turn "the demo looked good" into a number we can act on.
What You'll Do
Own the evaluation strategy for our policies in simulation and on real hardware.
Make evaluation results trustworthy and reproducible; the same evaluation, run by different people on different days, gives the same answer.
Ensure a checkpoint that improves one behavior can't silently break another, and that releases are gated on it.
Establish sim-to-real correlation, so we know which simulation results we can trust.
Own the internal benchmark our models are measured against, and keep results comparable as it evolves.
Know where we stand against the state of the art, benchmarked against external baselines and published results.
Make evaluation cheap enough to run continuously, without burning robot hours on every checkpoint.
Give the company clear answers to "is this checkpoint better than the last one" and "what is it failing at."
What We're Looking For
Built evaluation, validation, or benchmarking systems that a team made real decisions from.
Fluency with model training and evaluation workflows: comfortable reading a training curve, reasoning about generalization, and telling a real improvement from an overfit one.
Strong software engineering and programming skills.
Practical statistics for experiment design: sample sizing, significance, and knowing when a result is noise.
Demonstrated skepticism: a track record of finding the flaw in a result that everyone else was ready to accept.
Bonus Points
Experience in robotics: manipulation, locomotion, or mobile autonomy.
Experience with simulation environments (Isaac, MuJoCo, Gazebo) and sim-to-real transfer.
Experience evaluating imitation-learned or RL policies specifically.
Summary
Own evaluation for robot policies in simulation and on real hardware, building harnesses, task suites, and regression infrastructure to gate releases. Requires built evaluation systems used for real decisions, fluency with training curves and generalization, strong software engineering, and practical statistics.
Responsibilities
Own evaluation strategy for policies in simulation and on real hardware; Make evaluation trustworthy and reproducible; Gate releases on regression results; Establish sim-to-real correlation; Own internal benchmark and external baselines
Qualifications
Built evaluation, validation, or benchmarking systems that a team made real decisions from; Fluency with model training and evaluation workflows; Strong software engineering and programming skills; Practical statistics for experiment design