Results
Every video is a trained model driving two simulated SO-101 arms by itself. Nothing is scripted: the cubes, the bin and the target land somewhere new each time, and the model sees only its three cameras and one sentence, such as “Hand the blue cube over to the left arm and put it in the bin.”
Below is everything we tried, in order. Each dot is one experiment; click it to see what changed and to watch it run.
keptdroppedrunning now
Each branch starts from the checkpoint it was trained from.
Scores count a trial as a success only when the object was really picked up, not pushed into place. Two-arm models are tested on the same 30 random layouts for each of the four tasks (120 trials). Scores before October 17 were measured with an earlier renderer; the same model scores 86 there and 90 now.
The main scores use the simulator’s plain default look, while every training example had random table and cube colours, lighting and small camera shifts. “Does it still work with different lighting and colours?” in the tree scores the same models with that randomness switched on.