Väiski is a Unitree G1 with two Dex3 hands. Type an instruction in plain English and it sorts cubes, cylinders and prisms into the right bins on its own. It runs fully offline at 30 Hz, and if it isn't sure, it doesn't guess.
Every change sat behind a flag that was off by default. We A/B-tested each one on 20–30 seeded episodes in parallel cloud Isaac Sim before adopting it.
objects sorted in the latest sim A/B (baseline: 52)
wrong-bin placements and 0 safety clips
instruction resolver, blind held-out expected value
open-vocabulary test with the brain on (14 without it)
One Docker image serves the policy over a websocket. Inputs: the head RGB-D camera, two wrist cameras, joint states and the instruction text. Outputs: 28 joint targets, 30 times a second.
A grammar-based resolver maps the instruction to {colour / shape → bin} in 0.75 ms. When it's unsure, it abstains instead of guessing.
pure pythonA local Qwen3-VL-8B picks exactly one tool call (pick_place, skip or find) under a GBNF grammar. It can only decide where the resolver has no opinion.
0.15–0.3 s · offlineHSV colour segmentation, depth back-projected into the pelvis frame, cube/cylinder/prism classification and bin detection, with online camera-pitch self-calibration each episode.
classical CVChooses the next object and a grasp posture by shape: approach, close, lift, carry, release. After a miss it retries with a different grasp.
shape-awareIK with joint-limit clamps and velocity limits. Every plan gets a safety dry-run first, and the command loop never waits for the brain.
30 Hz loop“Red to the left, the rest to the right.” “Put cylinders in the centre.” It follows colour, shape, British spellings, paraphrases and leftover-bin phrasing.
The brain's find() tool locates objects from spatial, ordinal and relational phrases. On real robot frames: 96.5 % hit rate with 0 false positives.
Pinch grasps for cubes, a dedicated posture for cylinders and yaw-aligned grasps for prisms, plus a guard for inserting through the shape-hole lid.
A ball, marker, box or toy pan gets a generic grasp whose width limit comes from the URDF. A pan is picked by its handle, never the disc. Unknown objects become obstacles.
A correct sort scores +2, a wrong bin −1 and an object left on the table 0. So it acts only when p > ⅓.
Every feature sits behind a flag. With the flags off, behaviour is identical to the version we already validated.
Finals hardware is an x86 PC with an RTX 5090 over ethernet. Everything runs locally, the 8B VLM included.