How can robots purchase abilities via interactions with the bodily world? An interview with Jiaheng Hu

February 12, 2026

2

One of many key challenges in constructing robots for family or industrial settings is the necessity to grasp the management of high-degree-of-freedom programs similar to cellular manipulators. Reinforcement studying has been a promising avenue for buying robotic management insurance policies, nonetheless, scaling to advanced programs has proved tough. Of their work SLAC: Simulation-Pretrained Latent Motion Area for Entire-Physique Actual-World RL, Jiaheng Hu, Peter Stone and Roberto Martín-Martín introduce a technique that renders real-world reinforcement studying possible for advanced embodiments. We caught up with Jiaheng to search out out extra.

What’s the subject of the analysis in your paper and why is it an attention-grabbing space for examine?

This paper is about how robots (particularly, family robots like cellular manipulators) can autonomously purchase abilities through interacting with the bodily world (i.e. real-world reinforcement studying). Reinforcement studying (RL) is a common studying framework for studying from trial-and-error interplay with an surroundings, and has large potential in permitting robots to be taught duties with out people hand-engineering the answer. RL for robotics is a really thrilling discipline, as it might probably open potentialities for robots to self-improve in a scalable means, in direction of the creation of general-purpose family robots that may help folks in our on a regular basis lives.

What had been a number of the points with earlier strategies that your paper was attempting to handle?

Beforehand, a lot of the profitable purposes of RL to robotics had been carried out by coaching solely in simulation, then deploying the coverage within the real-world instantly (i.e. zero-shot sim2real). Nevertheless, such a technique has massive limitations: on one hand, it’s not very scalable, as you must create task-specific, high-fidelity simulation environments that extremely match the real-world surroundings that you simply wish to deploy the robotic in, and this will usually take days or months for each process. However, some duties are literally very laborious to simulate, as they contain deformable objects and contact-rich interactions (for instance, pouring water, folding garments, wiping whiteboard). For these duties, the simulation is commonly fairly totally different from the true world. That is the place real-world RL comes into play: if we will enable a robotic to be taught by instantly interacting with the bodily world, we don’t want a simulator anymore. Nevertheless, whereas a number of makes an attempt have been made in direction of realizing real-world RL, it’s really a really laborious downside since: 1. Pattern-inefficiency: RL requires a variety of samples (i.e. interplay with the surroundings) to be taught good habits, which is commonly inconceivable to gather in massive portions within the real-world. 2. Security Points: RL requires exploration, and random exploration within the real-world is commonly very very harmful. The robotic can break itself and can by no means be capable to recuperate from that.

May you inform us concerning the technique (SLAC) that you simply’ve launched?

So, creating high-fidelity simulations may be very laborious, and instantly studying within the real-world can be actually laborious. What ought to we do? The important thing thought of SLAC is that we will use a low-fidelity simulation surroundings to help subsequent real-world RL. Particularly, SLAC implements this concept in a two-step course of: in step one, SLAC learns a latent motion area in simulation through unsupervised reinforcement studying. Unsupervised RL is a method that permits the robotic to discover a given surroundings and be taught task-agnostic behaviors. In SLAC, we design a particular unsupervised RL goal that encourages these behaviors to be secure and structured.

Within the second step, we deal with these realized behaviors as the brand new motion area of the robotic, the place the robotic does real-world RL for downstream duties similar to wiping whiteboards by making choices on this new motion area. Importantly, this technique enable us to avoid the 2 greatest downside of real-world RL: we don’t have to fret about questions of safety for the reason that new motion area is pretrained to be at all times secure; and we will be taught in a sample-efficient means as a result of our new motion area is skilled to be very structured.

The robotic finishing up the duty of wiping a whiteboard.

How did you go about testing and evaluating your technique, and what had been a number of the key outcomes?

We take a look at our strategies on an actual Tiago robotic – a excessive degrees-of-freedom, bi-manual cellular manipulation, on a collection of very difficult real-world duties, together with wiping a big whiteboard, cleansing a desk, and sweeping trash right into a bag. These duties are difficult from three facets: 1. They’re visuo-motor duties that require processing of high-dimensional picture info. 2. They require the whole-body movement of the robotic (i.e. controlling many degrees-of-freedom on the similar time), and three. They’re contact-rich, which makes it laborious to simulate precisely. On all of those duties, our technique permits us to be taught high-performance insurance policies (>80% success fee) inside an hour of real-world interactions. By comparability, earlier strategies merely can not clear up the duty, and sometimes danger breaking the robotic. So to summarize, beforehand it was merely not doable to resolve these duties through real-world RL, and our technique has made it doable.

What are your plans for future work?

I feel there may be nonetheless much more to do on the intersection of RL and robotics. My eventual aim is to create really self-improving robots that may be taught solely by themselves with none human involvement. Extra not too long ago, I’ve been involved in how we will leverage basis fashions similar to vision-language fashions (VLMs) and vision-language-action fashions (VLAs) to additional automate the self-improvement loop.

About Jiaheng

Jiaheng Hu is a 4th-year PhD pupil at UT-Austin, co-advised by Prof. Peter Stone and Prof. Roberto Martín-Martín. His analysis curiosity is in Robotic Studying and Reinforcement Studying, with the long-term aim of growing self-improving robots that may be taught and adapt autonomously in unstructured environments. Jiaheng’s work has been printed at top-tier Robotics and ML venues, together with CoRL, NeurIPS, RSS, and ICRA, and has earned a number of greatest paper nominations and awards. Throughout his PhD, he interned at Google DeepMind and Ai2, and is a recipient of the Two Sigma PhD Fellowship.