A robot trained on internet-scale video can fold laundry, tidy a living room and operate a kitchen appliance from a spoken instruction. It still struggles to plug in a USB cable or turn a key in a lock. Vision-language-action models skip the sense humans use for precisely those jobs, and several labs are now trying to add it. Trevor Darrell's group at UC Berkeley reports 65% average success across 12 precision tasks — close to double the best VLA baseline — after training on 100 hours of tactile recordings. In Shanghai, a Fudan University team and its spinout NeoteAI have collected more than 30,000.
Darrell, a computer science professor at Berkeley, frames the gap simply: most precise manipulation is something a person can do with their eyes closed. Conventional cameras are poor instruments for judging force, slip and grip quality, and VLAs are built almost entirely on camera data.
Adding touch is harder than adding another camera. Tactile signals look nothing like the images these models are pretrained on, and tactile datasets are microscopic beside internet-scale collections of images and text. Darrell's team trained a model on existing datasets first, then bolted on tactile perception through a dedicated submodel. The tactile corpus came from 100 hours of high-quality recordings of ordinary actions — wiping, grasping, twisting, pouring — involving more than 200 different household objects. It became the T-Rex dataset, with an interactive viewer for a small slice of it.
Feeding that signal straight into the model did not work. A robot adjusting its grip has to react faster than a vision-language model typically responds, so the team split control between two experts. An action expert plans the motion. A tactile expert runs four times faster and rewrites that plan in real time based on what the robot feels. The combined model was then fine-tuned on roughly 100 teleoperated demonstrations of harder jobs: screwing in a lightbulb, squeezing toothpaste onto a brush, moving an egg from one tray to another. That is where the 65% comes from.
Darrell concedes the data was collected on a single hardware configuration, which is the field's structural problem rather than his. Robotic hands range from fully articulated five-fingered designs to simple pincer grippers, and tactile sensors do not even agree on physics — some measure changes in resistance, others photograph the deformation of a soft gel pad. Chengbo Yuan, a master's student at Tsinghua University in Beijing, points out the consequence: most tactile AI research is welded to one sensor type, so groups cannot share data or transfer what they learn.
Yuan's answer was to pool more than 3,000 hours of tactile data from public datasets spanning 21 sensor types and multiple robot designs, taking inspiration from the Open X-Embodiment collaboration, which gathered data from many robots and produced models that generalized to hardware they had never seen. His team built a hardware-agnostic model that converts different sensors' outputs into a common format and maps them onto marked positions on a template of a human hand. It beat the baseline noticeably, including on hardware it had not encountered, which Yuan attributes to training across such varied configurations giving the model "some general sense of tactile knowledge."
That normalization step is the most useful idea in this whole cluster of work, and it is getting less attention than the dataset sizes. Twenty-one sensor types is not a data problem, it is a standards problem, and the hand template is a proposal for a standard. Every hour of tactile data collected before such a format exists is worth a fraction of an hour collected after.
The scaling push is happening anyway. Yuan's group leads a collaboration of 80 organizations assembling a larger set of teleoperated demonstrations under a single recording and processing methodology. Fudan and NeoteAI have already produced a dataset an order of magnitude larger than anything before it, using their own sensor mounted on various robotic arms plus a handheld gripper operated by people, for more than 30,000 hours of demonstrations with synchronized vision and touch. On that data they trained a model that does not merely react to contact but predicts in advance what the robot should feel, using the prediction to steer the action and evaluate the result. Shunlin Lu, a Fudan researcher and NeoteAI's chief technology officer, reads the results as a direct link between access to large, diverse tactile data and performance.
A cheaper route is to mine the visual recordings robotics already has. Researchers at the University of Southern California recently showed a model that infers tactile information from images, trained on more than 2,700 demonstrations of household manipulation captured with a handheld gripper that logged touch data and camera footage at the same time. The model learned to associate the moment of contact with the pressure the sensors registered, giving robots with no tactile hardware at all a rudimentary sense of touch — most useful, the team found, in contact-rich tasks. The broader ambition is a generator that retrofits tactile data onto existing visual datasets.
Set against the enthusiasm, the most honest line in the field belongs to Yuan: so far, the main effect of tactile learning is that robots get faster at things they could already do, like picking things up and moving them. Tasks that are fundamentally impossible without touch, he suspects, may need new algorithms rather than more hours. That is a striking admission underneath a data-collection program of 80 organizations. Darrell's benchmark tasks make the same point from the other side — a lightbulb, a toothbrush, an egg are all things a sighted robot can attempt. Touch raised the hit rate from roughly a third to roughly two thirds. It did not add a capability that was not there.
Long Cheng of the Chinese Academy of Sciences argues the bottleneck is the shape of the signal, not its volume. Vision supplies a dense continuous stream; touch arrives rarely and in bursts, so models learn to ignore it. His fix, to be presented at IROS 2026 later this month, predicts from vision alone what the robot should feel and compares that against the actual reading. A large discrepancy means the sensor caught something vision missed, so surprising signals get amplified and predictable ones damped. Across five contact-rich tasks the method reached 62.8% average success against 28.2% for the same model without touch.
Lu is the optimist. He compares tactile scaling to what happened in language and vision and estimates that roughly 100,000 hours of recordings gathered in varied real-world settings, not just laboratories, could give robots capabilities they do not have now. He calls tactile intelligence the next stage of physical AI.
Two incompatible theories are being funded at once. Lu's says the missing ingredient is hours; Cheng's says it is a learning rule that stops models from discarding a sparse channel. The 80-organization collaboration will answer the first question by brute force within a couple of years. If Cheng is right, it will produce the data and the robots will still not be able to turn a key.