Alibaba's research group has released Qwen-Drive 1.0, a single model that reads the three-dimensional structure of a road scene, answers questions about traffic, and plans the car's route — and it is free for the research community on Hugging Face, ModelScope and GitHub. The most useful line in the paper is not a benchmark number. It is the team's own admission that the model's stated reason for a maneuver does not always match the maneuver it plans, and that its reasoning can blur a red light in the distance with a child running into the road, two situations with completely different reaction budgets.
Start with what the paper is arguing against. Driving models are usually built by taking a general vision-language model and fine-tuning it on road data, mostly question-and-answer pairs. The researchers name two costs of that recipe. A model trained mainly on traffic questions does not reliably judge distance, object position or free space. And a model over-specialised on driving data loses part of the general knowledge it acquired in pre-training — catastrophic forgetting — precisely the knowledge that matters in rare and unexpected situations on the road.
Qwen-Drive 1.0 is built on Qwen3.5-4B, released in February, with two components bolted on. The first produces a top-down view of the surroundings: it detects objects in 3D, marks occupied space, and reconstructs the shape of the road. The researchers note that this component doubles as a measuring instrument — it shows how much spatial information the language model is actually pulling out of the camera images, as opposed to how much it can talk about.
All driving functions run through one shared language model. The two added modules consume its intermediate results: one builds a 3D model of the surroundings, the other plans the car's route for the next few seconds
Source: the-decoder.com
The second component, Planning Expert, uses the model's internal representations to plan the vehicle's future motion. Here the team ran the experiment that gives the paper its spine. When they trained only the added component and left the vision-language model untouched, spatial accuracy stayed low. A model can describe an image in exhaustive detail and still have no grip on the three-dimensional space in it. Real gains appeared only once the vision-language part was itself trained on spatial tasks. Understanding a road scene, in other words, has to be built on purpose; it does not fall out of image captioning.
The model reconstructs a top-down view of each scene from the camera images, showing cars, pedestrians and lane markings. On the right, the prediction is placed next to the real situation
Source: the-decoder.com
In Qwen's own tests, Qwen-Drive 1.0 beats specialised models in most driving and perception categories
Source: the-decoder.com
The system also narrates. In each scenario it states the reason for the decision it took — braking for an animal on the road, stopping at a red light — alongside the trajectory it intends to follow.
In each scenario the model explains the decision it made, for example braking before an animal on the road or stopping at a red light. The blue line shows the planned route
Source: the-decoder.com
Training runs in stages: first the perception module; then perception combined with question answering; then route planning; and finally reinforcement learning to shape behaviour.
The data assembly is the unglamorous part and probably the more replicable contribution. For the vision-language component the team merged 24 publicly available datasets of road scenes. The formats differed and some contained errors, so an AI model normalised the questions and answers and re-aligned them with the underlying data. The team also wrote its own examples explaining why the car should take a particular decision — which specific object should trigger braking.
When processing video from other vehicles with different camera configurations, the model detects significantly fewer objects. Suitable training data for such configurations is still scarce
Source: the-decoder.com
There is a product argument underneath the research one. In modern cars the infotainment system and the driving system are converging onto a single compute unit rather than two separate controllers. The authors' point is that a model which traded away general ability for driving scores fits that architecture badly: the cabin would still need its own model, and its own silicon, to handle conversation and open-ended questions. Preserving general knowledge is not only a robustness argument. It is a bill-of-materials argument.
On that measure the results hold up. Qwen-Drive 1.0 is well ahead of the base Qwen3.5-4B on questions about road scenes, with the largest gap on causal explanation — saying why the car should brake or turn. General ability survives: on non-driving tests the model barely loses ground against the base version, and on some spatial tasks it scores slightly higher.
Planning was tested at several levels of difficulty, up to a simulator where errors compound over time. There the reinforcement-tuned version cut off-road excursions from 24% to 12%. It also drove more cautiously and covered less distance overall.
That last clause deserves more weight than the paper gives it. Halving off-road excursions while travelling less far is not a clean win; a car that creeps has fewer opportunities to leave the road. Without knowing the distance penalty, 24% to 12% is a number whose denominator moved. And a good deal of the evaluation rests on test procedures the authors designed or reassembled themselves — the team is honest that individual metrics therefore say little about behaviour in genuinely difficult real traffic. Add the camera-configuration result, where detection degrades sharply on footage from vehicles with different rigs, and what you have is a model demonstrated on its own terms, on its own sensors, against its own benchmarks.
The explanation problem is the one I would not wave through. Qwen's separate HopChain benchmark, built to measure exactly this gap, found vision-language models misclassifying objects and confusing spatial relations while performing well on image-text matching tests — the benchmarks that made them look ready for this job in the first place. A system that produces a fluent justification decoupled from the trajectory it actually plans is not interpretable. It is worse than opaque, because the text invites a trust the planner has not earned, and a human supervisor reading "braking for the pedestrian" has no way to know the brake command came from somewhere else.
The forgetting problem the paper sets out to solve has a known shape. In 2023 Google DeepMind built PaLM-E, a model spanning language, images and robot control. After training on robotics data the smaller versions lost a substantial share of their language ability; the largest version, at 562 billion parameters, lost almost nothing. Qwen-Drive 1.0 is a 4B model claiming retention at a scale where PaLM-E did not get it for free, which is the result worth independent replication.
Fusing a language model with vehicle control also opens an attack surface that a conventional perception stack does not have. Researchers at UC Santa Cruz placed a sign with text on it in a camera's field of view and made the DriveLM driving system swerve sharply toward pedestrians crossing the road — while the system was correctly detecting those pedestrians. Perception worked. The text channel overrode it.
Research is meanwhile moving on to World Action Models, which also predict how the environment will change in response to the agent's own actions. Qwen is releasing Qwen-Drive 1.0 into that shift, openly and at a size researchers can actually run. What the community receives is a car-driving model that will tell you why it braked, with the authors' own warning attached that the sentence and the steering may have come from different places.