DrivingBench: Can frontier LLM models drive a real car? We give them control of a Toyota Corolla’s steering, accelerator, and brakes, then evaluate them on a fixed cone course.

cross-posted from: https://piefed.world/c/tech/p/1434397/drivingbench-can-frontier-llm-models-drive-a-real-car-we-give-them-control-of-a-toyota-c

drivingbench.com/report/

Why the hell would you give a language model the wheel of a car?

They don’t even understand the concept of time and space. Some of them can parse an image, but typically only enough to pull text from it, and extremely slowly.

It’s like asking an overweight English teacher to play pro football.

We built visual recognition machine learning systems for this purpose.