Self Driving Vector Space

In Tesla's latest FSD Investor schpeel, Andrej Karpathy does a great presentation about using cameras to create a vector space.

What is a vector space? Well, it is basically the most minified 3d model you can get. There would be 3d co-ordinates [latitude, longitude, altitude] and functions contained within these coordinates that traverse this 3d space as lines, or planes. Bounding boxes are formed around objects to further the required vertex math. As they said, its linear algebra and I would add multi-variable calculus.

The linear algebra would be used to apply geometric translations on the objects. Multi variable calculus will allow you to compute expected trajectories with regards to speed and acceleration which is vital for collision detection.

Neural nets can be programmed to detect functions and find best fit functions. So instead of storing many points, you can store a small string that can be computed over R. Such a [tan(latitude) - 18, 2.75*longitude, 3+groundLevel].

That function would never actually work, but it could create functions which you could traverse for seconds in time and determine if there are any collisions in your bounding boxes.

It is one small piece in terms of full self driving. It is a mathematical projection that renders a view. But it is very crucial in terms of understanding the world around. It will be that world where intersecting trajectories will determine if collisions are avoided.

The car will likely know if you are about to get a water balloon thrown at you. And what should the car do in that situation? Whatever they tell it to.

The default will be nothing. They have to see the situation classify it and output the correct action.

What is the correct action? Well, what if somebody throws a water balloon at you? The default is do nothing. I would say you can safely do nothing. Take the water balloon hit. But maybe the output is to honk the horn Dukes of Hazard style. Defaults can be easily changed. Perhaps this is why Tesla was so keen on installing the Pedestrian Noise Maker

Posted In:

jsturgis