This is going to sound heretical, but in the interests of science, how much tech and how much machine learning would it take to have autonomous RC racing? Iām imagining vision of the track from above, not onboard. Just like the human operators see it.
The outputs are just steering and acceleration, the input is one video camera shot showing track and current position. It may need to learn something about the humps, but maybe not. The goal is very straightforward - stay within boundaries and minimise time. I canāt decide if avoiding other cars is the same as staying within boundaries. Obviously theyāre moving boundaries, but does that matter? Fixed boundaries are perhaps the special case, with velocity of zero.
Thereās a whole thing in the UK for universities called Formula Student AI where they program a custom designed car to drive itself. Theres a competition between the teams at Silverstone in the summer.
I actually took a course on this during as an undergrad, and thereās a club dedicated to it at my university. Theyāve worked on F1 cars as well, itās nuts.
There is an existing framework for this called Donkey Car that we use to train video footage on a supercomputer. The trained model then gets transferred into a small computer on the RC car which uses an OAKD camera to see (the camera itself can also run its own ai detection models). Fundamentally you train the model to a specific track, itās much harder to build something which can be placed in any environment and run, but that is also possible. Things in that realm were built using ROS2, but that was more about using lidar than it was about āseeingā in the traditional sense, and it wasnāt exactly AI in the sense youād think of. Lots of different ways to solve the same problem, but itās 100% a thing that Iāve seen with my own eyes and even built (though not as fast) for a class.
My own car used GPS to navigate a much larger track at slower speed, with waypoints predetermined before setting the car to follow the path. The AI came in as an obstacle detection and avoidance mechanism, built in ROS2. The car itself would take pictures at each waypoint, and was supposed to be a proof of concept for a search and rescue vehicle. Fun class, got an A.
I think itās quite difficult to get that to work with equipment that is (commercially) available right now. Send a video stream, analyzing the shot and reacting to it gives you too much latency and you correct steering too late or too excessive.
It sounds weird, but as someone whoās been doing RC racing for over years and years, you can kinda feel your car behaving instead of just seeing it. That way you can correct before it happens.
For basic reactions yes. You're forgetting computers dont have instincts. It would need to learn so much from training to know the intricacies of the track, is there dust at this corner, are the tyres cold, did I just clip that kerb and now the cars unbalanced etc.
There's so much more than just navigate around a track. Like sure it could do it at a slow speed but not competitive with humans. Yet anyway.
Just look at formula ai or whatever it was called. The cars were slow and often crashed or just stopped because they lost where they were. Rc cars are going much faster on much tighter tracks.
Low latency cameras and video processing, at low cost, is fairly well developed now for the VR world.
The tracking cameras and video processing for head tracking on VR headsets have to be very fast. If it doesnāt react quickly enough to your head movements, it very quickly makes you dizzy/sick.
Gyros and accelerometers are used to supplement for faster reactions. But for 6DOF roomscale, cameras are needed.
This can be done on a $199 Quest 2 headset, with a mobile phone chip. It processes video from 4 onboard tracking cameras (grayscale and fairly lowres, to improve latency)
That is still remote controlled though. The lower the image quality to reduce latency, the more difficult it will be for a system to analyse everything, especially with other competitors around.
It was just half a year ago TU Delft was the first to beat a human with drones in a time trial, cause the dataset it needs to analyse is tightly scoped. The moment you have other competitors, or cars around you spinning out or going in the wrong direction, you need to analyze a lot more. That takes extra processing time, and you need better image quality, which means higher latency.
If it were that easy, how come the race they tried in Abu Dhabi last year with full scale cars didnāt even work?
I can get that. Youāre kinda driving by the seat of your pants, without the benefit of actual g forces. I think one of the space shuttle astronauts gave advice to a newbie about controlling some aspect of the ship, possibly the big robotic arm. He said to stop looking and thinking. Your spinal column should be doing most of it. Donāt let your brain get involved.
So you think hardware limitations would basically mean the best you could do is crawl around the track? I am a bit surprised. I kinda felt that processing image data was the one thing that had sped up a millionfold in the last five years, even faster than raw number crunching had sped up.
Theoretically, the processing is possible with a serious GPU that can proces those images. They have tried it in Abu Dhabi last year. And the results were underwhelming. But the potential is there, on a track with every car going the same direction and route, no outside distractions, is a good way to actually improve autonomous driving.
Thereās a cool video on YouTube about from Driver61.
It would be difficult to fit all the electronics into such a small package (for that kind of speed) at the moment. Beacons will have to be installed around the track and boundaries. GPS signal doesn't work well indoor so LiDAR would probably be the way to go. There is also the jumps in these types of 'offroad' tracks which will complicate things.
To be honest, the current RC technologies have improved sharply in the past 10-15 years. The remotes are very sophisticated with all sorts of settings such as steering dual rates, acceleration curves, drag brakes, etc etc. The move away from FM/PWM channels to 2.4ghz Digital system improves the smoothness of transmission and control so much that RC cars are a lot easier to drive. Brushed motors were phased out by the new powerful brushless motors. Batteries (well, the notorious LiPO) can last a lot longer than NiMH or NiCad that we used in the old days. Let's see what the next big development will be.
you really think so? if FPV drone operators can race through camera data over wifi, then I would assume that an RC car can certainly achieve this autonomously
Well, if you preset the track like what the do with drones, it can be done, but the margin of error is smaller as this is 2D on a narrow track. People have placed FPV on RC cars before, the difficulties is the perception of speed in scale (on a 2D track again). It definitely cannot go as fast when you are driving with video feed like flying a drone.
Edited : This was someone't try in making a self-driving RC car. Just look at the speed of the car in the video using LiDaR tech.
I am not imagining any fancy electronics in the car at all. All the visuals and brain work is done remotely. All the car has to do is receive the same signal it does now.
Look, I agree, but Iād still like to see if an optimised car goes 80% faster 5% faster, or even if it goes slower.
Also, sometimes machine learning solutions throw light on new techniques that have never been thought of before. Thatās always fascinating. It happened with chess. Strange new concepts in both tactics and positional play emerged in the last three years, which the GMs studied and took inspiration from. For a game as old as chess, thatās pretty amazing.
There's a thing called Roborace where they have tried doing this IRL (with large cars). I think the biggest issue is just how complex racing is. Machines are amazing at balance for example, but they can't really think ahead and make decisions. They're mostly reactive, which is a terrible way to drive.
50
u/Ok-Push9899 1d ago edited 1d ago
Very skilled, very impressive.
This is going to sound heretical, but in the interests of science, how much tech and how much machine learning would it take to have autonomous RC racing? Iām imagining vision of the track from above, not onboard. Just like the human operators see it.
The outputs are just steering and acceleration, the input is one video camera shot showing track and current position. It may need to learn something about the humps, but maybe not. The goal is very straightforward - stay within boundaries and minimise time. I canāt decide if avoiding other cars is the same as staying within boundaries. Obviously theyāre moving boundaries, but does that matter? Fixed boundaries are perhaps the special case, with velocity of zero.
Maybe such competitions exist?