Wat AI

ML Engineer. Training a self-driving RC car with reinforcement learning.

You can't train a real RC car from scratch strictly in the real world. It would crash around thousands of times before learning anything useful. So: you train it in simulation, where crashing is free, and hope it transfers enough to the real thing. Then fine-tune from there.

When I joined, the project was basically a title and a tech choice. We knew we wanted to do an RL racecar with PyBullet, but that was about it. No track dimensions, no car scale, no hardware picked, no plan for what "done" meant. So I kind of just... took charge of the direction. Wrote docs so the team had something concrete to point at instead of vibes, figured out how to split the work so nobody was blocking each other, and handled the hardware side. I also made a cartpole example repo for people who were new to PyBullet.

I built the simulation environment and an interactive track creator. The whole point is that if you train on a ton of different tracks, the model should generalize. So when we put this on a real car, it already knows how to drive. All it needs to pick up is the physics of real life. That's the dream, anyway. I ran 40+ experiments testing PPO and SAC on different difficulties, and our best agent hit 96% of the theoretical max reward on the hardest track. Pretty satisfying to watch it nail corners it's never seen before. We're presenting this at CUCAI, the Canadian Undergraduate Conference on AI. Madhav, one of the directors, also got me into using AI for quick prototyping, which has been really useful.