RealBot
Idea: let someone looking at an apartment remotely do more than watch a walkthrough. They should be able to move around, inspect things, and interact with the space. With that, we built RealBot at Hack the North 2026 and won 1st for 'Best Use of BracketBot'.
We used BracketBot's hardware and existing software stack, then built the apartment-tour application on top. I worked on stereo calibration and built the local ACT training and inference pipeline for the door-opening task.
The head had a stereo camera pair; the grippers had cameras too. I already had experience building stereo rigs for Project Reality and Jarvis Lite, so the camera setup felt familiar. But now the measurements had to connect to a robot moving around an apartment.
The BracketBot team had someone going around to calibrate the cameras for each bot. He was busy helping other teams, and I wanted to get the calibration done so I could start working on mapping and waypoint positions. I made my own calibration scripts and printed an aruco board of our own and stuck it to some cardboard.
BracketBot already provided SLAM. My work here was working with the map and setting up the locations our application would use.
The Quest controllers let us move the arms and record demonstrations. We tried the wall cabinet, light switches, a kitchen cabinet and picking things up from the counter.
I also tried looking at the switches through the wrist camera and marking the parts we wanted to interact with. These were just experiments since we didn't decide to use a detection system in the end.
We hit a training issue early on where the robot mostly stayed still. I surfaced it to BracketBot, who diagnosed it. They explained that the task we were training (flipping a switch) was producing low loss early enough for training to stop prematurely - the task was too simple. And it makes sense... we were using pi0.5 for something that it could probably do without any training.
After a fix by the BracketBot team, the shared training and inference setup was still being worked on and held up by queues. I wanted a way to finish our manipulation task without depending on server availability. So I built a local training and inference pipeline around ACT, using LeRobot, and trained the policy directly on BracketBot's Jetson Orin Nano.
Training was a little slow initially, so I froze the ResNet-18 backbone and cached its image features instead of recomputing them every training step.
In roughly an hour, I had a locally trained policy that could open the door. It even worked in a run where the arms of the bot were not zero'd properly and we lifted the robot so its arms could reach. That was pretty cool.
The gripper came off - was being held only by plastic threads dug by a screw
The part we ran out of time on was getting click-to-drive working reliably. The map was picking up the robot's own hands and training wheels as obstacles. The navigation app already had some filtering, but it wasn't enough for our setup.
We demoed the apartment setup over Discord so the judges could see what was happening around the robot.
We won 1st Place, Best Use of BracketBot. And the prize was a robotic arm, which was exactly what I wanted. Before we'd even started building, I was already thinking about what I could make with it.
