I received a couple of requests to expand on how I got the AR Drone to fly though a hula hoop, as seen in this video:
The first step is identifying the hoop in the image. We take a binary mask for green pixels above a certain threshold. Since the hoop is wrapped in bright green tape, only the green pixels belonging to the hoop make it through the mask. However, the resulting binary image is often missing parts of the hoop. It could be partially off screen, such as in the image above. Therefore, a blob detector would fail to identify the true center of the hoop.
My solution is to fit the equation of a circle to binary data, much in the same way as Excel will fit a linear or quadratic trend line to data. Our goal is to minimize the expression printed above the hoop picture, where r is the radius of our circle equation, a and b are the center coordinates, and Xi & Yi are the location of each nonzero pixel. Luckily for us, there is a closed form solution to this problem, printed in the top right of the slide. We can evaluate this expression by calculating the summations listed below. This is what happens for each new image we receive from the drone. From this process we know the center and radius of our hoop.
That information is fed into a four state controller. We manipulate yaw and climb rate to orient the AR Drone to center the hoop in the front camera’s field of view, pitch to propel the drone through the hoop, and roll to stop any sideways drift that would slide our drone right past the hoop.