What could we do with a Pi 3 powered YetiBorg?

With any form of racing, alterations to the rules changes both what teams can do and what they should focus on. For example a race series which has a limited fuel supply as part of the rules would encourage teams to concentrate more on fuel efficiency than if there was no such rule. Our brand new MonsterBorg will change a lot of what the robot offers when it starts Formula Pi later this year:

1. Faster speeds
2. Improved steering response
3. Even more robust
4. Drives better when upside-down
5. Pi 3 based instead of Pi Zero based
6. Extended battery life
7. Less chance of robots being powered off by accident

I thought it would be good to take a look at how swapping the Raspberry Pi Zero for a Raspberry Pi 3 would change things for the Race Code, and why it will matter so much to our teams.

But to start things off, why did we choose a Pi Zero for the YetiBorg in the first place?

Why use a Raspberry Pi Zero for Formula Pi?

So there are a few things which make the Pi Zero great for building robots:

1. Low power consumption
2. Small size
3. Very low cost

While the small size does help (a Pi 3 barely fits) the main help here is the low power consumption. This is especially important as we are using a single rechargeable 9V battery for the entire race!

The main reason we chose the Pi Zero though is none of these things. We decided to use the Pi Zero because it is not as powerful as the other Raspberry Pi models. Like with the fuel efficiency example above we wanted to encourage teams to make efficient code changes. As a bonus this should demonstrate that the Pi Zero is powerful enough to do more complex tasks just like the bigger Pis :)

So why would I want a Pi 3 instead?

There are a lot of advantages to using the Pi 3 over the Pi Zero for robot building as well:

1. Faster quad-core processor
2. Twice the memory
3. Easier cable access (USB / Ethernet)

While the cable access is always helpful for diagnosing and setting up robots, they are not so useful for Formula Pi. What is useful is the extra horsepower for running software!

As our MonsterBorgs have access to the processing speed of a Pi 3 in Formula Pi it will open up the options available to teams. Those who have been following our forum conversations will probably know that a few teams struggled to make the changes they wanted to make within the limited power of the Pi Zero on their YetiBorg. However simply having a more powerful processor on its own does not do much, the code needs to actually make use of this extra speed!

There are two basic approaches to making use of our new found power:

1. Run the existing code faster
2. Have more complex processing code

Getting the code to run faster is a simple task of turning the frame rate from the camera up. This increases how many times the processing is performed each second. The strange thing is that we can only turn it up a little bit on a Pi 3, we should be able to run much faster. So why are we not going as fast as we could? Because to use the whole processor we need multi-threading.

What is multi-threading and why do I need it?

Multi-threading, often simply called threading, is where software is written so that multiple parts run at the same time as each other. Put simply this is how the programs you use can do more than one thing at the same time.

The reason we need to have the code threaded is due to how the processor in the Raspberry Pi 3 is designed. Like most modern PCs and more powerful smart phones the Pi 3 has a quad-core processor. What this means is that it is like having four miniature processors, each doing different tasks at the same time.

One way to think of this is a road with four separate lanes, and our program as a queue of traffic.

As you can see this is not a very good use of the road, our traffic is only using a quarter of the potential of the road. Now if we split our traffic into four separate queues...

Much better :)

In the same way we can split our code so that it runs as multiple queues of work running at the same time. We can do this by starting to process the next image from the camera without waiting for the first to finish.

In fact our standard code actually has several of these threads already:

1. The main thread
   Responsible for starting and ending the script, it manages the other threads
2. The image capture thread
   Grabs each new image from the camera and passes it to the processing thread
3. The image processing thread
   Takes an image and works out where the track is and any other processing
4. The control thread
   Takes input from the image processing and the race code and works out the power levels to set the motors to
5. The race code thread
   Runs the code which decides where the robot should be on the track and informs the control thread

This all sounds great but there is a problem, almost all of the time is spent in the image processing thread. This would be like having a large queue in one lane and a single car or two in the others. We can clearly do better than that.

What we really need is multiple copies of the image processing thread. With multiple image processing threads the image capture thread can give each new image (or car in our analogy) to the next available image processing thread. This results in filling up each processing core (or lane) with roughly the same amount of work to do, solving our problem :)

Those of you who have access to our standard code may have noticed that the code seems to do something that looks like this already, you would be right! The current code has a setting in Settings.py which controls how many image processing threads are created: processingThreads. Changing this to 4 will spread the load to all four cores, then you can go ahead and increase the frameRate setting.

On its own this increase in speed will improve how the MonsterBorg handles compared to a YetiBorg, as Team Enigma have demonstrated (see our explanation here). However to get the best results teams have the option to look beyond simple tweaking and try to improve the code in more significant ways.

What else could we do with the additional processing power?

So now we have a bit more processing time to play with I think we should take a look at some areas of the code which are worth considering for improvement but are probably a bit too much for the Pi Zero to handle.

Camera resolution

Our standard setting uses a very small image to keep the processing time short. Now that this is less of a problem the image size can be increased. This will help the accuracy of the image slightly, but should also allow more detailed analysis of what is ahead beyond simple lane detection.

Traffic recognition

The standard code decides there is another robot to overtake because there are too many unexpected edges in the image. This could be improved greatly by attempting to match shapes or colours for a MonsterBorg or YetiBorg.

Accurate position on the track

Our standard code currently looks at the track offset slightly in front of the MonsterBorg. This could be replaced by a more accurate version which tries to calculate where we are at the moment using some maths and maybe previous images as well.

Modelling the track

At the moment we are taking the identified points on the track and generating an indication of the track curvature. This is not too accurate and is confused by features like the S-curve. Instead a curve fit for the positions could be done which will try and predict where the track is actually going to mimic the same curve.

What can we do to push the limits though?

The suggestions above are a good start, but they still keep the same basic code doing most of the work. For those who feel like a challenge it would be possible to really make use of the Pi 3 by implementing a more complex algorithm. This means technologies like neural networks and point cloud mapping would now be possible where the Pi Zero simply could not keep up!

Another option would be to stick with the OpenCV style image processing, but to look at alternative ways of detecting track points. For example generating a map of the track that the code tries to follow and looks for well-known points on the track. Maybe using multiple track lanes to process from rather than relying on just one.

And finally...

All told using a Pi 3 instead of the Pi Zero opens up a lot of opportunity for code improvements for Formula Pi as well as general improvements for controlling robots for a Raspberry Pi. These changes are true for both a YetiBorg fitted with a Pi 3 as much as the MonsterBorg which is designed around one. This will mean that the Race Code in the MonsterBorg rounds will be as much of a performance improvement as the actual MonsterBorg itself is. I am looking forward to some impressive racing when these guys make it to our track :)