DIY Self-Driving. Part 8 - Testing

Waiting for "Go"

Please Note:  Unlike most of my projects which are fully complete before they are published, this project is as much a build diary as anything else.  Whilst I am attempting to clearly document the processes, this is not a step-by-step guide, and later articles may well contradict earlier instructions.

If you decide you want to duplicate my build, please read through all the relevant articles before you start.

The car is finally built, and it is sitting out on the pathway ready to show the world what you can do.  So of course it's going to work right.  Isn't it?

I don't really know how to break this to you but chances are it won't work right the first time.  This is not because you are no good at making things or the software could never have worked, it is because there are so many variables which you could not have dealt with during the build.  Don't give up hope because the platform will work for you eventually.

Remember that global car companies are spending millions, if not more, each year on building their own autonomous platforms and we are doing this with little if any budget and ourselves as the only technical resource.  If this stuff was easy we'd be riding around driverless already.

OK, pep-talk over.  Let's get back onto testing.  You're going to need some additional components to move onto the next stage of the project.  Here's a list of things I found useful:

• A flat open area, preferably concrete or hotmix with a smooth surface.  Make sure there is a reasonable of run-off area because chances are the car is going to get away from you at some point and we don't want to risk safety of yourself, others or the car.
• A laptop with your source code and other related software (Arduino IDE, etc).
• A portable table or workbench.  This will save your back!
• A Wireless Access Point or router.  This also needs to provide DHCP service.
• Ethernet cables if required.
• Extension cables for power.
• Multi-outlet power board(s).
• Battery chargers.
• Hand tools, tape, blue-tak, etc
• One or more video cameras.
• A few roles of wide (50mm / 2") masking tape or similar.
• Somewhere to make notes.
• Drinking water.

Test your network before you go out to try the car.  Make sure that everything can connect while it is convenient to repair things.  Nobody wants to drive across town and then have to abandon the day because you don't have the WiFi password.

On the subject of early testing, unless you have a very strong disposition, I would recommend you limit the number of witnesses to early tests, because unless they have lived and breathed this project they will probably not share your optimism if things don't go exactly to plan.

Getting set up

First thing to do is unpack and test all your gear.  With that done, mark out your first test course.  Start simple.  A long straight with a gentle curve to the left or right.  Mark out the course as two lines spaced about 10% more than the width of the car.

The first test track

You won't get the lines spaced exactly, but do your best.  Also remember that the perspective will change dramatically from your normal standing height to the height above ground of the camera.

It is also very important at this stage of the software to make sure there is a good contrast between the "road" and the lines.  It is the contrast which the software uses to find the edge of the road.

Testing

Set up your camera(s).  I held one and attached another to the back of the car so it gave me an on-board view.  This view can be rather useful in understanding that is happening.

Make a note of all your settings: camera angle, battery voltage, temperature, brightness of the sun, and so on.

Launch your car and record what happens.  Chances are that it will head off in an unexpected direction.  Assuming the wheels were pointing straight, then the camera picked up on something which looked like a border and tried to position itself.  Disappointing that it didn't work as hoped, but you can draw comfort that it moved and steered.

Just to prove I understand your pain, here's a video of my early failures:

Often, these mis-steering events can be caused by the camera not picking up the lines, or if it aimed too far downwards, not being able to discern a direction.  Keep experimenting with camera angles while you repeat the test run.

I found my camera worked best at an angle approximately 30 degrees below horizontal.

Sometimes however, there is no apparent reason why the car won't steer correctly, and more in-depth techniques may be required.  The videostream.cpp code in the development branch now has the ability to write out a video file to local storage.  This file is an overlay showing the original image from the camera plus positioning markers determined by the software.

In one case I found there was insufficient contrast between the ground and the tape markers.  Running the car in the reverse direction improved the situation, and this helped me to move forward.  It also allows you to experiment with the filtering processes being used and fine-tune them to your needs.  For more information, please refer to Part 6 of this series.

Still other things can catch you out.  The pebble surface on my test area bounced the car around a lot more than I expected, which made fixing on an edge difficult for the software.  I worked around this by increasing the frame rate to hopefully freeze the image better.

Also on this point, make sure your camera mount is secure and that the camera has no vibrations of its own.  Because my car has no spring or shock absorption, it is hard enough to get a stable image without the camera waving about as well.

In the end, I realised I could not get a stable result with the physically small camera I was using, even though it was the first to get me through the test path.  I believe the camera's CPU is just too slow compared to some other devices.  I switched back to a Logitech HD camera and the overall performance improved considerably.  But securely mounting this device is still a problem.

Persevere.  It can be done!  The next video shows my first successful run.  OK, it strayed off the path, but the car made the correct turn, and it gave me new things to analyse:

In this case, the steering response is far too slow.  This is in part because the steering increments need to be a greater value, and also because I had to slow the car down in testing.

When it does work, allow yourself a well-deserved celebration.  Why not take the car out and show off?

This is one way to get noticed...

Further Reading
Part 1 - Introduction
Part 2 - Preparing the car
Part 3 - Wiring Harnesses
Part 4 - Steering
Part 5 - Sensors
Part 6 - Software in depth
Part 7 - Final assembly and testing