# Finding the track lanes, Part II

As a quick recap from last time, we started with this image:

and using some processing we got to this:

We will be continuing the code from the previous part, which you can find here:

Finding the track lanes, Part I

So how do we decide where to place our points?

What we want to do is 'scan' along the image in many places and find the outsides of the three colours.

Before we can do that they need to be simplified to True or False arrays.

We can do this very simply by using the inbuilt functionality of numpy:

red = red > 0 green = green > 0 blue = blue > 0 walls = walls > 0

The only trouble here is that `cv2.imwrite`

will not handle this data type.

We can make a function to help us by making the values between 0 and 255 instead:

def WriteMask(name, mask): image = mask * 255 cv2.imwrite(name, image)

We can then write our masks like so:

WriteMask('blue-mask.jpg', blue) WriteMask('green-mask.jpg', green) WriteMask('red-mask.jpg', red) WriteMask('walls-mask.jpg', walls)

Which gives this for the colours:

Note how the image is simply black or white now, no grey like last time.

The next thing to do is decide where in the image we will take slices.

This code will generate 100 slices along the original image:

grid = 100 scanLines = [] for i in range(grid): # Work out the position in the original image position = (i / float(grid)) * height position = int(position) # Work out the cropped position if position < cropTop: # Above our cropped region pass elif position >= cropBottom: # Below our cropped region pass else: # In the cropped region, correct and add to our list croppedY = int(position - cropTop) scanLines.append(croppedY) # Show the list of positions print scanLines

There were 70 values within the crop we did, this is because we cropped away 30% of the image :)

We can better illustrate where these lines are by drawing them.

We do this by making a brand new image the same size as the cropped one.

Then we draw our lines on top of it:

# Make a black image the same size as our cropped image scanLineImage = numpy.zeros_like(cropped) colourWhite = (255, 255, 255) # Loop over each line for y in scanLines: cv2.line(scanLineImage, (0,y), (width-1,y), colourWhite, 1) cv2.imwrite('scanlines.jpg', scanLineImage)

Now we have some lines to scan, how do we scan a line?

We get numpy to help with that as well.

We make ourselves a little function which will find the edges in a mask image:

def SweepLine(mask, y): found = [] # Grab the line of interest line = mask[y, :] # Get numpy to give us a list of the positions where the line changes in value changed = numpy.where(line[:-1] != line[1:])[0] # Remove changes too close to the edge of the image for i in changed: if i < 2: pass elif i > (width - 3): pass else: found.append(i) # Return the found values return found

A quick test with a position of 135 in the green mask for example:

print SweepLine(green, 135)

shows `[476, 1075, 1640]`

in our example.

That would mean the green value changes three times on that line.

We can now process each line, but we still need to match the lines together.

We do this by looking for similar positions:

# The values of try1, try2, try3 are used to attempt a match with target # Any matches are added to existing lists matched1, matched2, matched3 # Any values which cannot be matched are added to the existing list unmatched def FindMatches(y, target, try1, try2, try3, matched1, matched2, matched3, unmatched): maxSeperation = int(width * 0.05) # Loop over all the values in target: while len(target) > 0: # Remove the next value from the list of targets xt = target.pop() matched = False # See if try1 can match it if try1: for x1 in try1: if abs(x1 - xt) < maxSeperation: # Matched, work out the point and add it matched = True try1.remove(x1) x = (xt + x1) / 2 matched1.append((x, y)) break if matched: continue # See if try2 can match it if try2: for x2 in try2: if abs(x2 - xt) < maxSeperation: # Matched, work out the point and add it matched = True try2.remove(x2) x = (xt + x2) / 2 matched2.append((x, y)) break if matched: continue # See if try3 can match it if try3: for x3 in try3: if abs(x3 - xt) < maxSeperation: # Matched, work out the point and add it matched = True try3.remove(x3) x = (xt + x3) / 2 matched3.append((x, y)) break if matched: continue # No matches unmatched.append((xt, y))

The function is fairly long but quite simple.

It takes a list of points to try and match and one or more lists to try and match it with.

Each point is then added to one of the matched lists, or is added to the unmatched list.

So we now have:

- Four image masks we can match
- A set of lines to scan over
- A function to scan for changes
- A function to find matches between two or more lists

We are now ready to find all of our points :)

What we do is go through each line in turn and:

- Scan each mask for the changes (if any) on that line
- Attempt to match each line with any it might be next to
- Keep all the matches in the same list for all of the lines

We do that like this:

# Make our matched lists matchRG = [] matchRB = [] matchRW = [] matchGB = [] matchGW = [] unmatched = [] # Loop over each line for y in scanLines: # Scan the masks edgeR = SweepLine(red, y) edgeG = SweepLine(green, y) edgeB = SweepLine(blue, y) edgeW = SweepLine(walls, y) # Do the matching FindMatches(y, edgeR, edgeG, edgeB, edgeW, matchRG, matchRB, matchRW, unmatched) FindMatches(y, edgeG, edgeB, edgeW, None, matchGB, matchGW, None, unmatched) # Add any left over points to the unmatched list others = edgeB[:] others.extend(edgeW) for x in others: unmatched.append((x, y))

It would help a lot if we could see our points at this stage.

We can make a quick function to draw a cross on the image like this:

def DrawCross(image, (x, y), (r, g, b)): crossSize = 5 width = image.shape[1] height = image.shape[0] # Build the list of points to change points = [] for i in range(-crossSize, crossSize + 1): points.append((x + i, y)) points.append((x, y + i)) # Change the points on the image for point in points: x = point[0] y = point[1] if (x >= 0) and (y >= 0) and (x < width) and (y < height): image.itemset((y, x, 0), b) image.itemset((y, x, 1), g) image.itemset((y, x, 2), r)

Now we have everything we need to plot all of our points onto a blank image:

pointImage = numpy.zeros_like(cropped) for point in matchRG: DrawCross(pointImage, point, (255, 255, 0)) for point in matchRB: DrawCross(pointImage, point, (255, 0, 255)) for point in matchRW: DrawCross(pointImage, point, (255, 0, 0)) for point in matchGB: DrawCross(pointImage, point, (0, 255, 255)) for point in matchGW: DrawCross(pointImage, point, (0, 255, 0)) for point in unmatched: DrawCross(pointImage, point, (127, 127, 127)) cv2.imwrite('points.jpg', pointImage)

We can do the same thing using the cropped image as a base instead:

pointImage = cropped[:,:,:] for point in matchRG: DrawCross(pointImage, point, (255, 255, 0)) for point in matchRB: DrawCross(pointImage, point, (255, 0, 255)) for point in matchRW: DrawCross(pointImage, point, (255, 0, 0)) for point in matchGB: DrawCross(pointImage, point, (0, 255, 255)) for point in matchGW: DrawCross(pointImage, point, (0, 255, 0)) for point in unmatched: DrawCross(pointImage, point, (127, 127, 127)) cv2.imwrite('points2.jpg', pointImage)

In the standard Race Code processing we also check if the changes are True to False or False to True.

This allows the code to tell the ordering of the changes when it makes a difference.

For example in the images above the green | blue line is labelled the same as the blue | green line.

While the changes to see if the edges are changing to on or off are simple, they do make the code a fair bit longer.

In fact our standard processing has a total of:

- Seven valid matching pairs
- Two invalid matches: wrong-way and unknown
- Two returned lists from
`SweepLine`

: rising and falling - Four calls to
`FindMatches`

to determine all of the matches

All of these points can be seen from `Race.py`

by calling `TrackLines()`

.

These are the points which we will use to determine where the track is and where it will be heading.

Stay tuned for Part III where we work this all out from the points!

## Add new comment