Real-Time Object Tracker In C++ / 四六文摘

Download demo - 127 KB Download source - 84.3 KB

Contents

Introduction

Background

Using the code

The library

Results

Points of interest

Introduction

This article is about tracking moving or static objects with a conventional web cam at real-time speed. A simple way of tracking is to compare a predefined background image with the same background frame when objects start to appear. This scenario is applicable in static surroundings where you can learn the background image without any foreground articles you’d expect to track (e.g., indoors, landings, offices, shops, warehouses etc...). In the case that an article moves and remains static for a prolonged period of time in the scene, you may consider it as becoming the part of the scene now, and superimpose its image to the background to avoid further detections. The same is applicable if some background item is removed from the scene. The drawbacks of this approach include illumination or cam position change. In that case, you will have to estimate the new background again. Another solution is to use edge operators to avoid illumination specific changes. But you will need additional code to devise if you’d like to identify the object’s body and not just its outline.

The conventional web cam 640x480 resolution is far redundant. You may downscale it about three times to remove the noise usually present and significantly boost the speed of processing without essential loss of tracking abilities, unless you expect to track very minute objects. The downscaling step allows to achieve great processing speeds in object tracking. It runs about 100fps on a 2Ghz single core when no objects are present, and from 30 to 90fps when there are. The bigger the object, the more time is required to estimate all pixels belonging to the blob.

Background

ImageBlobs

The article and code are based on my previous submissions, except the ImageBlobs class. You should have a look at the following articles if you have questions about the GUI interface or particular code fragments:

2D Fast Wavelet Transform Library for Image Processing Fast Dyadic Image Scaling with Haar Transform 2D Vector Class Wrapper SSE Optimized for Math Operations Video Preview and Frames Capture to Memory with SampleGrabber in Buffered Mode Face Detection C++ Library with Skin and Motion Analysis

The GUI is targeted at 640x480 web cams, and if you want to use it for different resolutions, have a look at the Face Detection article to add the required changes. You can not expect me to provide you a complete application for every custom video device.

Using the code

start tracking

Setup the camera and frames capture rate as described in Video Preview and Frames Capture to Memory with SampleGrabber in Buffered Mode and start video capture. Click the background radio box to start the background estimation process first. All the captured frames will be added together, and the mean background frame will be estimated and saved to a JPEG file, background.jpg, once you have clicked the start tracking radio box to start object tracking. I suggest you point your camera to some place where you can expect moving objects. Estimate the background for about several seconds to cope with camera noise or some casual moving articles appearing in the scene for short periods of time. The mean background estimate will effectively remove them. Now, introduce the objects to the scene you want to track, or wait for them to appear there if they are alive.

The library

The mean estimate of the background frame is advisable as it allows to filter out any noise or tiny movements.

cpp

Shrink ▲ Copy Code

The new classes in the library are:

MotionDetector

ImageBlobs

MotionDetector

ImageResize

MotionDetector

ImageResize

MotionDetector

ImageResize

The changes in the MotionDetector allow to set the background frame to which the comparison will be done with every new image frame, rather than taking the difference between consecutive frames as in the Face Detection article. The background frame and the new image frame are presented with ImageResize objects.

MotionDetector

First, you need to initialize the MotionDetector object with the video frame width, height, and downscaling factor:

void MotionDetector::init(unsigned int image_width, unsigned int image_height, float zoom);

zoom

To downscale the image three times, use zoom = 0.125f.

set_background()

Invoke the set_background() function once you finish the background estimation process:

cpp

Shrink ▲ Copy Code

detect()

Now, you may call the detect() function to estimate the pixels belonging to the foreground objects:

cpp

Shrink ▲ Copy Code

You may use either RGB values comparison or gray image data. The latter, I found not robust for similar looking colors when converted to gray values. The dilation and erosion operators allow to fill any gaps in the object’s blobs that might appear due to similar pixel colors of the background and the object, and remove the noise of some tiny movements or very thin objects.

ImageBlobs

The returned vector is used for blobs extraction with the ImageBlobs object. You will need the following functions to use the class:

void ImageBlobs::init(unsigned int width, unsigned int height);

int ImageBlobs::find_blobs(const vec2Dc& image, unsigned int min_elements_per_blob = 0);

void ImageBlobs::find_bounding_boxes();

void ImageBlobs::delete_blobs();

init(zoom * image_width, zoom * image_height)

find_blobs()

image

MotionDetector::detect()

image

init(zoom * image_width, zoom * image_height)

find_blobs()

image

MotionDetector::detect()

image

init(zoom * image_width, zoom * image_height)

find_blobs()

image

MotionDetector::detect()

image

First, you need to initialize the object to the downscaled image width and height (e.g., init(zoom * image_width, zoom * image_height)). Then, you may proceed estimating blobs with find_blobs() on the image returned from the MotionDetector::detect() function call. The function searches for non-zero elements in the image vector adjoining horizontally or vertically, forming a blob. Then, it marks every element with the current found blob number.

image

For example, the 10x10 vector used as image vector:

Shrink ▲ Copy Code

find_blobs()

image

const vec2Dc* ImageBlobs::get_image() const

find_blobs()

image

const vec2Dc* ImageBlobs::get_image() const

find_blobs()

image

const vec2Dc* ImageBlobs::get_image() const

find_blobs() will estimate three blobs from that image. You may get the vector containing the found blobs with the const vec2Dc* ImageBlobs::get_image() const function.

Shrink ▲ Copy Code

min_elements_per_blob

With min_elements_per_blob, you may discard small blobs from being detected (e.g., min_elements_per_blob = 5 will leave the third blob undetected).

To access the elements of the found blobs, you may use the following functions:

inline unsigned int ImageBlobs::get_blobs_number() const;

inline const struct Blob* ImageBlobs::get_blob(unsigned int i) const;

Blob

The blob is returned in the Blob structure:

cpp

Shrink ▲ Copy Code

elements_number

elements

Element

elements_number

elements

Element

elements_number

elements

Element

where elements_number is the number of elements in the blob contained in the elements array of Element structures.

cpp

Shrink ▲ Copy Code

neighbs

coord

image

neighbs

coord

image

neighbs

coord

image

The neighbs contain the directly adjoining neighboring elements, and coord is the element coordinate in the image vector.

cpp

Shrink ▲ Copy Code

find_blobs()

find_bounding_boxes()

Blobs::bounding_box

RECT

find_blobs()

ImageBlobs

delete_blobs()

find_blobs()

find_bounding_boxes()

Blobs::bounding_box

RECT

find_blobs()

ImageBlobs

delete_blobs()

find_blobs()

find_bounding_boxes()

Blobs::bounding_box

RECT

find_blobs()

ImageBlobs

delete_blobs()

After you call find_blobs(), you may optionally invoke find_bounding_boxes() to estimate the bounding boxes of all blobs found to Blobs::bounding_box window’s RECT structure. Before the next call to find_blobs(), you need to delete the found blobs from the ImageBlobs object with delete_blobs().

find_blobs()

The find_blobs() function is shown below:

cpp

Shrink ▲ Copy Code

add_*_neighbour()

Element

blob

Blob

add_*_neighbour()

Element

blob

Blob

add_*_neighbour()

Element

blob

Blob

The add_*_neighbour() functions check for the directly adjoining image element from above, below, left, or right from the i^th Element in the current blob and adds it to the Blob elements array:

cpp

Shrink ▲ Copy Code

has_neighbour()

is_element_present

blob

has_neighbour()

is_element_present

blob

has_neighbour()

is_element_present

blob

has_neighbour() and is_element_present determine if the new element is already present in the blob:

cpp

Shrink ▲ Copy Code

Results

Here, I have selected a static background and performed some object tracking experiments with the objects at hand. The background I used is depicted below. It was averaged over a period of several seconds.

The next two objects (an LCD-TFT screen cleaner and mobile phone) are tracked at 66.67fps.

The mobile phone power adaptor is tracked at 71.43fps. You see the cord is not detected due to erosion and dilation operators, as it is quite thin.

Now, some pens and keys: 90.91fps.

A more complex scenario: three different objects detected at 29.41fps.

Jimi Hendrix (the Are You Experienced album, very good sound) and 9v batteries at 55.56fps.

The other background setup:

Several 9v batteries and a roll on the carpet detected at 90.91fps. You see one battery is left undetected due to the minimum number of elements limit.

Power adaptor, LCD-TFT cleaner, and 9v batteries detected at 90.91fps. The tiny ones are left undetected again, but this time due to erosion and dilation operators.

Next, some candle and mobiles at 83.33fps.

Now, as you can see, the bigger the size of the object, the more time is required to estimate the elements of the blobs. Also, it does not handle the shadows cast by objects on the white back wall and on the table. However, more pronounced shadows will be tracked as pertaining to the object.

Points of interest

You may extend the algorithm to monitor an object’s bounding box positions. In case they will be static for a specified amount of time, you may add that region to the background scene image. Thus, the object becomes part of the scene (e.g., some object moved to the scene and remaining static).

Real-Time Object Tracker In C++

相关推荐