OpenTracker Compilation and Process Interpretation

After OpenCV is installed and compiled, you can run several open source projects to see how it works, and then learn the essence from the theory. Learning should first learn to apply, explore theory in application, improve innovation, rather than all tied in the deduction of mathematical formulas.
The introduction of OpenTracker project is Real-time C++ ECO tracker etc. speed-up by SSE/NEON, support Linux, Mac, Jetson TX1/2, raspberry pi. It changes ECO from MATLAB version to C++ version, and can run successfully on various embedded platforms. At the same time, it compares the effect of common target tracking algorithms, which is the first choice for beginners. Because I finally need to use practical applications to run in embedded systems like TX2, which requires C++ code and is a lightweight tracker. Project github address

1. Download the source code and run ECO quickly

git clone https://github.com/rockkingjy/OpenTracker
cd OpenTracker/eco
make -j`nproc`
sudo make install
./runecotracker.bin

After running, you can see the effect. The program running here is compiled by eco/runecotracker.cc.

2. ECO C++ Source Code Running Flow

Open the file eco/runecotracker.cc, and we can see the contents of the main function.

    // Database settings
	//There are so many kinds of data sets that you can add or use external cameras yourself.
    string databaseTypes[5] = {"Demo","VOT-2017", "TB-2015", "TLP", "UAV123"};
	//Which data set do we choose?
    string databaseType = databaseTypes[0];//4];
    // Read from the images ====================================================

A total of four datasets are provided to test performance indicators, Demo by default, and only a few images are provided to run.

if (databaseType == "Demo")
    {
        path = "../sequences/Crossing"; //Data Set Path
        // some of the dataset has '\t' as the delimiter, change it to ','.
		//fstream file operation function reads x y w h
        fstream gt(path + "/groundtruth_rect.txt");
        string tmp;
        size_t index = 1;
        while (gt >> tmp)  //Standardized txt file for later reading
        {
            if(tmp.find(',')<10)
            {
                break;
            }
            if (index%4 == 0)
            {
            }
            else
            {
                gt << ",";
            }
            index++;
        }
        gt.close();
        // Read the groundtruth bbox
        groundtruth = new ifstream(path + "/groundtruth_rect.txt");
        f = 1;
        getline(*groundtruth, s, ',');  //Read data line by line
        x = atof(s.c_str());  //Converting strings to data
        getline(*groundtruth, s, ',');
        y = atof(s.c_str());
        getline(*groundtruth, s, ',');
        w = atof(s.c_str());
        getline(*groundtruth, s);
        h = atof(s.c_str());
        cout << f << " " << x << " " << y << " " << w << " " << h << " " << endl;
        // Read images in a folder
		//std::setw: How many characters do you need to fill in? The default filling character is''space.
		//std::setfill: What characters will std::setw fill in?
        osfile << path << "/img/" << setw(4) << setfill('0') << f << ".jpg";
        cout << osfile.str() << endl;
    }

Specific data set paths can also be set by themselves, but since indicators need to be calculated later, standard data sets are needed for testing. In general, the tracker chooses the marked object in the first frame as the target. Here, the first frame data in Groundtruth file of standard data set is read first to determine the object to be tracked.

//Prepare the frame of the manual standard for drawing
    Rect2f bboxGroundtruth(x, y, w, h);
    //Read img from the specified path first
    cv::Mat frame = cv::imread(osfile.str().c_str(), CV_LOAD_IMAGE_UNCHANGED);
    cv::Mat frameDraw;
	//copy img
    frame.copyTo(frameDraw); 
    if (!frame.data)
    {
        cout << "Could not open or find the image" << std::endl;
        return -1;
    }
    // Draw gt;
    if (databaseType == "Demo")
    {
        ii(frameDraw, bboxGroundtruth, Scalar(0, 0, 0), 2, 1);
    }

Here is to read the coordinates in the standard data set and draw a box for intuitive comparison.

ecotracker.init(frame, ecobbox, parameters);
    float fpsecoini = getTickFrequency() / ((double)getTickCount() - timereco);

    while (frame.data)
    {
        frame.copyTo(frameDraw); // only copy can do the real copy, just equal not.
        timereco = (double)getTickCount();
        bool okeco = ecotracker.update(frame, ecobbox);  //To update
        float fpseco = getTickFrequency() / ((double)getTickCount() - timereco);
        if (okeco)
        {
            rectangle(frameDraw, ecobbox, Scalar(255, 0, 255), 2, 1); //blue
        }
        else
        {
            putText(frameDraw, "ECO tracking failure detected", cv::Point(100, 80), FONT_HERSHEY_SIMPLEX, 0.75, Scalar(255, 0, 255), 2);
            //waitKey(0);
        }

Here we begin to initialize the ECO, get the data ecobox output by the tracker, and draw a box in blue.

if (databaseType == "Demo") //Read data from standard data sets
        {
            getline(*groundtruth, s, ',');
            x = atof(s.c_str());
            getline(*groundtruth, s, ',');
            y = atof(s.c_str());
            getline(*groundtruth, s, ',');
            w = atof(s.c_str());
            getline(*groundtruth, s);
            h = atof(s.c_str());
            //cout << f << " " << x << " " << y << " " << w << " " << h << " " << isLost << endl;
            // Read images in a folder
            osfile << path << "/img/" << setw(4) << setfill('0') << f << ".jpg";
            //cout << osfile.str() << endl;
        }

//Artificially labeled rectangle
        bboxGroundtruth.x = x;
        bboxGroundtruth.y = y;
        bboxGroundtruth.width = w;
        bboxGroundtruth.height = h;
        frame = cv::imread(osfile.str().c_str(), CV_LOAD_IMAGE_UNCHANGED);
        if(!frame.data)
        {
            break;
        }
        // Calculate the metrics;
		//Calculating the Errors of Manual Annotation and ECO Algorithms
        float centererror = metrics.center_error(ecobbox, bboxGroundtruth);//Central error
        float iou = metrics.iou(ecobbox, bboxGroundtruth);  //IOU cross-parallel ratio
        CenterError.push_back(centererror); //Insert data at the end of container
        Iou.push_back(iou);
        FpsEco.push_back(fpseco);

Here is to read the standard data labeled manually first, and then calculate the output of the tracking algorithm with itself to get the performance evaluation index.

3. Precision plot of target tracking evaluation index

Precision plot. One widely used evaluation metric on tracking precision is the center location error, which is defined as the average Euclidean distance between the center locations of the tracked targets and the manually labeled ground truths. Then the average center location error over all the fr As the representative precision score for each tracker, we use the score for the threshold = 20 pixels. In Mr. Yi Wu's article Online Object Tracking: A Benchmark, it is mentioned that tracking can be measured by precision graphs. The overall performance of the tracker is the Euclidean geometric distance between the center point of the bounding box estimated by the tracking algorithm and the center point of the ground-truth target, which is less than the percentage of video frames with a given threshold (usually 20 pixels).

float Metrics::center_error(const cv::Rect2f bbox, const cv::Rect2f bboxGroundtruth)
{
    float cx = bbox.x + bbox.width / 2.0f;  //Get the coordinates of the center point
    float cy = bbox.y + bbox.height / 2.0f;
    float cx_gt = bboxGroundtruth.x + bboxGroundtruth.width / 2.0f;
    float cy_gt = bboxGroundtruth.y + bboxGroundtruth.height / 2.0f;
    float result = std::sqrt(std::pow((cx - cx_gt), 2) +
                             std::pow((cy - cy_gt), 2));  //Center coordinate difference square and root number
    return result;
}

This is a function in the eco / metrics.cc file of the ECO project to calculate the central error.

if(centererror <= 20) //20 is the threshold
        {
            AvgPrecision++;  //The center error is less than the threshold (usually set to 20 pixels), the accuracy is increased by 1, and the final division by the total number of frames is the accuracy.
        }

 AvgPrecision /= (float)(f - 2);

If the center error is less than the threshold (20 pixels), the accuracy can be increased by one, and then divided by the number of frames. For example, there are 102 frames in the data set, and 80 frames between the center point of the tracking algorithm and the center point of the standard data set are less than 20 pixels. When the threshold is 20 pixels, the accuracy is 0.8.

4. Target tracking evaluation index Success plot

In Professor Yi Wu's article Online Object Tracking: A Benchmark, accuracy is defined by using IOU (Intersection-over-Union). When the overlapping area is larger than the set threshold (usually 0.5), the recognition of this frame is considered successful.

float Metrics::iou(const cv::Rect2f bbox, const cv::Rect2f bboxGroundtruth)
{
    cv::Rect2f inter = Metrics::intersection(bbox, bboxGroundtruth);
    float area_bbox = bbox.area();
    float area_bbox_gt = bboxGroundtruth.area();
    float area_intersection = inter.area();  //The area of intersection
    float iou = area_bbox + area_bbox_gt - area_intersection; //Get the Union area of two box es
    iou = area_intersection / (iou + 1e-12);  //Intersection/union, in order to prevent denominator from being zero, add a small value
    return iou;
}
//Find the minimum right coordinate minus the maximum left coordinate of the intersection of two rectangles
cv::Rect2f Metrics::intersection(const cv::Rect2f bbox,
                                 const cv::Rect2f bboxGroundtruth)
{
    float x1, y1, x2, y2, w, h;
    x1 = std::max(bbox.x, bboxGroundtruth.x);  //Finding the Maximum Coordinate of the Upper Left Angle
    y1 = std::max(bbox.y, bboxGroundtruth.y);
    x2 = std::min(bbox.x + bbox.width, bboxGroundtruth.x + bboxGroundtruth.width); //Finding the Minimum Coordinates of the Lower Right Angle
    y2 = std::min(bbox.y + bbox.height, bboxGroundtruth.y + bboxGroundtruth.height);
    w = std::max(0.0f, x2 - x1);
    h = std::max(0.0f, y2 - y1);

    cv::Rect2f result(x1, y1, w, h);
    return result;
}

This is a function in the eco / metrics.cc file of the ECO project, which is used to calculate the union of two box es and the index IOU.

// Calculate the metrics;
		//Calculating the Errors of Manual Annotation and ECO Algorithms
        float centererror = metrics.center_error(ecobbox, bboxGroundtruth);//Central error
        float iou = metrics.iou(ecobbox, bboxGroundtruth);  //IOU cross-parallel ratio
        CenterError.push_back(centererror); //Insert data at the end of container
        Iou.push_back(iou);
        FpsEco.push_back(fpseco);

        cout << "iou:" << iou << std::endl;

        if(centererror <= 20) //20 is the threshold
        {
            AvgPrecision++;  //The center error is less than the threshold (usually set to 20 pixels), the accuracy is increased by 1, and the final division by the total number of frames is the accuracy.
        }
        if(iou >= 0.5) //0.5 as threshold
        {
            SuccessRate++; //When the IOU is larger than the threshold, the success rate is divided by the total number of frames.
        }
       

 SuccessRate /= (float)(f - 2);

Once IOU data is obtained, it is stored in a container and then considered successful as long as the data in the container is larger than the threshold value (0.5).

5. Interpretation of Target Tracking Principle

To be continued