Analyzing results and generating reports

There are many ways that the results of tracking evaluation can be analyzed. In this tutorial we will cover ranking analysis as it is done for VOT2013 or VOT2014 challenge as well as simplified ranking that is recommended for inclusion in a paper. The results of both approaches are different so it is important to know which approach is used, so that the results are interpreted correctly.

Generating challenge report

Assuming you have your working directory set up and configured vot-workspace, you have to do the following steps:

  • download the reference trackers results for the benchmark,
  • extract the results to the vot-workspace/results directory in your workspace,
  • edit vot-workspace/run_analysis.m and modify so that it looks like this:

    [sequences, experiments] = workspace_load();
    
    trackers = order_trackers(tracker_list(‘trackers.txt’));
    
    context = create_report_context(‘challenge’);
    
    report_challenge(context, experiments, trackers, sequences);
    

    Replace the trackers.txt filename with the name of the file that contains a list of the trackers and is included in the results bundle.
  • execute the run_analysis script, and
  • open the newly generated report documents in the vot-workspace/reports/challenge.

Adding another tracker to the ranking analysis

To add results of a tracker to the challenge analysis, you have to change the vot-workspace/run_analysis.m script to include your tracker. To do this you have to change line trackers = order_trackers(tracker_load('vot2014.txt')); to trackers = order_trackers(tracker_load('vot2014.txt', 'yourtracker'));, where yourtracker denotes the identifier of the additional tracker. Make sure that you have actually performed the evaluation on the correct benchmark before that. Also note that due to complexity of ranking interaction, the overal results of the analysis may change when the set of trackers is changed so the results of the analysis may differ from the official challenge results.

Generating article report

Assuming you have your working directory set up and configured vot-workspace and that you have already performed the evaluation on your tracker, you have to do the following steps:

  • download the reference trackers results for the benchmark,
  • extract the results to the vot-workspace/results directory in your workspace,
  • edit vot-workspace/run_analysis.m and modify so that it looks like this:

    [sequences, experiments] = workspace_load();
    
    trackers = order_trackers(tracker_list(‘trackers.txt’, ‘yourtracker’));
    
    context = create_report_context(‘article’);
    
    report_article(context, experiments, trackers, sequences, ‘spotlight’, ‘yourtracker’);
    

    Replace the trackers.txt filename with the name of the file that contains a list of the trackers and is included in the results bundle. Replace yourtracker with the identifier of your tracker.
  • execute the run_analysis script, and
  • open the newly generated report documents in the vot-workspace/reports/article.