<<

NAME

run_bug_detection.pl -- bug detection analysis for generated test suites.

SYNOPSIS

  run_bug_detection.pl -p project_id -d suite_dir -o out_dir [-f include_file_pattern] [-v version_id] [-t tmp_dir] [-D]

OPTIONS

-p project_id

The id of the project for which the generated test suites are analyzed. See Project module for available project IDs.

-d suite_dir

The directory that contains the test suite archives. See Test Suites.

-o out_dir

The output directory for the results and log files.

-f include_file_pattern

The pattern of the file names of the test classes that should be included (optional). Per default all files (*.java) are included.

-v version_id

Only analyze test suites for this version id (optional). Per default all test suites for the given project id are analyzed.

-t tmp_dir

The temporary root directory to be used to check out program versions (optional). The default is /tmp.

-D

Debug: Enable verbose logging and do not delete the temporary check-out directory (optional).

DESCRIPTION

Runs the following worflow for each provided test suite (i.e., each test suite archive in suite_dir):

1) Verify that test suite compiles and runs on the version it was generated for (i.e., the buggy or fixed version).
2) Run test suite on the opposite version and determine whether it fails (a test suite generated for a buggy version is executed on the fixed version, and a test suite generated for a fixed version is executed on the buggy version).
3) Determine the number of triggering tests.

The results of the analysis are stored in the database table out_dir/TAB_BUG_DETECTION. The corresponding log files are stored in out_dir/TAB_BUG_DETECTION_log.

For each step the database table contains a column, indicating the result of the step or '-' if the step was not applicable. Note that the workflow is interrupted as soon as one of the steps fails and the script continues with the next test suite.

Logging

By default, the script logs all errors and warnings to run_bug_detection.pl.log in the temporary project root. Upon success, the log file of this script and the detailed bug detection results for each executed test suite are copied to: out_dir/TAB_BUG_DETECTION_log/project_id.

Test Suites

To be considered for the analysis, a test suite has to be provided as an archive in suite_dir. Format of the archive file name:

project_id-version_id-test_suite_src(\.test_id)?\.tar\.bz2

Note that test_id is optional, the default is 1.

Examples:

<<