Run Results

Listing Run Results

To view the results, click on View Results from the menu on the left of the page. A page such as the following will be shown:

As soon as a run is started, it will show up on the results list with a status of STARTED. The current run is always showed on top of the list, if any, followed by the finished runs. The runs are ordered by run id and/or run date, in descending order. This means the most recent run is shown first and the oldest run is shown last.

The columns shown in the result list are the description, whether the run passed all criteria or not, the scale of the run (the user setting of the load) and the metric (final run result), the run status, and the date and time the run was last updated. This means the start time for a current run and the end time for a run that has already finished. Clicking on PASS or FAIL takes you to the summary report. Clicking on the run status takes you to the run logs. Either way, the result navigator menu will show up to allow you to drill in to the run results.

Result Navigation

The result navigator menu, shown on top of individual run results, allows you to drill into the details of the run. The summary report and detail report are not available for runs in progress and are the links are disabled. But the run configuration, the logs, and some statistics will always be available.

The Summary Report

The summary report is the place to get the final results of a run. It will clearly report the obtained metrics and a lot of the run statistics. If there are failures, the summary report also identifies the failures that have happened. Usually, the summary report shows you the transaction/operation counts, sucesses, failures, mixes, response time, and cycle time statistics.

The Detail Report

The detail report shows some detail statistics that cannot be represented by simple aggregated numbers as shown in the summary report. Usually it shows graphs of throughput and response times over time. It usually also shows distribution histograms of cycle times and response times which allows you to understand the characteristics much better than reading single number results in the summary report.

The run configuration shows the configuration files as it was submitted. While the submission is done using an easy-to-use form, the configuration file is shown as non-formatted XML allowing you to investigate the detail of the submission with ease.

Run Logs

The run logs show you what has happened or is happening in the run. The log server accumulates the logs from all the systems partaking in the run and presents a unified view of the logs. Such a unified view of the logs is also extremely interesting as it interposes the events that happen on each system allowing easier pinpointing of problems as they occur.

Statistics

The statistics screen shows both system configuration and statistics as soon as they occur. The Faban harness tools subsystem collects statistics on all the systems partaking in the run as specified in the run configuration and transfers them to the master shown through the statistics screen.

The first item shown for all systems is always the system information and configuration, followed by the statistics produced by tools, ordered aphabetically by tool name. Most statistics provide text results, some provide html results. For certain statistics, the Faban harness also postprocesses and graphs the results so they can be easily viewed and analyzed. The Faban harness calls Xanadu to do such postprocessing and graphing. For such statistics, both text and html results will be available. The html page also provides an entry point to use Xanadu for further analysis of the graphs. To do so, just click on one of the graphs that you wish to analyze in detail.

Removing Run Results

In certain cases, you'll want to clear up run results that are no longer needed. Not only do these results clutter up your result list. In some cases they also use up a lot of storage space. Removing run results is one of the few functions that are not provided through a GUI. One of the reasons is because of the lack of adequate security in the Faban harness GUI. Adding security will make installation and getting started with Faban much harder.

Still, it is easy to remove a Faban result. Login to the master system and just simply remove the directory containing the result under FABAN_HOME/output. For example, the following command shows an example removing the web1.1A run result:

        $ rm -rf $FABAN_HOME/output/web1.1A