Difference between revisions of "RTLLogFileAnalysis"

From Apache OpenOffice Wiki
Jump to: navigation, search
(Implementation)
(Implementation)
Line 23: Line 23:
 
In order to use RTL_LOGFILE as described above, two tasks need to be completed:
 
In order to use RTL_LOGFILE as described above, two tasks need to be completed:
  
* Additional RTL_LOGFILE macros have to be inserted into the OpenOffice.org source code that provide the information which documents have been loaded or saved.
+
* Additional macros have to be inserted into the OpenOffice.org source code that provide the information which documents have been loaded or saved. This is done as [[http://qa.openoffice.org/issues/show_bug.cgi?id=100301 RTL_LOGFILE task 100301] which is resolved in CWS mib10.
 
* A configurable analysis tool is required that allows comparisons between load/save data for individual documents, and between load/save data for individual OpenOffice.org instances. This tools is described below.
 
* A configurable analysis tool is required that allows comparisons between load/save data for individual documents, and between load/save data for individual OpenOffice.org instances. This tools is described below.
  

Revision as of 09:59, 18 March 2009

Performance 170.png
Performance Project

performance.openoffice.org

Quick Navigation

Team

Communication

Activities

About this template


RTL_LOGFILE Analysis

Introduction

RTL_LOGFILE is a developer feature of OpenOffice.org that can be used to get wall clock based time stamps for particular code points. It can be used for instance in automated performance tests. This section describes its use for profiling load and save operations of ODF documents.

Use Cases

The use of RTL_LOGFILE is advantageous, compared to for instance the use of a profiler, if the performance data is required for a large amount of load or save operations, or if the performance of two OpenOffice.org builds shall be compared with as little effort as possible. The data that can be collected by RTL_LOGFILE is neither precise nor detailed enough to actually profile for instance the loading or saving a particular document, but it is still is detailed and precise enough for a rough analysis, for regression tests or performance metrics, or to figure out which effect a particular performance improvement has for a large set of data.

Possible use cases are:

  • The comparison of load save times for a large set of document between a master build and a CWS.
  • The comparison of load save times for a large set of document between a master build and a developer build that implements a particular improvement.
  • To provide metrics that show performance improvements that have been achieved.
  • To find documents whose load or save time is significant above or below the average, and which therefore are either
    • worth a deeper investigation (with other tools), or
    • appear to be non appropriate as test documents for a performance improvement that is assumed to effect all documents.

Implementation

In order to use RTL_LOGFILE as described above, two tasks need to be completed:

  • Additional macros have to be inserted into the OpenOffice.org source code that provide the information which documents have been loaded or saved. This is done as [RTL_LOGFILE task 100301 which is resolved in CWS mib10.
  • A configurable analysis tool is required that allows comparisons between load/save data for individual documents, and between load/save data for individual OpenOffice.org instances. This tools is described below.

RTL_LOGFILE Analysis Tool (RTLLog2ODF)

Introduction

The new analysis tool is similar to the existing RTL_LOGFILE analysis perl scripts, from which the parser logic has been adopted. It takes one or more RTL log files as input, and creates an ODF spreadsheet. How the performance data is presented in the spreadsheet can be controlled by a set of XPath expressions.

The creation of the spreadsheet takes places in two steps. In a first step, the text based RTL log file is converted into an intermediate XML file. In a second step, the XML tree is converted into an ODF spreadsheet. Two kinds of conversions are available: In the detailed conversion, a set of sheets is created, there each sheet contains the data for a log file, a thread, a particular load save operation, or another sub tree of the log file. In the summary conversion, each sheet additionally contains column group where each group contains the data for load/save operation, ot another sub tree of the log file.

Source Code

The source code for RTLLog2ODF is available in the performance projects SVN repository.

Command Line

The RTL_LOGFILE analysis tool is invoked by one of these commands:

java -jar rtllog2odf.jar  -d <rtl log file> [<rtl log file> ...] <xml file>
java -jar rtllog2odf.jar  [-t <xpath>] [-T <xpath>] [-i <xpath>] <rtl log file> [<rtl log file> ...] <ods file>
java -jar rtllog2odf.jar  -s [-t <xpath>] [-T <xpath>] -g <xpath> [-G <xpath>] [-i <xpath>] <rtl log file> [<rtl log file> ...] <ods file>
java -jar rtllog2odf.jar  -p <profile> <rtl log file> [<rtl log file> ...] <ods file>

If -d is not provided, the given RTL log files <rtl log file> are transformed into a spreadsheet document <ods file>. A sheet is created for each node in the intermediate XML file] that is matched by the [http://www.w3.org/TR/xpath XPath expression specified with the option -t. A title for the spreadsheet can be specified by the option -T. The XPath expression specified by this option is evaluated with the nodes selected by -t as context nodes, and the resulting string value is used as title.

If -s is provided, the results on the sheet are grouped into columns. For each sub tree in the intermediate XML file, a set of columns is created that contains the entries for that group. The groups are selected by the -g option, which takes an XPath expression that is evaluated with the nodes specified by -t as context nodes. The number of entries must be the same for each group. The same applies to their function names and messages. A title for each group can be specified by the -G option, which takes an XPath expression that is evaluated with the nodes selected by the -g option as context nodes.

If -s is provided with a -t options that results in two sheets, then an additional sheet is generated that list the time differences between the sheets for the two log files.

The -i option allows to specify an include filter. It takes a boolean XPath expression, which is evaluated for each descendant node of the nodes selected by the -t and -g options. Only nodes where the result of evaluating the expression is true are included.

If -d is provided, the given RTL log files are transformed into an XML file only.

If -p is specified a profile is selected. A profile is a set of XPath expressions for the above options.

XPath Expressions

The XPath expressions that control the generation of the spreadsheet support the following extension functions:

Name Descriptions
rtllog:filename(path) returns the filename part of path

Profiles

The following profiles are predefined. To get a list of all profiles that are available RTLLog2XML may be called without arguments.

threads

Displays the data of each thread on a separate sheet. The corresponding options command line options are:

-t "//thread" -T "concat('Thread ',@thread-id)"

files

Displays the data of each log file on a separate sheet. The corresponding options command line options are:

-t "//file" -T "@name"

load-writer-summary

Displays the data of each log file on a separate sheet, where the data on the sheets is grouped by the ODT files that are loaded. The corresponding options command line options are:

-t "//file" -T "rtllog:filename(@name)" -g ".//block[@function='::SfxFrameLoader::load']" 
-G "rtllog:filename(substring-after(.//stamp[@function='SfxObjectShell::LoadOwnFormat' and starts-with(.,'loading ')],' '))"
-i "@function='ReadThroughComponent'"

save-writer-summary

Displays the data of each log file on a separate sheet, where the data on the sheets is grouped by the ODT files that are saved. The corresponding options command line options are:

-t "//file" -T "rtllog:filename(@name)" -g ".//block[@function='SfxObjectShell::SaveTo_Impl']" 
-G "rtllog:filename(substring-after(.//stamp[@function='SfxObjectShell::SaveTo_Impl' and starts-with(.,'saving ')],' '))" 
-i "@function='SwXMLWriter::WriteThroughComponent'"

Intermediate XML File Format

The intermediate XML File contains the following elements and attributes:

<root>

<root> is the root element. It has no attributes. The child elements are <comment> and <file>.

<comment>

The <comment> element contains a comment. It is created for all lines in the RTL log file which do not start with a time stamp.

<file>

The <file> element contains the data for a single log file. One <file> element is created for each file name that is passed as parameter to RTLLog2XML. The file name is included in the attribute name. The child elements are <thread>.

<file>

The <thread> element contains the data for a single thread. The thread id is included in the attribute thread-id. The child elements are <block> and <stamp>.

<block>

The <block> element contains the data of RTL log file lines which have a type { or }. A single <block> element is created for each matching pair of these lines. The function attribute contains the function name, if present in the log file line. The owner attribute contains the author, if present in the log file line. The component attribute contains the component, if present in the log file line. The start-time attribute contains the time stamp included in the { line. The end-time attribute contains the time stamp included in the } line. Both time stamps are specified as XSD durations.

<stamp>

The <stamp> element contains the data of RTL log file lines which have a type |. The function attribute contains the function name, if present in the log file line. The owner attribute contains the author, if present in the log file line. The component attribute contains the component, if present in the log file line. The time attribute contains the time stamp included in the | line as XSD duration.

Personal tools