Build Environment Effort/HID List

Build Environment Effort
Quick Navigation Customers Build Performance Elimination of Non-Pro Dependencies Module Problems Split Build Scalability HID List New Build System Requirements External Source Tarballs ToDo Build System Analysis Zen of gbuild Gbuild Bootstrapping Module Migration Module Migration Tips Build System Analysis:capstone2013 windows build Capstone 2013 Client Requirements Document Capstone 2013 Technology Review & Implementation Document
About this template

The problem

Function commands (e.g. entries in menus or toolbars) and GUI elements like controls, dialogs or windows in OOo have IDs. Such an ID is called a "help ID" (HID). The reason for that is - surprise! - that this ID is used to assign help content to the command or to the GUI element. As an example, a user can activate the "Extended Help" feature in OOo and click on a menu entry or a toolbar button. Then a HID is taken from the command that is bound to the selected menu entry or clicked toolbar button. In the same way a user can click on a button in a dialog. Then a HID is retrieved from the button that was clicked. In both cases the HID is "wrapped" into a help URL and the OOo Help Content Provider is asked to provide help content for this URL.

HIDs can also be used to automate OOo: the testtool application uses HIDs to instruct OOo to execute a command or operate a control. OOo contains a testtool client component that is instructed by the testtool application through a remote bridge. If this client receives a HID for a command, it executes this command using the Dispatch API of the Controller object representing the currently active document window. If the client receives a HID for a GUI element, it searches for the GUI element with that HID (first in in the currently active window, then in all windows) and performs an action on it.

How do these HIDs look like? In the testtool application and in the executed test scripts HIDs are strings, sometimes called "long names". The help content provider also uses these "long names" to reference a particular help content. Inside of OOo the HIDs are integer values (with some exceptions explained below).

Obviously OOo and the HID "customers" (Help Content Provider, testtool application) speak different languages and so a translator is needed. This translator is the famous (notorious?) "HID list", hid.lst. It defines a mapping from the "long names" to the integer values. If a test script contains a "long name" of a GUI element or command, the test tool "translates" it into the assigned numerical ID and uses it to address the GUI element or execute the command. OTOH if help is requested for a GUI element or a command, the help content provider internally "translates" the ID in the help URL to the "long name" und uses that one to find the content assigned to it. So far, so good.

Unfortunately the hid.lst imposes some problems on our build process:

• We have a hen and egg problem: the hid.lst is built at the end of the build, but it is needed to build the helpcontent2 module, where it is even committed to the SCM repository. So this list always is not the most current one. If this was the only problem, we could fix that by changing the build order (though this has some other drawbacks), but unfortunately there are more of them.
• The testtool can't work correctly if the hid.lst from the CWS is not "delivered" correctly; there is always some confusion and uncertainty which list is the correct one to use and the whole process is prone to errors.
• Some ID mappings are generated automatically; this mapping is susceptible for subtle changes that can happen unnoticed.
• The HID list generation is an awkward and time consuming process.

So it would be desirable to get rid of the hid.lst by letting all participants (help, testtool, OOo itself) work with the same "long names".

The current process

Note: parts of this description are outdated. No more hid.lst since DEV300m91, 
see http://eis.services.openoffice.org/EIS2/cws.ShowCWS?Path=DEV300%2Fchangehid

Commands and GUI elements are treated differently.

Commands

In the olden days of OOo our commands have been represented by an integer number only. These numbers are called "slot IDs". In the source code we have a lot of #defines for slot IDs, e.g. in sfx2/inc/sfx2/sfxsids.hrc where we have code lines like

#define SID_SAVEDOC 5505

From the problem description above it shouldn't be surprising that "SID_SAVEDOC" is used as the long name of the command while 5505 is the internal numerical ID. Thus the hid.lst contains a line

SID_SAVEDOC 5505

that does the mapping.

When we moved our command dispatching to a UNO based framework that always uses URL style command strings, we first simply used URIs like "slot:5505" to execute the corresponding command. Later on we gave them "speaking" names. The command with slot ID 5505 e.g. now is called ".uno:Save"⧼cite_reference_link⧽. This gives us an opportunity to get rid of the ID mapping as explained below.

GUI elements

There are three ways how a control can get a HID:

Automatic HIDs

GUI elements created from a resource file can get "automatic" HIDs when the resource is loaded. The corresponding "long names" of course must be created in the build process. Otherwise they couldn't get into the hid.lst. These IDs are generated in a special context (the particular resource) and so without further actions can't be used in a global list because of possible name clashes. We avoid these clashes in the same way as we do it in C++ source code: by using name spaces. Here's an example for an automatic HID:

sfx2:CheckBox:RID_DLG_ALIEN_WARNING:CB_WARNING_OFF 1111557136 

It defines the HID for the checkbox labeled CB_WARNING_OFF that belongs to the dialog RID_DLG_ALIEN_WARNING that is defined in the code module sfx2. The numerical value is calculated by the resource compiler (it doesn't matter here how this is done). Here's an explanation for the construction of that "long name":

RID_ALIEN_WARNING is the resource ID of the dialog. It is defined in a globally accessible header file as

#define RID_DLG_ALIEN_WARNING           ( RC_DIALOG_BEGIN + 0)

CB_WARNING_OFF defines a sub resource that is only valid in the context of the global resource RID_DLG_ALIEN_WARNING. It is defined in an internal header file that is only included by the resource (rsc) file defining the dialog and the code (cxx) implementing it:

#define CB_WARNING_OFF  16

How can we verify that the "long name" shown above is globally unique?

Usually several resource source files (src) are compiled into one binary resource (res) file. The resource compiler guarantees that

• in one binary resource file a particular resource ID value can not be used for more than one resource elements of the same type (here: a dialog)
• in one resource element a particular resource ID value can not be used for more than one sub resource elements of the same type (here: a checkbox)

So we can make the "long name" unique for the whole compiled binary resource file if we never use the same define for different numerical values in the same context (binary resource file or single resource), something that can be controlled easily. To make the "long name" globally unique, we add a prefix that identifies the resource file. In sfx2 we have only one binary resource file, so the module name is sufficient.

HIDs assigned by resource files

A HID can be assigned explicitly in the resource file:

ModalDialog RID_DLG_ALIEN_WARNING
{
   HelpId = HID_WARNING_ALIENFORMAT;
   (...)
};

defines a HID for the whole dialog. The hid.lst file contains

HID_WARNING_ALIENFORMAT 33388

The numerical value is taken from a header file that defines HIDs. Each code module where HIDs are used has its reserved integer value range:

#define HID_WARNING_ALIENFORMAT                     (HID_SFX_START + 320)

HIDs assigned by code

Not every GUI element is created from a resource file, some are only handled by code. So also the HIDs are assigned in the C++ code by calling the SetHelpId() method of the CUI element, inherited from the vcl class Window. As an example, here's the code that assigns a HID to the Navigator window:

pWindow->SetHelpId ( HID_NAVIGATOR_WINDOW );

Again the HID is taken from a header file:

#define HID_NAVIGATOR_WINDOW                        (HID_SFX_START +  86)

and ends up in the hid.lst as

HID_NAVIGATOR_WINDOW 33154

The generation of the hid.lst is completely based on resource (rsc) files. So the HID of the Navigator would be invisible for the build if we didn't apply a trick: each HID used in code only is collected in a special resource file called "hidother.src" that is processed by the HID compilation, but not included in the binary resource file. For the Navigator it contains

hidspecial HID_NAVIGATOR_WINDOW                 { HelpID = HID_NAVIGATOR_WINDOW; };

The proposal

Commands and GUI elements must be treated differently and code changes will happen at different places. So my proposal has two parts.

Commands

With the mentioned ".uno:..." command names we have strings that can be used as long names for commands that are directly understood by the OOo code without any "tranlastion" (mapping). And indeed for some commands they are already used in the test scripts. By converting all tests and all help indices to use these ".uno:..." names we can get rid of all command HID mappings in the hid.lst.

So the proposal for command HIDs is:

Convert all help content files and test scripts to use the ".uno:..." long names instead of the "SID_..." long names and always use the the command names (right hand part of the command "URLs") instead of the SlotIDs inside of help content URLs.

This will require changes in helpcontent2, testautomation and sfx2 (where the help URL construction is implemented). Perhaps some changes in "automation" are needed also.

GUI elements

GUI elements all derive from the Window class in vcl. This class has two methods that deal with HelpIds:

void SetHelpId ( ULONG nId );
ULONG GetHelpId() const;

All three described methods to get a HID operate on this property. So this is what we must replace by something that references the "long names".

We also have methods dealing with "SmartIds" that more or less are Strings that have been planned to replace numeric HelpIDs. It might be tempting to use them for direct "long name" support, but below I will explain why I don't think that Strings should be used but ByteStrings instead.

Let's start with the case where my proposed change can be understood most easily: the setting of HIDs by code.

Currently we have code like

// found in a header (hrc) file 
#define HID_CONTROL_ID (HID_SOMETHING_START + 42)

// found in a cxx file 
pControl->SetHelpId( HID_CONTROL_ID ); 

The simplest change to get the long name into the code is:

#define HID_CONTROL_ID     "HID_CONTROL_ID"

This can easily be achieved by some scripting.

Now we just need to change the help ID member in the Window class in vcl and the Get/SetHelpId methods so that they can take or return ByteStrings. A String obviously wouldn't be a good choice as it would convert the 8 bit character constant into a 16 bit UniCode value that occupies twice the space, only to fill it with NULL bytes.

With the new #defines as show above, the rsc compiler now also would see character constants where formerly it saw integer values:

ModalDialog RID_DLG_ALIEN_WARNING
{
   HelpId = HID_WARNING_ALIENFORMAT;
   (...)
};

So the rsc code also needs to be changed. The rsc compiler will save the HelpId member of controls as a UTF-8 coded strings into the binary resource file. The ResourceManager must become aware of the new data type and handle it accordingly, means: get the now alphanumeric HIDs and assign them to the ByteString HelpId members of the controls.

Finally we have the "automatic" HIDs. They are constructed in the following way:

name_of_module:resource_type_of_control:symbolic_resource_id_of_global_resource:symbolic_sub_resource_id_of_control 

Unfortunately it is impossible to create these IDs either in the src compilation or at runtime as the necessary information is not present - all symbolic names have been removed by the preprocessor of the resource compiler. After considering several tricks to transport this information (symbolic names of global and local resources) into the binary resource, it turned out that any way to do that would create build steps that are as ugly as the old ones we want to get rid of. So a different approach will be used.

A script will go through all src files and will create HelpIds following the same algorithm as the HIDCompiler does nowadays. These HelpIds will be written into the src files. Or in other words: the "auto-HelpId" feature will be removed. In future HelpIds always will be specified explicitly, either manually or by executing the mentioned script once on every new src file. As the auto-HelpId feature is limited to src files and this format is doomed, this won't create too much work in the future.

So the proposal for GUI element HIDs is:

Convert all HID defines from integer values to string constants and all automatic HIDs from using symbolic resource defines as sub-strings to using string representations of the integer values of these defines instead. Adjust vcl, resource compiler and resource manager to handle the new data type. Make auto-HelpIds to explicit HelpIds. Convert test scripts and help content to use the new automatic HIDs.

---

Influences on future work

Code developers

As the auto-HelpId feature has been removed, you will need to either specify HelpIds for every control that either shall be part of an automatic test script or that should get a help text or you will need to execute a script to generate these IDs for you. Not every control needs one, even the automatic HelpIds have been created only for certain resource types (though more types with HID exist than without).

If you work on src files, be aware that you must use (byte) string constants instead of numerical values for HelpIds. It is recommended to use module prefixes in these strings. As an example, a former #define HID_FOO_BAR in module "mod" has been converted by "MOD_HID_FOO_BAR" in cws changehid.

If you have used Help URLs in your code, nothing changes. The vcl and toolkit code is able to differentiate between HelpIds and HelpURLs (if they are provided through the suitable API calls).

Help content developers

Nothing has changed for you. You will continue to use the "long names" in your xhp files directly and OOo will still provide you with them in HELP_DEBUG mode (provided that a developer has assigned one to the control). The hid.lst always was evaluated by the Help linker in the build process, this step just has been removed.

Test automation developers

Nothing has changed for you - basically. You will continue to use the "long names" in your win and sid files direclty and OOo will still provide you with them through your "Declare" tool (provided that a developer has assigned one to the control).

There is a small hid.lst committed into the testautomation module that contains a few hard coded ids that we needed to keep until the testtool itself can be updated to a development milestone that contains the CWS changehid. By keeping a few slot ids in the hard coded hid.lst we can still use the old testtool though nearly the complete infrastructure has been exchanged.

There is one thing you need to take care of in the future: as all "long names" have been translated into integer values by basic code using the hid.lst, it was never necessary to treat long names case sensitive. In the future is is required that you write the long names in your win or sid files exactly as they are provided to you by the developers.

⧼cite_references_prefix⧽ ⧼cite_references_link_one⧽ ⧼cite_references_suffix⧽