Aw080 documentation

Branch aw080

This document describes the branch aw080 and changes done there up to now (18th May 2012). The branch (now at apache under https://svn.apache.org/repos/asf/incubator/ooo/branches/alg/aw080/) has a quite long history, I started developing it back at Oracle times. It is intended to move quite some stuff forward to get a better, faster and more stable DrawingLayer.

Motivation

The branch is following a mixture of goals, all intended to overcome some pretty old limitations of the DrawingLayer in the OpenOffice code. The codebase is now nearly 20 years old and was extended by quite some people. Naturally, these used all their own styles and ideas to do so and had various backgrounds (e.g. knowing about linear algebra or not). Together with the bugfixes done (which is good) sometimes by people not knowing the basic principles of the module (which is not good) quite some non-optimal places were changed and leaded to non-optimal situations. The DrawingLayer itself has to be seen as a big, very central component and can be split at least in three basic parts:

1. The DrawingLayer Core which defines the DrawingLayer Model
2. The Renderer which is responsible for visualisation of the Model
3. The Controller which implements and offers all basic interactions with Drawinglayer objects, including Drag&Drop and others.

There are currently ca. 14 users of Drawinglayer, not only the visible applications but also Dialogs and various others. All these have to be taken care of and being adapted when making changes to DrawingLayer. All in all it's like a gordian knot and it's hard to even find parts to cut out and to replace. The main users are the Draw/Impress application, the DrawingLayer is more or less the basic Application on which these are based.

Goals

The goals fall in different categories:

1. Reworks
2. Cleanups
3. Simplifications
4. Restructurings
5. Modernizations

It has shown that it is impossible to move the DrawingLayer code forward with just one goal in mind. The dependencies are too huge; often when trying to cleanup one aspect you stumble over something else which is in the way and needs to be changed first; usually this extends to closed circles and thus prevents the changes. The goal for aw080 is to make as view as necessary compromizes to do this, but also to be brave in breaking those dependencies; with each change the first thought should be: How would this look in an ideal Drawinglayer? This gives the necessary global overview to even change things in the right direction without seeing the whole picture. To do so it is necessary to have an ideal picture of the DrawingLayer, something I needed years to develop the needed overview. This alone shows that it's not too well structured.

Prerequisites

The branch is based on work done already, namely CWS (ChildWorkSpace, comparable with a branch) aw033 (very technical docu about it can be found here) and sume successors using the changes. It added Primtives, ViewObjectContacts and lead – together with extensions in VCL – to AntiAliasing. It already cut some places in that gordian knot and prepared the way to continue with aw080. It represents a good in-between break in the renewal of the Drawinglayer. From my overview it should be roughly the 2nd third of needed changes to get it to a future-proof state where using it further will be possible and useful.

Changes done

I will now try to list the changes already done in aw080 and comment them. I will use no special order. This is not easy, it already is made up of 3000 changed files, the diff has 289031 lines (not even the current one, there happened more until today). I will also not be able to list all changes done but will concentrate on the bigger ones which stand for fundamental changes.

Old RTTI removed and replaced with standard C++ RTTI

The classic OpenOffice RTTI, defined ca. 18 years ago since C++ had none at that time (defined in rtti.h) was not very reliable, it is completely removed and replaced in aw080. When replacing it I found a lot of errors since it had to be defined in a derived class by a macro (TYPEINFO) in the definition and implemented by a macro, too (TYPEINIT_FACTORY, TYPEINIT). This was not reliably done over derivated hierarchies, thus the corresponding calls to use it (ISA, HAS_BASE, PTR_CAST) were also unreliable and often did not represent the true (or not all) C++ derivations. I found one case where a wrong definition was used in the class definition/implementation, probably as a hack to avoid one of these problems and to show wrong derivations by purpose. All usages were replaced, the file rtti.h was moved to binfilter which as single exception was not adapted (what would simply be dangerous). All other usages were changed to use dynamic_cast and/or static_cast and the usual C++ RTTI type definitions.

The old RTTI mechanisms were also used in the half-automated slot definitions of the applications, thus I had to adapt the slot definition compiler accordingly. It was also used heavily in Writer model node travelling, also adapted completely to more modern and reliable C++ RTTI mechanisms.

All usages of old list classes in Drawinglayer adapted to STL

Wherever possible I replaced the old tooling list classes with STL mechanisms and adapted the code accordingly. The DrawingLayer in aw080 is pretty much free of the old list classes. These still exist and it maybe another project to completely remove these (this is not trivial since the tools lists have special functionality with a currently remembered entry which is used sometimes).

Removed unused/very simple Item types

Quite some Items could be identified to not being needed anymore. I also removed a lot of very simple Item definitions which were not really needed and replaced them with their base types (e.g. a lot of items derived from the basic bool type can just be set/get and used using the basic type and the slot ID). This significantly reduces the number of symbols defined by svx.

Selection mechanism for SdrObject at SdrView completely rewritten

The selection mechanism holding SdrObjects at the views was quite unsafe. This leads still to many situations where there are still objects in the selection which were removed or even deleted already, a reason for many crashes. The classes for holding and maintaining the selection were completely isolated form the view classses. They use STL mechanisms (sets) now which are sorted and also avoid double selected objects by definition. All non-reading (non-const) accesses to the selection at the views were isolated and reduced to a minimum to guarantee safe handling. Quite some changes were needed to do so, but it coud be done. The controller part of DrawingLayer still will need a redefinition after aw080, but security and stability will already be increased.

All mechanisms working on selections wre adapted accordingly. There exist many methods in many view implementations in the applications which are based on the view's selection, all were adapted to the safer mechanism. As an example, most usages were somehow iterating over the selection while at the same time changing it. Many diverse mechanisms to avoid problems with this were in place, most the result of old fixes. Most of these could be adapted to get the current selection at method entry and delete it at the view, cleanly working on the SdrObjectVector (based on STL) fetched from the view as selection and building a new one containing the new slection. At method exit, either the old (when no change was made) or the new one is set as new selection in one single call. This also avoids too much broadcasting over selection changes, another source of common errors.

All SdrView classes cleaned up

I secured the whole SdrView derivated hierarchy, all constructors were corrected (member initialization). Neraly all constructors were reduced to a single one. The hierarchy from SdrPaintView up to SdrView is a single entity, thus the internal constructors went all to protected mode. Cases where in-between classes were incarnated were corrected. All members of all SdrView hierarchy classes were identified, evtl. renamed to make them unique and better readable and adapted to hungarian notation (leading m*) added.

Most of the methods at the SdrView hierarchy were checked and also updated. Many were removed, some replaced. Inlines (obviously from earlyer changes where no one wanted to get incompatible) were removed and replaced with the real calls. Methods which need not to be virtual were made non-virtual. Names were corrected and adapted to make calls more readable and self-explaining. A lot of parameters were removed since they were no longer needed or could be avoided with better mechanisms. Also cleaned up and checked all friend declarations to create more safety.

OrdNums for SdrObjects and SdrPages secured/reimplemented

The OrdNums at SdrObjects which describe the position of the SdrObject in the parent class (list or Page) were on demand refreshed in an unsecure way, many instances triggered it by hand or called it directly. This is now secured and centralized, only one explicit friend method used in a single place is now allowed to do that changes. This also was an ongoing root of errors. Also renamed all usages to NavigationPosition to increase readability/understandability. The same has happened to PageNumbers in the SdrPage class hierarchy.

Page and List classes both based on one base class

The classe for pages (SdrPage) and group obejcts (SdrObjGroup) were derived from different base classes. This lead to many special handlings of traversing over object hierarchies and others, also a cause for many extra code and errors. Both are now based on SdrObjList. SdrObjList itself was adapted to support this using virtual methods for getting child and parent access. The derived page and group classes implement these accordingly. All places using page and group traversioning were adapted to this simplificaton. Insertion, removal and change of position/NavigationPosition are reimplemented and secured, STL classes are used to handle this.

One basic definition for the exchange of list of SdrObjects

A single basic definition SdrObjectVector is defined for this in SdrObject.hxx and used in all places/occurrences where lists of SdrObjects need to be exchanged (selection, travelling over hierarchies, etc.). Of couse this is using a STL vector class as base. All places in Drawinglayer where SdrObject lists were exchanged were adapted to this mechanism, many extra handlings and list conversions could be removed. All iterations can now use this base class. Even the SdrObjListIter is internally such a list created at construction time and could be used directly instead of using the iterator anymore.

Many base classes of DrawingLayer are now based on boost::noncopyable

Most classes in the DrawigLayer Model are now derived from the boost::noncopyable class to securely avoid unwanted (evtl. implicit) cloning of these. This includes the whole SdrObject hierarchy, the SdrObjList/SdrPage hierarchy and others.

SdrHint reimplemented/redefined

The SdrBaseHint used for SdrObject change broadcasting got simplified, all unused types of broadcasts removed. All places using boroadcasts adapted, many simplified or removed, too. In some cases broadcasting could be completely avoided and a more intelligent automatic mechanism being used, mainly when insterting/removing SdrObjects. There is also a class now (SdrObjectChangeBroadcaster) which supports broadcasting better by doing it on destruction time. Adapted all broadcasts to make use of this.

SdrObjectUser mechanism removed

The SdrObjectUser mechanism which existed to allow classes using SdrObjects to register and to react on it's (unexpected) deletion was completely removed and all usages were adapted to use the already existing broadcasting mechanism.

New SdrObject/SdrModel paradigm

One of the most basic changes in aw080 is the relationship of SdrObject/SdrPage to SdrModel. Currently, SdrObjects can be created and exist without SdrModel. This is problematic and a source of many crashes, mainly because SdrObjects use a SfxItemSet to hold properties which itself relies on an SfxItemPool which is part of the SdrModel. No model, no SfxItemSet. In aw080, all SdrObjects need a SdrModel at construction time and it it is defined to not change during SdrObject's lifetime. This ensures much more stability in SdrObject and thus in DrawingLayer Model handling. It made migrating SdrObjects between SdrModels superfluous, too, so MigrateItemPool mechanisms could be removed. An objects instertion to a page is now completely decoupled from it's existance and always has a SdrModel and a Pool for it's PoolItems.

All usages were adapted to this, all SdrObject constructors, too. It is now safe to have helper SdrObjects vor varoius purposes; this was used in the past, but always was more than risky. It was neccessary to insert those SdrObjects somewhere (to a SdrPage or group) to have them get a SdrModel; existing hacks to make these invisible or similar were removed.

The old mechanism to insert an existing SdrObject to another model and thus really moving it was very dangerous and the source of many errors. It is now replaced with the SdrObject cloning mechanism; it now allows to clone an SdrObject to an alternative given new SdrModel or inside the current one (default). This makes removal/insertion of SdrObjects to SdrModels explicit and safe. All usages of this mechanism were adapted to the new ones, all SdrObject implementations of cloning adapted to do this safely and reliably.

The same is done for SdrPage and it's hierarchy, also complete with adaptions to all usages and all derivations. Constructors are reduced to a single one per class, members are adapted to hungarian notation and cleaned up. SdrPage has a method to also clone it to another model if it needs to be copied, completely implemented for all derivations.

SdrObject geometric definitions changed to basegfx classes

This is the change of all geometry definitions at SdrObjects to double precision, e.g. using basegfx::B2DVector, basegfx::B2DPoint, basegfx::B2DPolyPolygon or simply double values as needed. Nearly all interfaces in DrawingLayer adapted to use these, all usages adapted in all applications and implementations.

SdrObject transformation changed to basegfx::B2DHomMatrix

This is a very basic paradigm change for SdrObjects. They now have a matrix-based definition for their basic object transformation, based on the basegfx::B2DHomMatrix class. Object size (scale), position (translate), shear and rotation are combined by linear combination as defined in Linear Algebra. This allows setting and getting these values back unchanged in double precision for the first time with SdrObjects.

The current implementation just applies geometrical changes to the object's geometry: When e.g. currently mirroring an object, it's geometry gets actively mirrored, but the state of the object – if it is mirrored or not – is lost. This is the reason why hor/ver mirroring cannot be switched on/off currently for SdrObjects or better controlled. This will change with aw080.

This is a very big change, adaption of all usages of all old changing methods is ongoing work. To support this, bridging helper classes are defined in svx/inc/svx/svdlegacy.hxx and used currently. These methods are defined as temporary helpers and will all be removed and replaced at the end of the adaptions. This is the main working point which is still ongoing with aw080 today. All usages of SdrObjects need to be adapted (all 14 users), the full Draw/Impress application with all modifers, Writer and Calc DrawingLayer usages, but also all import/export filters and UNO API implementations.

All SdrObjects are now constructed getting an initial transformation in the constructor, all constructors for SdrObject hierarchy are adapted to that. The object itself is always seen as being in a 'generic' state e.g. a rectangle object internally uses the unit rectangle from (0,0) to (1,1) and the transformation is thought to be adapted to it. This requires no geometry data any longer at most objects, it's implicit. All internal SdrObject handling can be unified in these unit coordinate system, e.g. glue points are now always relative in the ranges from 0 to 1 in X and Y. There are two exceptions for lines and measure objects (which have the generic form from (0,0) to (1,0)) and for polygons: the contained geometry is kept in world coordinates for practical reasons, but there is an interface supported to handle the contained geometry scaled to unit coordinates always being inside (0,0) to (1,1). This also means that principally all SdrObjects will be rotatable and shearable after aw080. This is already prepared in the working trunk, one of the reasons to allow this early for OLE and GraphicObjects.

SdrPage geometry changed to double precision (size, borders)

The PageSize and it's border definitions are no longer based on integer, but on double precision. The page size is called PageScale now and is a basegfx::B2DVector. All interfaces adapted to this, all usages, too. There are some usages of this, all calculations based on page size and all interaction adaptions e.g. limitations to page and surrounding area needed to be adapted.

Primitive creation paradigm change (also for 3D primitives)

Currently when the Primitves for an SdrObject are fetched using the ViewObjectContact a new Primitive representing the highest Primitive to represent the graphic representation of the object is generated. It is then compared to the existing one to reliably detect changes in visualisation (primitives contain all needed information for visualisation by definition). In most cases when nothing changed the old one is kept, preserving all it's buffered decompositions and reusing it. The reason for doing this was at the time when Primitives were implemented that the reset of the highest level Primitive was not reliable. In aw080 this is changed since the mechanisms to detect an object change are more reliable, so this is no longer needed. All operator==() of all primitives were removed in aw080, they are no longer needed.

BoundRange definition view dependency

A very old problem is the correct BoundRange (bounding rectangle) calculation for graphic content. It is problematic since all older implementations did try to calculate this independent from a View, only based on model data. This is insufficient due to the fact that many graphic parts are view dependent. For example, each hairline is view dependent (and hairlines are the default in AOO); imagine zooming in and out, the lines defined to cover one pixel will cover various areas in logical coordinates. When zooming far out a hairline may have a corresponding logical width of some centimeters.

Thus, to calculate this correctly and to avoid refresh and repaint errors in the future which happen because the calculated range is too small, aw080 makes this calculation view-dependent as far as possible since the ViewTransformation is needed for correct calculation. As a consequence this means that objects have one BoundRange per view. Up to some rare occasions it was possible to change this. The last ocurrences use a fallback to an empty ViewTransformation and are marked for replacement in the future.

EditEngine changes

The EditEngine, the tooling used for layouting and editing text in DrawingLayer did not really fit into the overlay mechanisms offered by previous changes because it was implemented to only work on Windows (the class Window from Vcl) directly. The overlay used to speedup and avoid flicker uses a virtual device. This means that in older offices each time text edit started the buffering using overlay had to be replaced by unbuffered viewing and vice-versa at text edit ending. In aw080 I changed EditEngine to be able to paint to OutputDevices of any kind, the Window part is only used for the interactive part while editing (key input), thus separating the view and controller part.

This change is already in the trunk as bugfix, extracted for this from aw080. It is also the preparation to use Primitives for EditEngine painting in edit mode in the future. This will also allow using the selection schemata already used in Writer and Calc, using the system's default selection color in a transparent shaded fashion and to get rid of the XOR selection painting. It also removes the necessity to paint a background for expanding text editing in a 'guessed' background color (it's calculation was always unreliable).

Mirroring of CustomShapes

Only one existing SdrShape has the ability – despite having no transformations – to remember that it's mirrored, it's the CustomShape. This was necessary and the guy implementing it had to do some heavy coding to get it working. With transformations this extra code can be removed (and is already in aw080), but will need thorough testing for functionality, especially in MS imports/exports.

All NBC* methods removed

The old code has a NBC* method for nearly every method on SdrObjects which change object attributes or geometry on the DrawingLayer Model. NBC is short for NonBroadCasting. Broadcasting the changes of objects always was expensive, thus this possibilities were offered. It did not only lead to having always two calls for each function, but also forced the programmer using it to know enough about the DrawingLayer and the Application/User programming to decide if he could possibly call the non-NBC method. This led to what it had to lead, lot of notifications were missing and even after years of fixing bugs this conceptional error still causes errors. In aw080 all NBC-methods are removed for security reasons and the broadcasts are enhanced to happen less often and to be cheaper.

SdrVirtObj isolated and moved to Writer

The SdrVirtObj (used as a kind of proxy for repeating existing SdrObjects) is only used from Writer, I moved it there exclusively to have it out of the way for further DrawingLayer changes in svx (it does not fit in any working concept anyways, the plans to solve problems with SdrObject handling longtime in Writer will not use it).

SdrVirtObj usage in Writer and corresponding offset to anchor revamped

Despite it is only used in Writer, it is hosting an offset to the anchor of the Writer model. This offset and it's calculation was highly blurred with old stuff. I completely sorted out the stuff happening and reduced it to the needed stuff. Changed/adapted the in/exports in Writer accordingly and tested it.

All Interactions overhauled, all on double precision

All implemented interactions in Draw/Impress, Writer and Calc are adapted to double precision. Classes partially reimplemented based on new basegfx base classes. All snapping, cropping, grid calculations, edges of other objects, page boundaries and other stuff transferred to double precision. All stuff from other usages adapted as far as needed, still some work to do here.

All object creators revamped to double precision

All object creators (when activated an object insertion from the object toolbar) adapted or partially reimplemented for double precision. Same snapping and precision as the interactions.

All snapping/alignment reworked

All snpping to grids, forcing to angles, aequidistant modes (pressing SHIFT) and other stuff adapted to double precision.

Many methods renamed to self-explanatory names

Often it is complicated to grep the office code for some method defined for a specific class. This makes cleanup and redefiition work pretty complicated. I have renamed many methods to more unique, self-explanatory names, e.g. GetModel() which is available on many classes is now split into getSdrModelFromSdrObjList, getSdrModelFromSdrView, getSdrModelFromSdrObject and others.

Many varaiable exchanges in methods changed to reference

A common cause for errors are null-pointers and not checking them, in most cases because it makes no sense to not have the needed object handy in the current context. This can often be avoided by not passing the object to a method by pointer, but by reference. This makes clear that this method is only to be called with an existing object and the method itself often gets much easier to implement, read and understand. One exampe is the SdrObject construction which now takes a reference to a SdrModel and a reference to a basegfx::B2DHomMatrix. I changed many methods in this direction and adapted the callers. In nearly all of the cases the overall handling became simpler and safer, in the rest checking for object existence before calling functionality which relies on the object is better anyways.

Many MapMode usages changed to getting ViewTransformation from OutputDevice

For quite some time now I had added methods at the OutputDevice to support the ongoing changes. These include getting the MapMode which is the ViewTransformaton as basegfx::B2DHomMatrix, and also it's inverse for back-transformations. I added for aw080 some more convenience methods at OutputDevice to directly convert MapModes to transformations and other handy stuff. I adapted most usages in DrawingLayer in aw080 to use these instead of the old LogicToPixel() and others, thus complete geometries can be transformed with single calls and the ViewTransformation can easily be added to combined matrix stacks.

Extended basegfx matrix tooling (UNO converters, buffered decompositions, …)

The matrix tooling in basegfx was extended for converters for UNO API classes like Matrix, sequence of points and other useful stuff. Base classes to have a basegfx::B2DHomMatrix in it's combined form and buffered in it's decomposition were added, this is e.g. used as member in SdrObjects so that it is not necessary to get the matrix, decompose each time just to get e.g. object scale (size). Convenience methods at SdrObject hand out these directly, e.g. the method getSdrObjectScale(). Caution: Of course this scale can be negative in aw080, expressing that the object is mirrored in X and/or Y, so it may be necessary to get the absolute of the vector.

PresObj creation completely transfered to new SdrModel paradigm and adapted to double precision

All default object creation for the Title/Content logic on MasterPages and Pages in Draw/Impress, their layout and handling were changed to use double precision and new mechanisms.

SdrView has a delayed notify mechanism

One of the reasons the broadcast mechanisms are expensive is that every change is broadcasted immediately. For aw080 I added a buffered, delayed notification for all actions on SdrViews. e.g. when 1000 objects get moved, there are no 1000 broadcasts disrupting the work in progress, but only a collected one when the action is done.

SdrHandle creation and lifetime revamped, double precision adaption

All classes for SdrHandles were relayouted, members checked and changed, adapted to double precision, adapted to transformations and hungarian notation.

No insert reason needed for object construction

Inserting a SdrObject to a SdrPage or at a SdrView needed a insert reason in the current office. I redesinged the logic using this, so this is no longer needed.

FASTBOOL removed in aw080

Usage of FASTBOOL completely removed on aw080, except for binfilter module. As you might guess, there were places to be found where it was misused to hand over values except true and false, e.g. '2' and others, these were corrected and removed.

UI elements adapted (Helplines, etc.)

All DrawingLayer UI elements like helplines, grid, extended cursor on object move, object replacement during interacion and others are adapted to double precision and the changed SdrObject/SdrModel paradigm, also using transformations.

SdrObject Attribute handling simplified

In the current office there are mechanisms to handle attribute changes when a SdrModel changes from one SdrModel to another. This was necessary to repair various aspects, e.g. unique name necessities and others. Since a SdrObject in aw080 can no longer change it's SdrModel during lifetime this complicated and error-prone mechanisms are no longer needed and are removed. When now cloning SdrObjects to another SdrMopdel these mechanisms are in place anyways since they always also corrected on pure insertion.

Drag and create methods adapted to double precision

All interactive methods for creation and/or modification of SdrObjects are adapted to double precision and do manipulate SdrObjects by using transformations. This allows real interactive mirrorings where e.g. the evtl. contained text can be seen mirrored during the running interaction without tricks.

GluePoint handling changed to always be relative to object in unit coordinates

As described SdrObjects have an implicit unit coordinate system ranging from (0,0) to (1,1). Thus, glue points use this definition now too. Unfortunately there is a 'Distance from Edge' mode for them which needed special treatment, but this is possible and done. This makes handling much simpler, e.g. to get the discrete position (Pixel coordinates) of a GluePoint in a view just multiply it's unit coordinates (e.g. 0,0) with the object transformation and with the ViewTransformation from the used OutputDevice.

Things to be done

This is work in progress. A lot of the changes need to be tested carefully. While the built version already runs stable, a lot of investigations and fixes in not yet working properly mechanisms has to be done before it can be integrated. There will be no further deep and basic paradigm changes (at least I have none planned), but stabilizing and finishing the current changes will still take some work. Also, the overhaul of all SdrObjects is not finished yet. There also needs to be taken care for connector stuff, gluepoints and other special things which are often forgotten, but need to be taken into account, too.