User Experience/DirectManipulationSnippets

From Apache OpenOffice Wiki
Jump to: navigation, search


Quick Navigation




DiMaS Header.png


This page intends to start a discussion on how to interact with “special” elements in the Writer document text and to solve some typical interaction issues with the document itself. At the moment there is no fancy name for it, therefore it will be called Direct Manipulation Snippets.

The target audience for the revised functionality is the “Small Business User”.

This proposal has been announced on the User Experience discuss mailing list. You can find the messages at “Proposal to work with special objects in OOo”.

The Problem

The following paragraphs list some of the “special” elements and the issues from my point of view. Please note that some of the descriptions are not very accurate and do only state the Writer module. I just want you to get the basic idea.


Working with hyperlinks in is not easy, although there has been some improvements in 2.3 (“Ctrl+Click-Behavior.”)

For future version of, it is proposed to add more items into the context menu to enable direct manipulation (Edit Hyperlink..., Select Hyperlink, …) In my opinion, this will lead to longer, less readable and more cluttered menus. As a result, we need a more specific approach.

For more information on hyperlinks, please refer to the Specification of Hyperlinks Behavior.

Smart Tags

Smart Tags scans the text and provide actions for recognized items, e.g. names of cities.

The actions are both placed in a separate menu and the context menu of the document text. The separate smart tag menu is accessed by Ctrl+Click, which is very similar to the handling of Hyperlinks. The same content is contained in a sub-menu of the text's context menu. The sub-menu is called “Open Smart Tag Menu”; the name tells us it is a menu (like the sub-menu arrow tells us) and it can be opened (like the sub-menu arrow tells us too). ;-)

For more information on the Smart Tags functionality, please refer to:


AutoSpellcheck checks the text and marks questionable items with the well known red wiggly line.

The spell check proposals are contained in a separate menu which is accessed by right clicking on the word. That behavior is a bit questionable, because the user can not access the conventional context menu for the text. Anyway, the good thing is that the menu is kept well arranged. From experience, many people do just not want to correct a word (e.g. a unusual forename) but want access to the context menu.

Agreed: Most people (including me) want the Context Menu. --Discoleo 15:17, 25 October 2007 (CEST)


The AutoCorrect functionality automatically corrects small mistakes in the text and helps the user to format the document.

Very often, people do not understand why the document content has been magically changed. To solve that problem, the Help Agent is used, a small windows that pops up at the lower right of the screen. Personally, I like the general idea of the Help Agent, but people do not use it or do not like this functionality. (There will be more information on the Help Agent later on.)

The changes by the AutoCorrect feature will possibly be detected later by the user. Unfortunately, AutoCorrect changes can only be reverted with the Undo functionality, that only allows to go several steps back. A specific change is not possible, e.g. revert changes made half an hour of work ago.

Consequently, there is a need for a better explanation why changes have been made and how to revert them afterwards.

For more information on the Help Agent, please refer to:

Is it possible to insert a smart tag where the change occurred?
  • One could store the initial value using this smart tag and revert later the changes.


At the moment, the Notes functionality of the Writer module is revised (Notes2.) One of the topics to be worked on is the direct access to Notes related functions inside the document text.

Today, a double click on the “Notes Anchor” (the tiny little yellow box between the characters) opens a dialog to edit the note. In future version of it will be possible to apply a note to a selection of text; the double-click does not work any more for Notes2 and interferes with “selection of word.”

Consequently there is a need to provide functionality for Notes2 that allows to edit a note when the “Note Window” (the box next to the Writer page) is not visible.

For more information on the revised Notes functionality, please refer to the Notes2 Wiki Page.

Action Refinement

Microsoft Word provides ActionRefinements, little drop-down menus next to the text that are used for e.g. AutoCorrect.

Unfortunately, the items in the ActionRefinement very often do not conform to the ones in the conventional menus. If something is inserted from the clipboard into the text, the items in the ActionRefinement menu are not identical to the ones in “Edit – Paste Special…”.

Some short comments on the ActionRefinement and AutoCorrect can be found in German language: Study of Microsoft Office 2003


At the end we can conclude that we have multiple options to access similar items (Double Click, Ctrl+Click, Right Click) and different kind of menu structure (stand-alone menus that replace the context menu, stand-alone menus that coexist to the context menu and sub-menus that extend the conventional context menu.) Sometimes, the user needs to instantly identify changes in the document to revert them and most of the time there is no good feedback of what is going on. And, some people do not even know that a context menu exists. So, how to solve that?

The Proposal

Please note that the following information is not intended to specify the behavior in a detailed way. It is just a starting point for a nice discussion to estimate the potential of that kind of functionality. At the moment, there is no intention to replace the current kind of formatting for of e.g. graphics and tables.

Provide Direct Manipulation In The Document Text

The general idea is to provide a well structured user interface that makes it possible to directly manipulate the items in the text. It is similar to the Action Refinement of Microsoft Word but covers more items, provides more consistency with regard to the conventional menu structure and provides more feedback to the user. It consists of:

  • Marker for the element (e.g. a word in the document text)
  • Heading (similar but not identical to a title bar)
  • Access to the help (to e.g. substitute Help Agent)
  • Content area (dependent on the kind of “special” element)
  • Access to basic actions (pre-defined most often used actions)
  • Access to more actions and options (to integrate the conventional menu structure)

At the moment I am a bit unclear how to access the “Direct Manipulation Snippets”:

  • Idea 1 “Substitution for Tooltip”: Here, the tooltip is replaced by the proposed UI. The bad thing is that the user has to know that there is a special element and is forced to wait until the Direct Manipulation UI appears.
  • Idea 2 “Here is something special”-control: Mark up the area around (e.g.) the word with some special control. Maybe a kind of options button directly inside the text.


Mockup Time! I hope that the following graphics speak for themselves, therefore I only want address the specialties.

Hyperlinks The mockup for the Hyperlinks shows actions to open or to delete (or remove) the Hyperlink in the text. The menu “More” could contain the other items listed in the previous table (e.g. Edit Hyperlink)
DiMaS Hyperlink 1.png

Another mockup shows the idea to integrate a preview to the target link. Maybe it is possible to preview other documents like PDFs or graphic files.

DiMaS Hyperlink 2.png
Smart Tags The mockup for the Smart Tags provides direct actions by simple buttons. Just click and go...
DiMaS SmartTag.png
DiMaS Spellcheck.png

At the moment I do not like the idea that the user has to select the correct word and then is “forced” to additionally confirm it. Hopefully, there is some smart way to integrate the language selection.

AutoGrammacheck No Image

As a few people have suggested, there is no reason grammar checking can't be added to this list. Presumably, (based on the above mock-ups) the grammar arrow would appear at the end of the sentence or grammatically incorrect section in question. If perchance a misspelled word (or other snippet) is the last word of the grammatically incorrect section, the tab concept proposed below may be helpful, to allow the user to choose to correct the grammar or the spelling.

DiMaS AutoCorrect.png

For some more special corrections it would be nice to let the user directly choose the state “before” and “after” in a kind of preview.

DiMaS Note.png

Another mockup (made several days before working on the proposal for the Direct Manipulation Snippet) for accessing the Notes. It is a bit different, but I'm sure you will get the idea:

Notes2 DesignProposal NotesPreview.png

And because of on question by Andreas, here another mockup that belongs to the previous one. It shows the area which is used to keep the pop up window active (e.g. when trying to move the mouse pointer from the Notes Anchor to the Button "Edit Note"):

Notes2 DesignProposal NotesPreview MouseOverRegion.png
User JaronBaron's suggested changes Suggested changes are menu access from the bottom and tabbed access to different snippet menus. Additionally, the mock-up shows an example the reference menu would show if an auto-reference recognition feature was added to OOo 3.
User-JaronBaron Snippets Mod.jpg
Action Refinement Due to the fact that no module supports some kind of Action Refinement, I simply skip it.

Clean Up the Context Menus and Toolbars

With the given proposals it should be possible to re-structure the context menus (e.g. Spellcheck, Smart Tags, Hyperlinks).

Please note that it is not intended to completely replace existing menus! At the moment the existing menus try to balance the concurrent requirements of fast access and logical menu structure. When using the Direct Manipulation Snippets, the conventional menus could be re-structured to make it a bit more logical. Example: Extend the basic “right-click” context menu with sub-menus items for: Notes, Spellcheck Proposals and Hyperlinks and avoid using stand-alone menus for that. The same sub-menu items could be used in the corresponding “More” section of the Direct Manipulation Snippets. This would keep consistency for all types of users.

How to Access the Direct Manipulation Snippets

Collection of Alternatives

Enhance the Conventional Context Menu

If some meta actions are available, the conventional context menu could be enhanced to: first show the snippet and let the user access the conventional context menu through the snippet.

Example: The user right clicks on a word marked with the blue Smart Tag mark up and the Smart Tag snippet is presented to him. It also contains a button “All Context Options >” which opens the conventional context menu.


  • The right click behavior is known to a large user base.
  • The user requests the corresponding action, therefore the annoyance factor is relatively low.


  • The right click behavior is not known to the user base who is targeted with the Direct Manipulation Snippets.
  • Needs some good integration with keyboard access (because the use needs to open the context menu from the snippet).

Personal Thought: Before we design such behavior, we should ask ourselves if it would make more sense to clean up our today's context menus.

Automatic Pop Up with Some Delay

If actions are available for the meta information, then a snippet is opened when hovering the mouse cursor for a certain amount of time on an element with meta information. Example: The user moves the mouse cursor on a word marked with the red AutoSpellchecking line and waits a short period of time. A pop up appears to inform the user what spell checking proposals are available.


  • There is no need for another key binding or modifier/mouse-combination. In fact, you just need “to sit down and wait”.


  • The people on the UX list gave some remark that this behavior could be very annoying especially for advanced users.
  • The people have to know that something will happen if they just wait.

Separate Key Binding

If actions are available for the meta information, then a snippet is opened after using a separate (not yet used) key binding or modifier/mouse-combination after placing the cursor or the mouse pointer on that element.

Example: The user moves the mouse cursor on a word marked with the red AutoSpellchecking line and presses “special key” (e.g. CTRL+Click). The snippet appears and shows the spell checking proposals that are available.


  • The behavior does not interfere with any of the already implemented menu functions.


  • The user has to know this separate key binding or modifier key / mouse key combination.
  • This behavior does not really help to understand the remaining functionality of (it is just another dialog window without reference to the other interaction elements).

Personal Thought: If the user needs knowledge about the new key binding or modifier key / mouse key combination, we have to provide this information in e.g. another tooltip. Then, the some of the restrictions already discussed in “Automatic Pop Up With Some Delay” apply here.

Control Element Inside the Document Text (The Text is the Button)

If actions are available for the meta information on e.g. a word, then a snippet is opened after simply clicking on that word. To represent this behavior, a button is draw a the position of the word.

Example: The user moves the mouse cursor on a word marked with the red AutoSpellchecking line. The visual appearance of the “normal word” changes to a button containing the same word. The user clicks on that button and the snippet appears.

DiMaS Alternative TheTextIsTheButton.png


  • Easy to discover due to the graphical representation.
  • Direct interaction with the object. Maybe it is too simple ;-)


  • Normal word processing is made difficult (positioning of the cursor, text selection starting at this word, ...). [In other modules like e.g. Calc this is less problematic because there is an additional input bar, but in Writer this is a real killer argument.]

Personal Thought: This behavior may be horrible if e.g. a user opens a text in a foreign language and every word is marked by the AutoSpellchecking algorithm. Kind of playing “catch me” with the text ;-)

Control Element Inside the Document Text (Separate Button Next to Text)

If actions are available for the meta information on an element and the user hovers the mouse pointer on that element, the element is highlighted (by e.g. a border) and a separate button is made available next to the object.

Example: The user moves the mouse on top of the word which is marked with the blue Smart Tag mark up. A frame is drawn around the word and an additional button is shown next to it. The user moves the mouse pointer to that button and clicks to open the snippet.


  • Easy to discover due to the graphical representation.
  • Text editing is not restricted (placing the cursor into the word or starting to selection the word is possible).
  • Similar to the solution available in the Microsoft Office Suite, therefore low effort for retraining the users.


  • The user will have to move the mouse from the “target” object to the button. This creates additional effort.

Proposed Solution

Based on the estimations in the previous section and the discussion taking place at the UX mailing list, I would like to propose the following behavior:

  • In general, the Direct Manipulation Snippets should behave like discussed in Control Element Inside the Document Text (Separate Button Next to Text). The given disadvantage could be addressed by a good snippet layout and an appropriate position when opening it.
  • Due to– in special circumstances – the necessity of providing a preview for the Notes, a behavior like Automatic Pop Up With Some Delay seems appropriate for the Notes functionality.
  • For the Hyperlinks, the behavior of CTRL+Click could be maintained for the advanced users. The CTRL+Click behavior for Smart Tags should be removed and implemented into the the snippets and the conventional context menu.
  • Less color is used to represent control element like behavior and to differentiate the controls from the “colorful” Notes Anchors. (The colors from the operating system could be used as well for the controls.)
  • Small symbols represent the group of the functions. This should also enable the consistency across the whole, because the symbols are already in use.
  • The mark ups in the text (e.g. the red AutoSpellcheck line) should be kept on the screen, even if we draw our controls. Otherwise the user might be confused if the spelling mistake vanished or was corrected automatically. (Still, I have no better idea after playing around with my graphics program for some hours ;-)


Here are the mockups showing how the feature could be implemented:

Step 1 There seems to be a spelling mistake in the text...
DiMaS DesignProposal Access WordOnly Step1.png
Step 2 The user can place the cursor on that spelling mistake and some more control elements appear:
DiMaS DesignProposal Access WordOnly Step2.png
Step 3 Clicking on the little action button (the one which shows the Spellchecking symbol), opens the Direct Manipulation Snippet. Please note that the position of the initial position of the snippet allows the user to just move the mouse upwards to select the most important items.
DiMaS DesignProposal Access WordOnly Step3a.png

But what happens if the space on the screen is not suffiecient to present the snippet in such a way? Then, it should move a bit:

DiMaS DesignProposal Access WordOnly Step3b.png
Special Case What if the meta data is contained in a longer text or we do have some strange zoom factor in the document? This is the proposal:
DiMaS DesignProposal Access LongText Step2.png
Open Points
  • Scaling the Document Text: Because of the fact that the graphical representation is directly drawn inside the Document Text, the control elements should be accessible even if the zoom factor is huge or small. Proposal: Keep the size of the action button.
  • Direction of the drop-down/up/left/right/somewhere:
    • Drop-Down: Similar to the behavior of conventional drop-down selections.
    • Drop-Up:
      • Here it is possible to use additional information like e.g. a header which does not increase the mouse distance.
      • Similar to the behavior of context menus.
    • Proposal: Choose Drop-Down/-Up according to the kind of element containing the meta information.
  • Position: Where to place the little button if the text is very long or the object does have a more random shape? Proposal: Lower right corner.
  • Graphical representation of the object highlighting:
    • The graphical representation should be...:
      • ... distinctive in comparison to the Document Text, even if the user has a bad taste and uses some very strange colors for foreground and background elements.
      • ... distinctive in comparison to the visual representation of the Notes Anchors (which hopefully will gain importance in Writer).
      • ... indicate that it is a control element (maybe we can respect the color palette given by the graphical operating system).
    • Examples: different background color, border lines, animations (Provide some nice and useful bling?)
    • Can we modify the markups for e.g. AutoSpellcheck or Smart Tags to make it less dominant when we are already showing the control elements?

Same Functionality For The Document

At the moment there are some basic ideas to how provide a UI for elements in a document, e.g. the Hyperlinks in the Writer. But are there issues with the Document itself?

Provide Direct Manipulation For The Document

In my opinion, makes use of too many tiny dialogs and does not help in typical situations. Some examples:

  • Example 1 “Edit File”: If an user opens a document previously stored on a CD-ROM, the document is marked read-only. The user needs to change the write permission or create a new document in Writer by clicking on “Edit File” in the standard toolbar. This is not really obvious...
  • Example 2 “Macros”: If a user opens a document containing Macros, she/he is asked to confirm the execution of the macros. Until the confirmation, the user has no chance to see the document content. So why not loading the document without executing the Macros and asking the user afterwards for the permission?
  • Example 3 “Password Protected Documents”: If the user opens a document that is “saved with a password” (which seems to mean that it is password protected *g*), the dialog is not visible in the taskbar/panel of Linux environments [Ubuntu 7.04]. Instead, it is “masked” by an already opened window without any useful feedback towards the user.

An intensive discussion has taken place on the UX mailing list concerning these examples. It seems that there are technical constraints that would not allow to realize e.g. Example 2. So please keep in mind that these examples are just used to show something in mockups. In the discussion, some more use cases for the information bar concept were mentioned.

The general idea is to provide an user interface to enable the direct interaction within the Document. If you look carefully, you will notice that this idea is based on the information bar in Mozilla Firefox. It consists of:

  • Time-to-Disappear (the kind of clock that states when the information bar will automatically disappear)
  • Heading (bold)
  • Explanatory text (normal)
  • Access to basic actions (pre-defined most often used actions)
  • Access to more actions and options
  • Access to the help (to e.g. substitute Help Agent)
  • Button to close the bar


Read Only and Notes The mockup shows two “stacked” information bars:
  • information on the “read only” mode of the document
  • information on “hidden” Notes inside the document
DiMaS Document HiddenNotesPane.png
Passwords The mockup shows an information bar. The advantage is that, if the document is unlocked, the document content will be contained in the same window. From the user's point of view, the object on the screen keeps the same position and size.
DiMaS Document PasswordProtected.png

Another idea is to replace the big lock graphic with a representation of the document type (e.g. Calc Document Symbol).

(Some of the graphics are taken from the amazing Tango Desktop Project released under the Creative Commons Attribution Share-Alike license.)

Clean Up The Context Menus And Toolbars

Within the given proposals it would be possible to clean up the context menus and the toolbars like:

  • remove the “Edit File” icon in the standard toolbar
  • substitute some dialogs (e.g. “Macros”, “Password Protected Documents”)
  • ...

Current Status

This wiki page contains just a proposal which is not part of any current or planned development activity. If somebody wants to implement it, I propose the following steps:

  • Identify more use cases (consider the current feature set of
  • Decide if this functionality really makes sense and will ease the interaction with
  • Finalize the UI and make it robust (e.g. for the information bars: what happens if the window has a low height, what happens if an information bar disappears after the specified time and the user want to recall it, ...)
  • Start coding ;-)

If you have any questions or if you need some help, then please contact me. --ChristophNoack 16:16, 26 November 2007 (CET)

Some Personal Notes

I do think that the overall structure of is very good. In any case it can even get better and solve some of the problems we have today. Of course, I'm aware of the point that this proposal will – at the same time – introduce other issues ;-) So I ask you to take part in the discussion to tell me if this is just nonsense or if there is enough potential for mature functionality including accessibility.

Some other discussion on how to improve the user interface of has taken place at UX mailing list e-mail "OpenOffice user interface". I don't have access to Microsoft Office 2007 nor Apple iWorks, therefore I may be not aware of already available better/similar solutions to address this topic.

Thank your for reading this article!

Personal tools