Quaste TCS Tool Specification

From Apache OpenOffice Wiki
Jump to: navigation, search

This specification is a DRAFT still in discussion.


There is a need of a web-based tool for writing TCSs (Test Case Specifications) and manage them over QUASTe.
Such a tool already exists (SUN internally) but its version 1.0 has many usability drawbacks and missing features.

The new version of the TCS tool should be easy to use but fulfill expectations and needs of the TCS writers.

In terms of UI design, we rejected as far as possible the former use of "wizards" - which don't give the user a global overview of his current work - and the pop-ups which even break the work flow.

We adopted a "tabbed" design which offers all the features on 1 page.

When final, the TCS tool will consist of two hyperlinks in QUASTe: "List Test Case Specifications" and "Create Test Case Specification".


Reference Document Check Location (URL)
Prerequisites [passed/failed] n/a
Product Requirement, RFE, Issue ID (required) [available/not available] <PLEASE ENTER LOCATION HERE>
Accessibility Check (required) See accessibility section for check list
Test case specification (required) [available/not available] <PLEASE ENTER LOCATION HERE>
IDL Specification [available/not available] <PLEASE ENTER LOCATION HERE>
Software Specification Rules n/a n/a
Other, e.g. references to related specs, Product Concept Document <PLEASE ENTER LOCATION HERE>


Role Name E-Mail Address
Developer Helge Delfs hde@openoffice.org
Quality Assurance Éric Savary es@openoffice.org
Documentation Helge Delfs
Éric Savary
User Experience Éric Savary es@openoffice.org

Acronyms and Abbreviations

Acronym / Abbreviation Definition
TCS Test Case Specification
QUASTe Quality Assurance Statuspage

Detailed Specification

The needs: easily create and find TCS.

  • TCS Creation:
    A TCS is first created by defining its general properties, then filled with life by adding test cases to it and finally displayed in a preview summarizing all this information:
    • Properties: Define the general properties of a TCS (which OOo version, application, how often should the TCS be run).
    • Test Cases: Manage (add, edit, delete) Test Cases contained in the TCS
    • Preview: Output a print- and web-friendly page of the whole TCS.
  • Find a TCS:
    • a list of all existing TCS filtered by fixed criteria.

Create Test Case Specification

General Design and Content

Following the 3 mentioned categories (properties, test cases and preview), the creation/edition of a TCS is displayed into 3 tabs corresponding to those categories.

A first step, when clicking on "Create Test Case Specification", shows a single Tab [Properties].

The [Properties] tab.

This tab contains 3 controls:

  • Version (listbox | mandatory): defines when the enhancement/feature has been first implemented.
  • Title (text field | mandatory): the name of the TCS [can include quite every characters, spaces special characters]
  • [N]ext (Button): saves the creation of the previous fields into the database and leads to the next steps.
    The access key for this button is Alt+N as noted "[N]" in the caption.

In a seconds step, 3 fields are added below the previous ones and complete the general properties of the TCS:

  • Application (listbox | mandatory): The user selects the OOo component the TCS will be part of.
    The entries correspond to the 10 automated OOo applications.
    [Thinking about a 11th entry called "Other" to include 1 time use non yet defined applications...].
  • Category (listbox | mandatory): The user selects how often the TCS should be run accross the milestones. The five automation categories plus a "never run" category.
  • Motivation (Listbox | mandatory): This field indicate what motivated the creation of the TCS. The listbox content 4 entries:
    • Select Cause (default): has no effect but to make the choice mandatory.
    • Specification: has to be selected if a written specification exists. This is the normal case.
    • Issue: has to be selected if no specification but one ore more issue ID(s) exist(s).
      Ex.: the TCS is written to avoid a regression of a fixed bug or
      a small enhancement has been implemented which didn't require a specification.
      Note: in most cases both specification AND issue ID - which is mandatory to motivate a specification - exist. The user might then want to document both items. Though the form is presented as a an alternative (either specification OR issue ID), it is possible to fill in both items by switching afterward between Specification and Issue ID and vice versa. This method might sound awkward but it was the only way to assure that 1 of those cause is filled (mandatory) without making both mandatory at the same time.
    • Own initiative: has to be selected if the motivation of the TCS is neither due to a specification nor an issue.

      Depending on the selected cause, a form corresponding to this cause appears below those fields:
    • If Specification is selected:
      • URL (text field | mandatory): The URL pointing to the specification.
      • Summary (text field | mandatory): The description has to be short!]</span>(text field | mandatory): A short summary which will be displayed in the list TCS list view.
      • Preconditions (multiline text field | optional): specific conditions needed to run the test (special settings in OOo, platform dependency, external tools, extensions...).
      • CWS Name (text field | optional): if one or more CWS exist at the time the TCS is beeing written.
    • Comment on changes (text field | mandatory): Each changes to properties tab page on an existing testcase specification must have a short comment about the changes made before saving. This is listed on history tabpage.
    • [S]ave (button): Will be use to save the defined properties.
      The access key for this button is Alt+S as noted "[S]" in the caption.
      Note: The former version of the tool had 2 other fields which we removed for they where useless:
        • Specification version: was good minded but away from the reality. Frankly said, though specs have versions, nobody cares about it or sets a correct version.
          "New versions" are simply updates in the spec document or new specs completing or replacing the former spec.
          Furthermore a spec versioning without versioning system does not make much sense...
          -> Field removed.
        • Specification part: also a good idea without practical application. It is very rare to only write a TCS for a part of a spec only. If then, the Preconditions field can be used to notice:"Further test cases will be added when implemented. Planned for version X.x..."
          Furthermore, the field consisted in a list box showing "1" to "10" while "parts" in a spec have a sub-leveled ("1.1.3", "2.3") structure which can have much more than 10 parts!
          -> Field removed
  • If Issue is selected:
    • Issue ID (text field | mandatory): one or more issue ID.
      The other fields are the same as for Specification...
  • If Own initiative is selected:
    The same common fields as for Specification and Issue ID (Description, Preconditions and CWS name).

  • A click on [S]ave, saves the defined properties, reloads the page and displays 2 further tabs: [Test Cases] and [Preview].

    The [Test Cases] tab

    The first time a TCS is created this tab page displays an invitation to create a test case:
    No test cases found for this TCS. Add test case by clicking the 'New' button

    As soon as test cases exist, they are displayed in this view of the tab. It is the so called list view.

    By clicking on the [N]ew button (which access key is Alt+N as noted "[N]" in the caption), a test case edit form appears in the tab.

    This is the edit view.

    The Edit View

    This form contains following fields and controls:

    • Name (edit text | mandatory): the user defines there a name for the test case.
    Tip: as the vocation of TCS is to get automated, it is a good practice to name the test cases using the test script name rules: a short description of what the test case tests does, all words without spaces and CamelCase like and preceded by a small "t".
    Example: tInsertNewFrame
    • Status (listbox | mandatory): gives a statement about the finalization and the actuality of the test case
      • Select a Status (default): has no effect but to make the choice mandatory.
      • Preliminary ("draft"?): when the test case is not finished yet.
      • Standard (complete?): when the test case is final.
      • Obsolete: when the test case is deprecated but is kept as archive or reference.Should checkbox 'published' be unchecked if status is 'Obsolete' ?
    • Published (checkbox | optional | Default: checked): A testcase is only shown in testcase specification 'preview'-page if a testcase is set to published. Otherwise it is only visible in list on tabpage 'Testcases'
    • Description ( multiline edit text | optional): This is the content of the test case. The control allows plain text only.
    Plain text only: for export reasons (to Wiki, HTML...) and with regard to a certain level of standardization, the test case description offers no formatting (bullets, font attributes, colors...) possibility..
    • Upload Test Document:
      • Two buttons are visible: Browse and [U]pload.
      • First the browse button must be pressed and then a file from file system selected.
      • Afterwards the [U]pload button must be pressed to upload the file and attach it to the testcase.
      • Once uploaded it will be listed underneath the Bugdoc section. There is a 2MB file-size restriction, number of files to be attached to a testcase is not regulated.
      • Once a bugdoc is listed it has 2 buttons beneath the name:
        • [D]ownload: This button is to have a possibility downloading or viewing the file
        • [R]emove: This button is to remove the bugdoc from testcase
    • Comment on changes (text field | mandatory): Each changes on existing testcases must have a short comment about the changes made before saving. This is listed on history tabpage.
    • [S]ave (button): clicking on save saves the content of the form to the database.
      The page doesn't reload and remains in the edit view.
      Proposal: the background of the form could be light colorized as soon as something is changed into the form. This would indicate an "usaved" state to the user. As soon as Save is pressed, the background would turn back to white and a message could show displaying: "Data successfully saved".
    • [B]ack (button): used to go back to the list view.
      Two different possibilities:
      • The form has not been modified: the view goes back to the list view.
      • The form has been modified and not saved: a message box pops up asking
        "Testcase has been modified. Do you really want to go back ?[Ok][Cancel]".
        • Ok: Go back to list view without saving changes
        • Cancel: leave in testcase edit mode
    The List View

    This view of the tab displays a table in which the test cases are listed. The table is made of 5 columns and as many rows as test cases exists. Those columns have following headers and contents:

    • Select to Delete\n([ ] All) (text and checkbox): (un)selecting the checkbox (un)selects automatically all the checkboxes contained in the column.
      The [D]elete control (see below) then (dis)appears at the bottom of the table to allow the deletion of the selected rows.
      Content: this columns contains at each row a checkbox allowing to select each test case separatly for deletion.
    • Num. (text): meaning the chronological creation number of the test case.
      Content: this columns contains at each row a chronological creation number of the test case. Formatted as a hyperlink which resolves the corresponding test case in edit view.
    • Name (text): meaning the name of the test case.
      Content: this columns contains at each row a the name of the test case. Formatted as an hyperlink which resolves the corresponding test case in edit view.
    • Status (text): meaning the status of the test case.
      Content: This status can be Preliminary, Standard or Obsolete.
    • Published (text): Yes/No.
      • 'Yes' means the testcase is shown in testcase specification preview and testcase specification document
      • 'No' means the testcase is *not' shown in testcase specification preview and testcase specification document.

    At the bottom of the table, 2 controls:

    • [N]ew (button): already defined above
    • [D]elete (button | default: hidden):
      this button is hidden as long as no check box is checked in the column Select to Delete\n([ ] All).
      clicking on this button pops up a message box asking
      "Do really want to delete the selected test case(s)?.
      • Delete: delete the selected test case(s) and goes back to the list view.
      • Cancel: stays in the list view. Nothing is deleted.

    The [Preview] tab

    List Test Case Specification

    Look and Feel Guidline

    The general design (colors, icons...) stays in keeping with the corporate designs:

    • SUN corporate design internally
    • OOo Web site design on OOo.

    Specific Guidlines:

    • Mandatory fields:
      • Default: bold and black formatted. Show an asterisk ("*") at the end of their labels.
        Sample: Mandatory Field*:
      • On Error (fields empty when the form is sent):
        • Background (of the table row): CSS name "yellow"
        • Font color: CSS name "red"
        • Attributes: Bold + Italic.
          Sample: Mandatory Field*:
    • Optional fields and labels: regular sans serif.

    Help | User Interface Element Templates | Example Spec


    Bugs & Features

    1. Feature: Implement mouse hover context help on every controls.
    2. Feature: Add a component 'All' to have the opportunity to list all components at time
    3. Feature: Missing description for required fields (*)
    4. Bug: Test Case specification title is not shown in preview: 02/17/2010 -> Fixed
    5. Bug: FIX incorrectly imported HTML text in test cases description: 02/17/2010 -> Fixed. Formattings like e.g. indents must be adapted by tcs owner
    6. Feature: Add possibility to link exiting bugdocs to more than 1 test case


    Bugs & Features

    # Add a "My TCS" link on the right navigation bar on the QUASTe main page (between "CWS create" and "Add test case specification").
    Due to too much amount of implementation details a checkbox 'Show my test case specifications only' was added to the form.

    - Same view as the TC overview.
    - 2 lists: the TCs I am Owner of + the TCs I am NOT owner of but made a change in.
    Currently only all test case specifications and testcase specifications I own (optional) are shown.
    - 2 columns containg as link "Edit" and "Delete" (Delete appears only when the user is Owner of the TCS)

    # dynamic 1 step ("Application" -> OOo) or 2 step ("Product"/"Application" -> internal) Overview wizard

    1. [internal] Remember with a cookie what product the user chose and preselect this product in the listbox in every further sessions.

    # Overview columns:
    ## "#" (query output order NOT record ID)
    ## [Internal] Product (yes/no)
    ## ApplicationQUESTION: For what reason 'Application' should be listed here. It is shown in header of overview !?</STRIKE>
    ## TCS name (hyperlink to TCS, mouse hover containing the summary - please change the name of the the floating box accordingly from "Description" to "Summary")
    ## Owner
    ## Published (yes/no)
    ## [Internal] Confidential (yes/no)
    # Remove the "Product" sub-header
    # Toggling column header to "sort by"
    # After clicking on Overview on the QUASTe main page, the header shows "Test case specification"
    -> Should be "Test case specification overview".

    TC EDIT - General # Apply defined Look&Feel:
    ## Mandatory fields = bold with an asterisk*
    ## Mandatory fields not filled while submitting = bold, red, italic, with an asterisk and a yellow cell background*
    ## Remove ANY table borders</br>

    TC EDIT - Tabpage: Properties
    # BUG:
    - choose "Add Testcase Specification"
    - enter a Title name which already exists
    - submit form
    -> instead of an "Alert" MsgBox, the field label "Title" should be bold,red,italic,bg yellow and
    a highlighted message "Test Case Specification already exists.\n Please choose a different Version or Title" (<- beware of the <STRIKE>wording!)

    TC EDIT - Tabpage: Testcases
    # Make Changes comments Optional!!!
    # See 1. -> REALLY!! ;)
    # set the focus in the first field (listbox Application)<STRIKE>
    <STRIKE># [Internal] Confidential: default must be "checked"

    # Saving a TC: the info string "test case specification was saved" should be placed below the form (left aligned below the "[S]ave" javascript link) and highlighted (any bright color) or completed with a "tick mark" icon (a green "V", you know...)
    This doesn' t work because it is Joomla default that can't be changed easily

    P1 - [S]ave button doesn't work
    - add a new TC to a TCS
    - fill all the fields
    - click [S]ave
    -> nothing happens
    Expected: a message and a color change show the user that something as be saved.
    Nothing has been saved: clicking on [B]ack will warn for data loss.
    But "something" happens: clicking on [S]ave when mandatory fields are not fields correctly highlights those fields. not reprducable by me (HDE)
    Still reproducible (ES)

    <b>2009-12-17: This was fixed. Due to a hidden checkbox that was missing in Javascript code the testcase wasn't saved (HDE)

    • # New saved TCS won't show in the Overview (won't be saved?)

    - create a new TCS
    - save it
    - call the Overview
    - select the corresponding Application
    -> New created TCS is not listed (you will only notice that "something has changed" because a new OOo version is listed as header if you have applied your TCS to an other version than OOo 3.2)
    2009-12-15: This has been fixed. Reason: TCS was saved but not shown if it had no testcases.

    • weird accesskeys on js-buttons: should be [N]ew [B]ack [D]elete [C]ancel.
    • in the first form (Product/Title), the Next button should be placed at the left side of the form.
    • Many fields are not saved:
      • Description in test case edit view
      • Specification URL, Issue IS and the Description of the TCS when switching between Spec/Issue ID after having saved
    • Wording (to be discussed):
      • In first form there's "Product Name" and "Title". After having defined the fields, it is called "Valid for version" and "Name" in the next page -> inconsistency.
        I advocate for "Product" and "Title" everywhere.
      • Category: though this corresponds to what we call CAT0, CAT1... putting together "category" and "run every 2nd milestone" is not intuitive.
        Proposal: change caption to "Test Run Frequency".
      • Cause: doesn't sound good. AFAIK has cause most of the time a negative meaning. "Motivation" would be better.
      • Description (TCS Properties): use "Summary" instead.
    • On Page 2 of the properties: show both Product and Title in the 2nd Table column header.


    Accessibility is the responsibility of the I-Team, beginning with UX, DEV and QA, to ensure that the following requirements are fulfilled:

    1. Is the feature fully keyboard accessible?
      (Ex: "I can go everywhere and use every function using the keyboard only"

    2. Have I specified visual alternatives for the case that the specified feature includes audio as output?
    3. Are text alternatives for all icons and graphics available?
      <Start typing here>
    4. Don't provide important information in colors alone
      (Ex: marking important information hard coded in red)

    5. Does the specified feature respect system settings for font, size, and color for all windows and user interface elements?
    6. Have I ensured that flash rates do not exceed 2 hertz for blinking text, objects, or other elements? In any case, try to avoid flashing UI elements
    7. Ensure that assistive technology (AT) (like ZoomText or Orca) is able to read everything.


    If you need help while designing, implementing or testing the accessibility of the UI, ask/visit:

    1. The accessibility check list at the OpenOffice.org Wiki
    2. accessibility@ui.openoffice.org (The accessibility mailing lists, preferred)
    3. For specific implementation details, architecture: mt@openoffice.org (Malte Timmermann)
    4. For specific UX and testing questions: es@openoffice.org (Éric Savary)


    <START TYPING HERE --- If this part is irrelevant state a reason for its absence.>


    <START TYPING HERE --- If this part is irrelevant state a reason for its absence.>

    Help | Configuration Table Template

    File Format

    <START TYPING HERE --- If this part is irrelevant state a reason for its absence.> Help

    Help | File Format Table Template

    Open Issues

    <State a bulleted list of issues Issue here> Bold text

    Personal tools