Difference between revisions of "User:Foral/automation/testcase"

From Apache OpenOffice Wiki
Jump to: navigation, search
Line 210: Line 210:
 
==step 4:See the Testcase==
 
==step 4:See the Testcase==
  
where is the real testcase in script.
+
===search the testscript which have a error===
  
 
<pre>
 
<pre>
Line 218: Line 218:
 
</pre>
 
</pre>
  
get test case . actually this is depend on person who make a testscript but normally you can you this command.
+
===get test case from script===
 +
sometimes the style of writing is deference as this sample.
 +
it depend on person who write testscript.
  
 
<pre>
 
<pre>

Revision as of 08:49, 13 August 2009

After automation test

After automation test.we should fill out what kind of error and warnings ...etc.

  • step 1:Make Summary
  • step 2:Test again one by one
  • step 3:See the Log
  • step 4:See the Testcase
  • step 5:Do it by your hand and filling out the reason.

Step 1:Make Summary

To get sammary,you can use tools/analyze/make_summary.pl and make a summary of test result. and you can explore the test result.

this sample from 3.1.1rc1 release testing. i tested on Fedora 11 with OOo3.1.1rc1 Linux without JRE version.

bash-4.0$ ./make_summary.pl ../../errorlogs/*.res

Filename                      Errors  Warnings  Warnings_during_initialization
============================  ======  ========  ==============================
c_import_general.res               0         0         0
c_updt1.res                        1         1         0
c_updt2.res                        0         0         0
c_xml_scenario.res                 0         0         0
ch2_flexible_source_range_se       2         2         0
ch2_losa_ods_to_ods.res            0         0         0
ch2_losa_ods_to_xls.res            0         0         0
ch2_losa_sxc_to_ods.res            0         0         0
ch2_losa_sxc_to_sdc.res            0         0         0
ch2_lvl1_wizard.res                0         0         0
ch2_updt_calc.res                  1         1         0
ch_xml_japanese_candlestick.       0         0         0
closeoffice.res                    1         0         0
d_export_graphic_a.res             0         0         0
d_export_graphic_b.res             0         0         0
d_opt.res                          0         0         0
d_updt.res                         0         0         0
dba_Main.res                       0         0         0
dba_ReportBuilder.res              0         0         0
dba_frm_FormFilter.res             0         0         0
dba_frm_Forms.res                  1         3         0
dba_misc_RegisterDatabase.re       0         0         0
dba_rpt_DateTime.res               0         0         0
dba_rpt_ExecuteReport.res          0         0         0
dba_rpt_Formating.res              0         0         0
dba_rpt_Grouping.res               0         0         0
dba_rpt_PageNumber.res             0         1         0
dba_rpt_PropertyBrowser.res        0         0         0
dba_xf_Submission.res              0         0         0
dba_xforms_Main.res                0         0         0
e_fileopen.res                     0         0         0
e_location.res                     0         0         0
e_options.res                      0         0         0
e_update.res                       0         0         0
f_basic_dialogs.res                0         0         0
f_basic_issues.res                 0         0         0
f_basic_protected_libraries.       0         0         0
f_basic_shared_modules.res         0         0         0
f_configuration.res                0         2         0
f_extras_labels.res                0         0         0
f_extras_tableautoformat.res       0         0         0
f_extras_templatepreview.res       0         0         0
f_filedlg_cjk_files.res            0         0         0
f_filedlg_reserved_names_uni       0         0         0
f_first.res                        0         0         0
f_help_browser.res                 0         0         0
f_help_compare_applications.       0         0         0
f_help_search.res                  0         0         0
f_printer_administration.res       0         0         0
f_script_organizers.res            0         0         0
f_security_certification_dia       0         0         0
f_security_dialogs.res             0         0         0
f_security_incorrect_passwor       0         0         0
f_security_trusted_path.res        0         0         0
f_standard_toolbar.res             0         0         0
f_tools_customize.res              0         0         0
f_tools_options.res                0         0         0
f_topten.res                       0         0         0

Filename                      Errors  Warnings  Warnings_during_initialization
============================  ======  ========  ==============================
f_window_functions.res             0         0         0
f_wizards.res                      1         1         0
g_autocorrection.res               0         0         0
g_character.res                    0         4         0
g_customshapes.res                 0         0         0
g_export_html.res                  0         0         0
g_group.res                        0         0         0
g_imagemap.res                     0         0         0
g_line.res                         0         0         0
g_mediaplayer.res                  0         0         0
g_ole.res                          0         0         0
g_paragraph.res                    0         0         0
g_print.res                        0         0         0
g_shaddow.res                      0         0         0
g_tools.res                        0         0         0
g_zoom.res                         0         0         0
i_opt.res                          0         0         0
i_slidecopy.res                    0         0         0
i_updt_1.res                       0         0         0
i_updt_2.res                       0         0         0
m_lvl1.res                         0         0         0
m_updt.res                         0         0         0
oasis_graphics_update.res          0         0         0
resetoffice.res                    0         0         0
w_numbering.res                    0         5         0
w_section.res                      2        16         0
w_shortcuts.res                    0         0         0
w_sorting.res                      0         0         0
w_spellcheck.res                   0         5         0
w_updt.res                         0         1         0
writer_xml_7_export.res            0         0         0
==============================================================================
Total 89 files                     9        42         0

step 2:Test again one by one

sometimes error/warning come from just timing or setting so just test again one by one and some of error/warning might be gone.especially you should test again one by one when you get a lot of error/warning.

  • c_updt1.res
    • error has gone.
    • it was timing error.(navigater Rendering is too slow)
  • ch2_flexible_source_range_se
    • error has gone.
    • it was timing error.(navigater Rendering is too slow)
  • ch2_updt_calc.res
    • error has gone.
    • it was timing error.(navigater Rendering is too slow)
  • dba_frm_Forms.res
    • error has gone.
    • it was timing error.
  • f_wizards.res
    • error has gone.
    • it was timing error.
  • w_section.res
    • error has gone.
    • it was timing error.

step 3:See the Log

you know which test script made error and warnings. so you should find out the log from result file with tools/analyze/translater_res_file.pl or use testtool.

bash-4.0$ ./translate_res_file.pl ../../errorlogs/ch2_flexible_source_range_selection.res |grep ^ERROR -B 5
MESSAGE: Close document
TEST_CASE: 
TEST_CASE: tDataRangeDialogWriter
MESSAGE: Open test document gTestToolPath/chart2/required/input/ch_flexrange07.odt
MESSAGE: Select chart using navigator
ERROR: Auswahlliste : Entry number 5 is too large in Select. Max. allowed is 0
--
WARNING: BORDERWINDOW "無題 1 - OpenOffice.org Writer TTD :localhost.localdomain[12479]" geschlossen, RType = BORDERWINDOW, UId = Active
TEST_CASE: 
TEST_CASE: tDataSeriesDialogWriter
MESSAGE: Open test document gTestToolPath/chart2/required/input/ch_flexrange08.odt
MESSAGE: Select chart using navigator
ERROR: Auswahlliste : Entry number 5 is too large in Select. Max. allowed is 0

this log show us

  • testcase [tDataRangeDialogWriter] had a error
  1. Open test document gTestToolPath/chart2/required/input/ch_flexrange07.odt
    1. Success
  2. Select chart using navigator
    1. Fail:this happen may be come up on the Navigator or near and error message help you to get true:p
  • testcase [tDataSeriesDialogWriter] had a error
  1. Open test document gTestToolPath/chart2/required/input/ch_flexrange08.odt
    1. Success
  2. Select chart using navigator
    1. Fail:this happen may be come up on the Navigator or near and error message help you to get true:p


source of this log

bash-4.0$ cat ch2_flexible_source_range_selection.res |grep "tDataRangeDialogWriter\|tDataSeriesDialogWriter" -A 5
1;~chart2\required\includes\ch2_flexible_source_range_selection01.inc;438;27;61;"tDataRangeDialogWriter"
4;~chart2\required\includes\ch2_flexible_source_range_selection01.inc;443;4;65535;"Open test document gTestToolPath/chart2/required/input/ch_flexrange07.odt"
4;~chart2\tools\ch_tools_common.inc;163;4;65535;"Select chart using navigator"
2;~global\tools\includes\required\t_doc2.inc;294;48;65535;"Auswahlliste : %ResId=22021%%Arg1=Select%%Arg2=5%%Arg3=0%"
3;~global\tools\includes\required\t_doc2.inc;294;48;65535;"--/home/goodday/OOO310_m18/testautomation/global/tools/includes/required/t_doc2.inc: fSelectFirstOLE%()"
3;~chart2\tools\ch_tools_common.inc;164;4;65535;"--/home/goodday/OOO310_m18/testautomation/chart2/tools/ch_tools_common.inc: fOpenTestDocumentAndSelectDataRange(Byref sInputFile$)"
3;~chart2\required\includes\ch2_flexible_source_range_selection01.inc;294;48;65535;"--/home/goodday/OOO310_m18/testautomation/chart2/required/includes/ch2_flexible_source_range_selection01.inc: tDataRangeDialogWriter()"
3;~chart2\required\ch2_flexible_source_range_selection.bas;58;12;65535;"/home/goodday/OOO310_m18/testautomation/chart2/required/ch2_flexible_source_range_selection.bas: main()"
4;~global\system\includes\master.inc;561;12;65535;"   **    Error in TestExit -Routine      **"
5;~global\system\includes\master.inc;562;12;65535;"DOCKINGWINDOW "ナビゲータ" geschlossen, RType = DOCKINGWINDOW, UId = ViewNavigator"
5;~global\system\includes\master.inc;562;12;65535;"BORDERWINDOW "無題 1 - OpenOffice.org Writer TTD :localhost.localdomain[12479]" geschlossen, RType = BORDERWINDOW, UId = Active"
1;~chart2\required\includes\ch2_flexible_source_range_selection01.inc;596;222;234;""
1;~chart2\required\includes\ch2_flexible_source_range_selection01.inc;600;28;63;"tDataSeriesDialogWriter"
4;~chart2\required\includes\ch2_flexible_source_range_selection01.inc;609;4;65535;"Open test document gTestToolPath/chart2/required/input/ch_flexrange08.odt"
4;~chart2\tools\ch_tools_common.inc;163;4;65535;"Select chart using navigator"
2;~global\tools\includes\required\t_doc2.inc;294;48;65535;"Auswahlliste : %ResId=22021%%Arg1=Select%%Arg2=5%%Arg3=0%"
3;~global\tools\includes\required\t_doc2.inc;294;48;65535;"--/home/goodday/OOO310_m18/testautomation/global/tools/includes/required/t_doc2.inc: fSelectFirstOLE%()"
3;~chart2\tools\ch_tools_common.inc;164;4;65535;"--/home/goodday/OOO310_m18/testautomation/chart2/tools/ch_tools_common.inc: fOpenTestDocumentAndSelectDataRange(Byref sInputFile$)"
3;~chart2\required\includes\ch2_flexible_source_range_selection01.inc;294;48;65535;"--/home/goodday/OOO310_m18/testautomation/chart2/required/includes/ch2_flexible_source_range_selection01.inc: tDataSeriesDialogWriter()"
3;~chart2\required\ch2_flexible_source_range_selection.bas;59;12;65535;"/home/goodday/OOO310_m18/testautomation/chart2/required/ch2_flexible_source_range_selection.bas: main()"
4;~global\system\includes\master.inc;561;12;65535;"   **    Error in TestExit -Routine      **"
5;~global\system\includes\master.inc;562;12;65535;"DOCKINGWINDOW "ナビゲータ" geschlossen, RType = DOCKINGWINDOW, UId = ViewNavigator"
5;~global\system\includes\master.inc;562;12;65535;"BORDERWINDOW "無題 1 - OpenOffice.org Writer TTD :localhost.localdomain[12479]" geschlossen, RType = BORDERWINDOW, UId = Active"
1;~chart2\required\includes\ch2_flexible_source_range_selection01.inc;784;222;234;""

step 4:See the Testcase

search the testscript which have a error

bash-4.0$grep -f *.inc |grep tDataRangeWizardWriter
ch2_flexible_source_range_selection01.inc:' #1 tDataRangeWizardWriter                 ' Updatetest for tabpage DataRange via chart wizard
ch2_flexible_source_range_selection01.inc:testcase tDataRangeWizardWriter

get test case from script

sometimes the style of writing is deference as this sample. it depend on person who write testscript.

bash-4.0$cat ch2_flexible_source_range_selection01.inc |grep "testcase*\|'"
testcase tDataRangeWizardWriter
    '///Open test document <i>gTestToolPath</i>/chart2/required/input/ch_flexrange05.odt
        '///Choose the Chart type <i>line</i> (should be the fifth in the top-down list)
        '/// Click Next button
        '///Check data range in edit field value (ChartFlexRange.A1:E5)
        '///Check if -Data series in rows- is enabled
        '///Check if -Data series in columns- is enabled and chosen
        '///Check if -First row as label- is enabled and checked
        '///Check if -First column as label- is enabled checked
        '///Enter invalid range ChartFlexRange.A1:E
        '///Check if -Data series in rows- is disabled
        '///Check if -Data series in columns- is disabled
        '///Check if -First row as label- is disabled
        '///Check if -First column as label- is disabled
        '///Check that Back, Next and Finish does not work
        '///Enter valid range ChartFlexRange.A1:E5
        '///Check if -Data series in rows- is enabled
        '///Check if -Data series in columns- has been enabled and chosen
        '///Check if -First row as label- is enabled and checked
        '///Check if -First column as label- is enabled checked
        '///Press Cancel button
        '///Close document
'
'-------------------------------------------------------------------------
'
testcase tDataSeriesWizardWriter
    '///Open test document gTestToolPath/chart2/required/input/ch_flexrange06.odt
        '///Choose the Chart type <i>area</i> (should be the fourth in the top-down list)
        '/// Click Next button
        '/// Click Next button
        '///Check if the data series listbox is enabled and contains four entries
        '///Check if the ADD button is enabled and click the button
        '///Check if there are now five entries in the data series listbox
        '///Check if the new data series was added below the highlighted one
        '///Check if the REMOVE button is enabled and click the button
        '///Check if there are now four entries in the data series listbox
        '/// Select third entry
            '/// The Move Up button should be enabled
            '/// Click the Move Up button twice
        '/// Select second entry
            '/// If Move Down button is enabled
            '/// Click Move Down button twice
        '///Select first entry in Data Ranges
        '///Verify that the range matches the one in edit field
        'GetSelText(line,column) is a new parameter since CWS:SRC680gh13
        'Instead of 1,2 there must be 1,4 used becqause there are two invisible
        'label-columns in that treelist-box        
        '///Select second entry in Data Ranges
        '///Check that the range in the edit field has changed, too
        'GetSelText(line,column) is a new parameter since CWS:SRC680gh13
        'Instead of 1,2 there must be 1,4 used becqause there are two invisible
        'label-columns in that treelist-box        
        '///Change the range in edit field 
        '///Check that the range in Data Ranges has changed, too
        'GetSelText(line,column) is a new parameter since CWS:SRC680gh13
        'Instead of 1,2 there must be 1,4 used becqause there are two invisible
        'label-columns in that treelist-box        
        '/// Set ChartFlexRange.A2:A4 as range for categories
        '///Enter invalid range ChartFlexRange.A2:A
        '///Check that the BACK button is disabled
        '///Enter valid range ChartFlexRange.A2:A5
        '///Press Cancel button
        '///Close document
'
'-------------------------------------------------------------------------
'
testcase tDataRangeDialogWriter
    '///Open test document <i>gTestToolPath</i>/chart2/required/input/ch_flexrange07.odt
        '///Check data range in edit field (ChartFlexRange.A2:A3;ChartFlexRange.B1:E3)
        '///Check if -Data series in columns- is enabled and chosen
        '///Check if -Data series in rows- is enabled
        '///Check if -First row as label- is enabled and checked
        '///Check if -First column as label- is enabled and checked
        '///Enter invalid range ChartFlexRange.A1:E
        '///Check that -Data series in rows- is disabled
        '///Check that -Data series in columns- is disabled
        '///Check if -First row as label- is disabled
        '///Check if -First column as label- is disabled
        '///Check that OK button is disabled
        '///Enter valid range ChartFlexRange.A1:E3
        '///Check if -Data series in columns- is enabled and chosen
        '///Check if -Data series in rows- is enabled
        '///Check if -First row as label- is enabled and checked
        '///Check if -First column as label- is enabled and checked
        '///Press Cancel button
        '///Close document
'
'-------------------------------------------------------------------------
'
testcase tDataSeriesDialogWriter
    '/// Open test document <i>gTestToolPath</i>/chart2/required/input/ch_flexrange08.odt
        '/// Switching to tab page Data Series
        '///Check if the data series listbox is enabled and contains four entries
        '///Check if the ADD button is enabled and click the button
        '///Check if there are now five entries in the data series listbox
        '///Check if the new data series was added below the highlighted one
        '///Check if the REMOVE button is enabled and click the button
        '///Check if there are now four entries in the data series listbox
        '///Use UP button and verify that it it disabled on first entry
        '///Use DOWN button thrice and verify that it is disabled on last entry
        '///Select first entry in Data Ranges
        '///Verify that the range matches the one in edit field
        'GetSelText(line,column) is a new parameter since CWS:SRC680gh13
        'Instead of 1,2 there must be 1,4 used becqause there are two invisible
        'label-columns in that treelist-box        
        '///Select second entry in Data Ranges
        '///Check that the range in the edit field has changed,too
        'GetSelText(line,column) is a new parameter since CWS:SRC680gh13
        'Instead of 1,2 there must be 1,4 used becqause there are two invisible
        'label-columns in that treelist-box        
        '///Change the range in edit field to ChartFlexRange.B2:B3
        '///Check that the range in Data Ranges has changed, too
        'GetSelText(line,column) is a new parameter since CWS:SRC680gh13
        'Instead of 1,2 there must be 1,4 used becqause there are two invisible
        'label-columns in that treelist-box        
        '///Edit range for Categories (ChartFlexRange.A2:A3)
        '///Enter invalid range ChartFlexRange.A2:A
        '///Check that the OK button is disabled
        '///Enter valid range ChartFlexRange.A2:A4
        '///Press Cancel button
        '///Close document
'
'-------------------------------------------------------------------------
'
    '///Menu Insert Object Chart to open chart wizard



step 5:Do it by your hand and filling out the reason

try this testcase by my hand. sometimes you need to see the testscript. because this comment of testcase does not have detail.

testcase tDataRangeDialogWriter
    '///Open test document <i>gTestToolPath</i>/chart2/required/input/ch_flexrange07.odt
        '///Check data range in edit field (ChartFlexRange.A2:A3;ChartFlexRange.B1:E3)
        '///Check if -Data series in columns- is enabled and chosen
        '///Check if -Data series in rows- is enabled
        '///Check if -First row as label- is enabled and checked
        '///Check if -First column as label- is enabled and checked
        '///Enter invalid range ChartFlexRange.A1:E
        '///Check that -Data series in rows- is disabled
        '///Check that -Data series in columns- is disabled
        '///Check if -First row as label- is disabled
        '///Check if -First column as label- is disabled
        '///Check that OK button is disabled
        '///Enter valid range ChartFlexRange.A1:E3
        '///Check if -Data series in columns- is enabled and chosen
        '///Check if -Data series in rows- is enabled
        '///Check if -First row as label- is enabled and checked
        '///Check if -First column as label- is enabled and checked
        '///Press Cancel button
        '///Close document
'
'-------------------------------------------------------------------------

knowledge

ERROR: Auswahlliste : Entry number 5 is too large in Select. Max. allowed is 0

normally this error is just tools problem. some time rendering the navigater is too slow so it can not sucssess.

MESSAGE: Select chart using navigator
ERROR: Auswahlliste : Entry number 5 is too large in Select. Max. allowed is 0
  • Solution 1:Just try again this testscript.
  • Solution 2:open the navigater before starting the test.
  • Solution 3:Do the testcase by your hand

Glossary

  • Auswahlliste(German)
    • shortlist(English)
    • 選択リスト(Japanese)
Personal tools