Difference between revisions of "User:Foral/automation/testcase"
From Apache OpenOffice Wiki
(No difference)
| |
Revision as of 07:59, 14 August 2009
After automation test
After automation test.you should find out the reason why you get error or warnings. you may find out the reason from log if you only get warning(that time just do step 3) but you have to do some if you get error and sometime you have to retest it which you had error.
- step 1:Make Summary
- step 2:Test again one by one
- step 3:See the Log
- step 4-A:See the Testcase and Do it by your hand and filling out the reason.
- step 4-B:Debag with TestTool
Step 1:Make Summary
To get summary,you can use tools/analyze/make_summary.pl and make a summary of test result. and you can explore the test result.
this sample from 3.1.1rc1 release testing. i tested on Fedora 11 with OOo3.1.1rc1 Linux without JRE version.
bash-4.0$ ./make_summary.pl ../../errorlogs/*.res Filename Errors Warnings Warnings_during_initialization ============================ ====== ======== ============================== c_import_general.res 0 0 0 c_updt1.res 1 1 0 c_updt2.res 0 0 0 c_xml_scenario.res 0 0 0 ch2_flexible_source_range_se 2 2 0 ch2_losa_ods_to_ods.res 0 0 0 ch2_losa_ods_to_xls.res 0 0 0 ch2_losa_sxc_to_ods.res 0 0 0 ch2_losa_sxc_to_sdc.res 0 0 0 ch2_lvl1_wizard.res 0 0 0 ch2_updt_calc.res 1 1 0 ch_xml_japanese_candlestick. 0 0 0 closeoffice.res 1 0 0 d_export_graphic_a.res 0 0 0 d_export_graphic_b.res 0 0 0 d_opt.res 0 0 0 d_updt.res 0 0 0 dba_Main.res 0 0 0 dba_ReportBuilder.res 0 0 0 dba_frm_FormFilter.res 0 0 0 dba_frm_Forms.res 1 3 0 dba_misc_RegisterDatabase.re 0 0 0 dba_rpt_DateTime.res 0 0 0 dba_rpt_ExecuteReport.res 0 0 0 dba_rpt_Formating.res 0 0 0 dba_rpt_Grouping.res 0 0 0 dba_rpt_PageNumber.res 0 1 0 dba_rpt_PropertyBrowser.res 0 0 0 dba_xf_Submission.res 0 0 0 dba_xforms_Main.res 0 0 0 e_fileopen.res 0 0 0 e_location.res 0 0 0 e_options.res 0 0 0 e_update.res 0 0 0 f_basic_dialogs.res 0 0 0 f_basic_issues.res 0 0 0 f_basic_protected_libraries. 0 0 0 f_basic_shared_modules.res 0 0 0 f_configuration.res 0 2 0 f_extras_labels.res 0 0 0 f_extras_tableautoformat.res 0 0 0 f_extras_templatepreview.res 0 0 0 f_filedlg_cjk_files.res 0 0 0 f_filedlg_reserved_names_uni 0 0 0 f_first.res 0 0 0 f_help_browser.res 0 0 0 f_help_compare_applications. 0 0 0 f_help_search.res 0 0 0 f_printer_administration.res 0 0 0 f_script_organizers.res 0 0 0 f_security_certification_dia 0 0 0 f_security_dialogs.res 0 0 0 f_security_incorrect_passwor 0 0 0 f_security_trusted_path.res 0 0 0 f_standard_toolbar.res 0 0 0 f_tools_customize.res 0 0 0 f_tools_options.res 0 0 0 f_topten.res 0 0 0 Filename Errors Warnings Warnings_during_initialization ============================ ====== ======== ============================== f_window_functions.res 0 0 0 f_wizards.res 1 1 0 g_autocorrection.res 0 0 0 g_character.res 0 4 0 g_customshapes.res 0 0 0 g_export_html.res 0 0 0 g_group.res 0 0 0 g_imagemap.res 0 0 0 g_line.res 0 0 0 g_mediaplayer.res 0 0 0 g_ole.res 0 0 0 g_paragraph.res 0 0 0 g_print.res 0 0 0 g_shaddow.res 0 0 0 g_tools.res 0 0 0 g_zoom.res 0 0 0 i_opt.res 0 0 0 i_slidecopy.res 0 0 0 i_updt_1.res 0 0 0 i_updt_2.res 0 0 0 m_lvl1.res 0 0 0 m_updt.res 0 0 0 oasis_graphics_update.res 0 0 0 resetoffice.res 0 0 0 w_numbering.res 0 5 0 w_section.res 2 16 0 w_shortcuts.res 0 0 0 w_sorting.res 0 0 0 w_spellcheck.res 0 5 0 w_updt.res 0 1 0 writer_xml_7_export.res 0 0 0 ============================================================================== Total 89 files 9 42 0
step 2:Test again one by one
sometimes error/warning come from just timing or setting so just test again one by one and some of error/warning might be gone.
especially you should test again when you get a lot of error/warning.
- c_updt1.res
- error has gone.
- it was timing error.(navigater Rendering is too slow)
- ch2_flexible_source_range_se
- error has gone.
- it was timing error.(navigater Rendering is too slow)
- ch2_updt_calc.res
- error has gone.
- it was timing error.(navigater Rendering is too slow)
- dba_frm_Forms.res
- error has gone.
- it was timing error.
- f_wizards.res
- error has gone.
- it was timing error.
- w_section.res
- error has gone.
- it was timing error.
step 3:See the Log
now you knew which test script made error and warnings. so you should find out the log from result file with tools/analyze/translater_res_file.pl or use testtool.
I recommend you to use testtool because of it make you more easy to trace the error.
bash-4.0$ ./translate_res_file.pl ../../errorlogs/ch2_flexible_source_range_selection.res |grep ^ERROR -B 5 MESSAGE: Close document TEST_CASE: TEST_CASE: tDataRangeDialogWriter MESSAGE: Open test document gTestToolPath/chart2/required/input/ch_flexrange07.odt MESSAGE: Select chart using navigator ERROR: Auswahlliste : Entry number 5 is too large in Select. Max. allowed is 0 -- WARNING: BORDERWINDOW "無題 1 - OpenOffice.org Writer TTD :localhost.localdomain[12479]" geschlossen, RType = BORDERWINDOW, UId = Active TEST_CASE: TEST_CASE: tDataSeriesDialogWriter MESSAGE: Open test document gTestToolPath/chart2/required/input/ch_flexrange08.odt MESSAGE: Select chart using navigator ERROR: Auswahlliste : Entry number 5 is too large in Select. Max. allowed is 0
this log show us
- testcase [tDataRangeDialogWriter] had a error
- Open test document gTestToolPath/chart2/required/input/ch_flexrange07.odt
- Success
- Select chart using navigator
- Fail:this happen may be come up on the Navigator or near and error message help you to get truth.
- testcase [tDataSeriesDialogWriter] had a error
- Open test document gTestToolPath/chart2/required/input/ch_flexrange08.odt
- Success
- Select chart using navigator
- Fail:this happen may be come up on the Navigator or near and error message help you to get truth.
source of this log
bash-4.0$ cat ch2_flexible_source_range_selection.res |grep "tDataRangeDialogWriter\|tDataSeriesDialogWriter" -A 5 1;~chart2\required\includes\ch2_flexible_source_range_selection01.inc;438;27;61;"tDataRangeDialogWriter" 4;~chart2\required\includes\ch2_flexible_source_range_selection01.inc;443;4;65535;"Open test document gTestToolPath/chart2/required/input/ch_flexrange07.odt" 4;~chart2\tools\ch_tools_common.inc;163;4;65535;"Select chart using navigator" 2;~global\tools\includes\required\t_doc2.inc;294;48;65535;"Auswahlliste : %ResId=22021%%Arg1=Select%%Arg2=5%%Arg3=0%" 3;~global\tools\includes\required\t_doc2.inc;294;48;65535;"--/home/goodday/OOO310_m18/testautomation/global/tools/includes/required/t_doc2.inc: fSelectFirstOLE%()" 3;~chart2\tools\ch_tools_common.inc;164;4;65535;"--/home/goodday/OOO310_m18/testautomation/chart2/tools/ch_tools_common.inc: fOpenTestDocumentAndSelectDataRange(Byref sInputFile$)" 3;~chart2\required\includes\ch2_flexible_source_range_selection01.inc;294;48;65535;"--/home/goodday/OOO310_m18/testautomation/chart2/required/includes/ch2_flexible_source_range_selection01.inc: tDataRangeDialogWriter()" 3;~chart2\required\ch2_flexible_source_range_selection.bas;58;12;65535;"/home/goodday/OOO310_m18/testautomation/chart2/required/ch2_flexible_source_range_selection.bas: main()" 4;~global\system\includes\master.inc;561;12;65535;" ** Error in TestExit -Routine **" 5;~global\system\includes\master.inc;562;12;65535;"DOCKINGWINDOW "ナビゲータ" geschlossen, RType = DOCKINGWINDOW, UId = ViewNavigator" 5;~global\system\includes\master.inc;562;12;65535;"BORDERWINDOW "無題 1 - OpenOffice.org Writer TTD :localhost.localdomain[12479]" geschlossen, RType = BORDERWINDOW, UId = Active" 1;~chart2\required\includes\ch2_flexible_source_range_selection01.inc;596;222;234;"" 1;~chart2\required\includes\ch2_flexible_source_range_selection01.inc;600;28;63;"tDataSeriesDialogWriter" 4;~chart2\required\includes\ch2_flexible_source_range_selection01.inc;609;4;65535;"Open test document gTestToolPath/chart2/required/input/ch_flexrange08.odt" 4;~chart2\tools\ch_tools_common.inc;163;4;65535;"Select chart using navigator" 2;~global\tools\includes\required\t_doc2.inc;294;48;65535;"Auswahlliste : %ResId=22021%%Arg1=Select%%Arg2=5%%Arg3=0%" 3;~global\tools\includes\required\t_doc2.inc;294;48;65535;"--/home/goodday/OOO310_m18/testautomation/global/tools/includes/required/t_doc2.inc: fSelectFirstOLE%()" 3;~chart2\tools\ch_tools_common.inc;164;4;65535;"--/home/goodday/OOO310_m18/testautomation/chart2/tools/ch_tools_common.inc: fOpenTestDocumentAndSelectDataRange(Byref sInputFile$)" 3;~chart2\required\includes\ch2_flexible_source_range_selection01.inc;294;48;65535;"--/home/goodday/OOO310_m18/testautomation/chart2/required/includes/ch2_flexible_source_range_selection01.inc: tDataSeriesDialogWriter()" 3;~chart2\required\ch2_flexible_source_range_selection.bas;59;12;65535;"/home/goodday/OOO310_m18/testautomation/chart2/required/ch2_flexible_source_range_selection.bas: main()" 4;~global\system\includes\master.inc;561;12;65535;" ** Error in TestExit -Routine **" 5;~global\system\includes\master.inc;562;12;65535;"DOCKINGWINDOW "ナビゲータ" geschlossen, RType = DOCKINGWINDOW, UId = ViewNavigator" 5;~global\system\includes\master.inc;562;12;65535;"BORDERWINDOW "無題 1 - OpenOffice.org Writer TTD :localhost.localdomain[12479]" geschlossen, RType = BORDERWINDOW, UId = Active" 1;~chart2\required\includes\ch2_flexible_source_range_selection01.inc;784;222;234;""
step 4-A:See the Testcase
search the testscript which have a testcase
you can find out from log or this kind of command.
bash-4.0$grep -f *.inc |grep tDataRangeWizardWriter ch2_flexible_source_range_selection01.inc:' #1 tDataRangeWizardWriter ' Updatetest for tabpage DataRange via chart wizard ch2_flexible_source_range_selection01.inc:testcase tDataRangeWizardWriter
get test case from script
testscripts are written in OpenOffice.org Basic. so the comment for this langage is [']. if you want to know about basic you can read OpenOffice.org Basic Guide
sometimes the style of writing is deference as this sample. it depend on person who write testscript.
bash-4.0$cat ch2_flexible_source_range_selection01.inc |grep "testcase*\|'"
testcase tDataRangeWizardWriter
'///Open test document <i>gTestToolPath</i>/chart2/required/input/ch_flexrange05.odt
'///Choose the Chart type <i>line</i> (should be the fifth in the top-down list)
'/// Click Next button
'///Check data range in edit field value (ChartFlexRange.A1:E5)
'///Check if -Data series in rows- is enabled
'///Check if -Data series in columns- is enabled and chosen
'///Check if -First row as label- is enabled and checked
'///Check if -First column as label- is enabled checked
'///Enter invalid range ChartFlexRange.A1:E
'///Check if -Data series in rows- is disabled
'///Check if -Data series in columns- is disabled
'///Check if -First row as label- is disabled
'///Check if -First column as label- is disabled
'///Check that Back, Next and Finish does not work
'///Enter valid range ChartFlexRange.A1:E5
'///Check if -Data series in rows- is enabled
'///Check if -Data series in columns- has been enabled and chosen
'///Check if -First row as label- is enabled and checked
'///Check if -First column as label- is enabled checked
'///Press Cancel button
'///Close document
'
'-------------------------------------------------------------------------
'
testcase tDataSeriesWizardWriter
'///Open test document gTestToolPath/chart2/required/input/ch_flexrange06.odt
'///Choose the Chart type <i>area</i> (should be the fourth in the top-down list)
'/// Click Next button
'/// Click Next button
'///Check if the data series listbox is enabled and contains four entries
'///Check if the ADD button is enabled and click the button
'///Check if there are now five entries in the data series listbox
'///Check if the new data series was added below the highlighted one
'///Check if the REMOVE button is enabled and click the button
'///Check if there are now four entries in the data series listbox
'/// Select third entry
'/// The Move Up button should be enabled
'/// Click the Move Up button twice
'/// Select second entry
'/// If Move Down button is enabled
'/// Click Move Down button twice
'///Select first entry in Data Ranges
'///Verify that the range matches the one in edit field
'GetSelText(line,column) is a new parameter since CWS:SRC680gh13
'Instead of 1,2 there must be 1,4 used becqause there are two invisible
'label-columns in that treelist-box
'///Select second entry in Data Ranges
'///Check that the range in the edit field has changed, too
'GetSelText(line,column) is a new parameter since CWS:SRC680gh13
'Instead of 1,2 there must be 1,4 used becqause there are two invisible
'label-columns in that treelist-box
'///Change the range in edit field
'///Check that the range in Data Ranges has changed, too
'GetSelText(line,column) is a new parameter since CWS:SRC680gh13
'Instead of 1,2 there must be 1,4 used becqause there are two invisible
'label-columns in that treelist-box
'/// Set ChartFlexRange.A2:A4 as range for categories
'///Enter invalid range ChartFlexRange.A2:A
'///Check that the BACK button is disabled
'///Enter valid range ChartFlexRange.A2:A5
'///Press Cancel button
'///Close document
'
'-------------------------------------------------------------------------
'
testcase tDataRangeDialogWriter
'///Open test document <i>gTestToolPath</i>/chart2/required/input/ch_flexrange07.odt
'///Check data range in edit field (ChartFlexRange.A2:A3;ChartFlexRange.B1:E3)
'///Check if -Data series in columns- is enabled and chosen
'///Check if -Data series in rows- is enabled
'///Check if -First row as label- is enabled and checked
'///Check if -First column as label- is enabled and checked
'///Enter invalid range ChartFlexRange.A1:E
'///Check that -Data series in rows- is disabled
'///Check that -Data series in columns- is disabled
'///Check if -First row as label- is disabled
'///Check if -First column as label- is disabled
'///Check that OK button is disabled
'///Enter valid range ChartFlexRange.A1:E3
'///Check if -Data series in columns- is enabled and chosen
'///Check if -Data series in rows- is enabled
'///Check if -First row as label- is enabled and checked
'///Check if -First column as label- is enabled and checked
'///Press Cancel button
'///Close document
'
'-------------------------------------------------------------------------
'
testcase tDataSeriesDialogWriter
'/// Open test document <i>gTestToolPath</i>/chart2/required/input/ch_flexrange08.odt
'/// Switching to tab page Data Series
'///Check if the data series listbox is enabled and contains four entries
'///Check if the ADD button is enabled and click the button
'///Check if there are now five entries in the data series listbox
'///Check if the new data series was added below the highlighted one
'///Check if the REMOVE button is enabled and click the button
'///Check if there are now four entries in the data series listbox
'///Use UP button and verify that it it disabled on first entry
'///Use DOWN button thrice and verify that it is disabled on last entry
'///Select first entry in Data Ranges
'///Verify that the range matches the one in edit field
'GetSelText(line,column) is a new parameter since CWS:SRC680gh13
'Instead of 1,2 there must be 1,4 used becqause there are two invisible
'label-columns in that treelist-box
'///Select second entry in Data Ranges
'///Check that the range in the edit field has changed,too
'GetSelText(line,column) is a new parameter since CWS:SRC680gh13
'Instead of 1,2 there must be 1,4 used becqause there are two invisible
'label-columns in that treelist-box
'///Change the range in edit field to ChartFlexRange.B2:B3
'///Check that the range in Data Ranges has changed, too
'GetSelText(line,column) is a new parameter since CWS:SRC680gh13
'Instead of 1,2 there must be 1,4 used becqause there are two invisible
'label-columns in that treelist-box
'///Edit range for Categories (ChartFlexRange.A2:A3)
'///Enter invalid range ChartFlexRange.A2:A
'///Check that the OK button is disabled
'///Enter valid range ChartFlexRange.A2:A4
'///Press Cancel button
'///Close document
'
'-------------------------------------------------------------------------
'
'///Menu Insert Object Chart to open chart wizard
Do it by your hand and find out the reason
try this testcase by my hand. sometimes you need to see the testscript. because this comment of testscript does not have detail.
testcase tDataRangeDialogWriter
'///Open test document <i>gTestToolPath</i>/chart2/required/input/ch_flexrange07.odt
'///Check data range in edit field (ChartFlexRange.A2:A3;ChartFlexRange.B1:E3)
'///Check if -Data series in columns- is enabled and chosen
'///Check if -Data series in rows- is enabled
'///Check if -First row as label- is enabled and checked
'///Check if -First column as label- is enabled and checked
'///Enter invalid range ChartFlexRange.A1:E
'///Check that -Data series in rows- is disabled
'///Check that -Data series in columns- is disabled
'///Check if -First row as label- is disabled
'///Check if -First column as label- is disabled
'///Check that OK button is disabled
'///Enter valid range ChartFlexRange.A1:E3
'///Check if -Data series in columns- is enabled and chosen
'///Check if -Data series in rows- is enabled
'///Check if -First row as label- is enabled and checked
'///Check if -First column as label- is enabled and checked
'///Press Cancel button
'///Close document
'
'-------------------------------------------------------------------------
step 4-B:Debag with TestTool
set the break point and find out the error. sometime it is easy way for get the reason.
- open the testtool.
- run the testscript.
- after that ,you may get error.
- click above line from error message
- set the breakpoint on the testscript.
- run the testscript again.
- it will be stop on the break point.
- you can try the testscript step by step (short cut key is "F8")
sometimes you can not see the error on debug mode.it is fine.
knowledge
ERROR: Auswahlliste : Entry number 5 is too large in Select. Max. allowed is 0
normally this error is just tools problem. some time rendering the navigater is too slow so it can not sucssess.
MESSAGE: Select chart using navigator ERROR: Auswahlliste : Entry number 5 is too large in Select. Max. allowed is 0
- Solution 1:Just try again this testscript.
- Solution 2:open the navigater before starting the test.
- Solution 3:modify the testscript
add wait() or sleep() function.
- Solution 4:Do the testcase by your hand
Glossary
- Auswahlliste(German)
- shortlist(English)
- 選択リスト(Japanese)
Any Quotation
please contact qa-dev mailing list.
