Difference between revisions of "User:Foral/automation/testcase"
(16 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
=After automation test= | =After automation test= | ||
− | After automation test. | + | After automation test.you should find out the reason why you get error or warnings. |
+ | you may find out the reason from log if you only get warning(that time just do step 3) | ||
+ | but you have to do some if you get error and sometime you have to retest it which you had error. | ||
*step 1:Make Summary | *step 1:Make Summary | ||
*step 2:Test again one by one | *step 2:Test again one by one | ||
*step 3:See the Log | *step 3:See the Log | ||
− | *step 4:See the Testcase | + | *step 4-A:See the Testcase and Do it by your hand and filling out the reason. |
− | + | *step 4-B:Debag with TestTool | |
==Step 1:Make Summary== | ==Step 1:Make Summary== | ||
− | To get | + | To get summary,you can use tools/analyze/make_summary.pl and make a summary of test result. |
and you can explore the test result. | and you can explore the test result. | ||
Line 117: | Line 119: | ||
==step 2:Test again one by one== | ==step 2:Test again one by one== | ||
− | sometimes error/warning come from just timing or setting so just test again one by one and some of error/warning might be gone.especially you should test again | + | sometimes error/warning come from just timing or setting so just test again one by one and some of error/warning might be gone. |
+ | |||
+ | especially you should test again when you get a lot of error/warning. | ||
*c_updt1.res | *c_updt1.res | ||
Line 136: | Line 140: | ||
*f_wizards.res | *f_wizards.res | ||
− | |||
− | |||
− | |||
− | |||
**error has gone. | **error has gone. | ||
**it was timing error. | **it was timing error. | ||
Line 209: | Line 209: | ||
</pre> | </pre> | ||
− | ==step 4:See the Testcase== | + | ==step 4-A:See the Testcase== |
− | ===search the testscript which have a | + | ===search the testscript which have a testcase=== |
+ | you can find out from log or this kind of command. | ||
<pre> | <pre> | ||
Line 220: | Line 221: | ||
===get test case from script=== | ===get test case from script=== | ||
+ | testscripts are written in OpenOffice.org Basic. | ||
+ | so the comment for this langage is [']. | ||
+ | if you want to know about basic you can read [[Documentation/BASIC_Guide|OpenOffice.org Basic Guide]] | ||
+ | |||
sometimes the style of writing is deference as this sample. | sometimes the style of writing is deference as this sample. | ||
it depend on person who write testscript. | it depend on person who write testscript. | ||
Line 350: | Line 355: | ||
' | ' | ||
'///Menu Insert Object Chart to open chart wizard | '///Menu Insert Object Chart to open chart wizard | ||
− | |||
− | |||
− | |||
</pre> | </pre> | ||
− | == | + | ===Do it by your hand and find out the reason=== |
try this testcase by my hand. | try this testcase by my hand. | ||
sometimes you need to see the testscript. | sometimes you need to see the testscript. | ||
− | because this comment of | + | because this comment of testscript does not have detail. |
<pre> | <pre> | ||
Line 384: | Line 386: | ||
'------------------------------------------------------------------------- | '------------------------------------------------------------------------- | ||
</pre> | </pre> | ||
+ | |||
+ | ==step 4-B:Debag with TestTool== | ||
+ | set the break point and find out the error. | ||
+ | sometime it is easy way for get the reason. | ||
+ | |||
+ | #open the testtool. | ||
+ | #run the testscript. | ||
+ | #after that ,you may get error. | ||
+ | #click above line from error message | ||
+ | #set the breakpoint on the testscript. | ||
+ | #run the testscript again. | ||
+ | #it will be stop on the break point. | ||
+ | #you can try the testscript step by step (short cut key is "F8") | ||
+ | |||
+ | sometimes you can not see the error on debug mode.it is fine. | ||
=knowledge= | =knowledge= | ||
==ERROR: Auswahlliste : Entry number 5 is too large in Select. Max. allowed is 0== | ==ERROR: Auswahlliste : Entry number 5 is too large in Select. Max. allowed is 0== | ||
− | normally this error is | + | normally this error`s reason is timing. |
− | + | the script can not access correctly to control | |
+ | if the control rendering soon. | ||
<pre> | <pre> | ||
Line 395: | Line 413: | ||
</pre> | </pre> | ||
− | * | + | *Try it 1:Just try again this testscript. |
− | * | + | *Try it 2:open the navigater before starting the test. |
− | * | + | *Try it 3:modify the testscript add wait() or sleep() function.(your system might be need some waiting time.) |
+ | *Try it 4:Do the testcase by your hand | ||
+ | |||
+ | ==ERROR: DocumentConverter : Window/Control could not be accessed.== | ||
+ | normally this error come from environment on automation test. | ||
+ | the script might be get no Expected window or Control and | ||
+ | the script could not access expected window/control. | ||
+ | |||
+ | *try it 1:Delete all of the work file ,configuration file and retest. | ||
+ | *try it 2:See the screen saver/power management configuration on system and retest. | ||
+ | *try it 3:check security software/configuration. | ||
=Glossary= | =Glossary= | ||
Line 403: | Line 431: | ||
**shortlist(English) | **shortlist(English) | ||
**選択リスト(Japanese) | **選択リスト(Japanese) | ||
+ | |||
+ | =Any Quotation= | ||
+ | please contact qa-dev mailing list. |
Latest revision as of 02:02, 25 August 2009
After automation test
After automation test.you should find out the reason why you get error or warnings. you may find out the reason from log if you only get warning(that time just do step 3) but you have to do some if you get error and sometime you have to retest it which you had error.
- step 1:Make Summary
- step 2:Test again one by one
- step 3:See the Log
- step 4-A:See the Testcase and Do it by your hand and filling out the reason.
- step 4-B:Debag with TestTool
Step 1:Make Summary
To get summary,you can use tools/analyze/make_summary.pl and make a summary of test result. and you can explore the test result.
this sample from 3.1.1rc1 release testing. i tested on Fedora 11 with OOo3.1.1rc1 Linux without JRE version.
bash-4.0$ ./make_summary.pl ../../errorlogs/*.res Filename Errors Warnings Warnings_during_initialization ============================ ====== ======== ============================== c_import_general.res 0 0 0 c_updt1.res 1 1 0 c_updt2.res 0 0 0 c_xml_scenario.res 0 0 0 ch2_flexible_source_range_se 2 2 0 ch2_losa_ods_to_ods.res 0 0 0 ch2_losa_ods_to_xls.res 0 0 0 ch2_losa_sxc_to_ods.res 0 0 0 ch2_losa_sxc_to_sdc.res 0 0 0 ch2_lvl1_wizard.res 0 0 0 ch2_updt_calc.res 1 1 0 ch_xml_japanese_candlestick. 0 0 0 closeoffice.res 1 0 0 d_export_graphic_a.res 0 0 0 d_export_graphic_b.res 0 0 0 d_opt.res 0 0 0 d_updt.res 0 0 0 dba_Main.res 0 0 0 dba_ReportBuilder.res 0 0 0 dba_frm_FormFilter.res 0 0 0 dba_frm_Forms.res 1 3 0 dba_misc_RegisterDatabase.re 0 0 0 dba_rpt_DateTime.res 0 0 0 dba_rpt_ExecuteReport.res 0 0 0 dba_rpt_Formating.res 0 0 0 dba_rpt_Grouping.res 0 0 0 dba_rpt_PageNumber.res 0 1 0 dba_rpt_PropertyBrowser.res 0 0 0 dba_xf_Submission.res 0 0 0 dba_xforms_Main.res 0 0 0 e_fileopen.res 0 0 0 e_location.res 0 0 0 e_options.res 0 0 0 e_update.res 0 0 0 f_basic_dialogs.res 0 0 0 f_basic_issues.res 0 0 0 f_basic_protected_libraries. 0 0 0 f_basic_shared_modules.res 0 0 0 f_configuration.res 0 2 0 f_extras_labels.res 0 0 0 f_extras_tableautoformat.res 0 0 0 f_extras_templatepreview.res 0 0 0 f_filedlg_cjk_files.res 0 0 0 f_filedlg_reserved_names_uni 0 0 0 f_first.res 0 0 0 f_help_browser.res 0 0 0 f_help_compare_applications. 0 0 0 f_help_search.res 0 0 0 f_printer_administration.res 0 0 0 f_script_organizers.res 0 0 0 f_security_certification_dia 0 0 0 f_security_dialogs.res 0 0 0 f_security_incorrect_passwor 0 0 0 f_security_trusted_path.res 0 0 0 f_standard_toolbar.res 0 0 0 f_tools_customize.res 0 0 0 f_tools_options.res 0 0 0 f_topten.res 0 0 0 Filename Errors Warnings Warnings_during_initialization ============================ ====== ======== ============================== f_window_functions.res 0 0 0 f_wizards.res 1 1 0 g_autocorrection.res 0 0 0 g_character.res 0 4 0 g_customshapes.res 0 0 0 g_export_html.res 0 0 0 g_group.res 0 0 0 g_imagemap.res 0 0 0 g_line.res 0 0 0 g_mediaplayer.res 0 0 0 g_ole.res 0 0 0 g_paragraph.res 0 0 0 g_print.res 0 0 0 g_shaddow.res 0 0 0 g_tools.res 0 0 0 g_zoom.res 0 0 0 i_opt.res 0 0 0 i_slidecopy.res 0 0 0 i_updt_1.res 0 0 0 i_updt_2.res 0 0 0 m_lvl1.res 0 0 0 m_updt.res 0 0 0 oasis_graphics_update.res 0 0 0 resetoffice.res 0 0 0 w_numbering.res 0 5 0 w_section.res 2 16 0 w_shortcuts.res 0 0 0 w_sorting.res 0 0 0 w_spellcheck.res 0 5 0 w_updt.res 0 1 0 writer_xml_7_export.res 0 0 0 ============================================================================== Total 89 files 9 42 0
step 2:Test again one by one
sometimes error/warning come from just timing or setting so just test again one by one and some of error/warning might be gone.
especially you should test again when you get a lot of error/warning.
- c_updt1.res
- error has gone.
- it was timing error.(navigater Rendering is too slow)
- ch2_flexible_source_range_se
- error has gone.
- it was timing error.(navigater Rendering is too slow)
- ch2_updt_calc.res
- error has gone.
- it was timing error.(navigater Rendering is too slow)
- dba_frm_Forms.res
- error has gone.
- it was timing error.
- f_wizards.res
- error has gone.
- it was timing error.
step 3:See the Log
now you knew which test script made error and warnings. so you should find out the log from result file with tools/analyze/translater_res_file.pl or use testtool.
I recommend you to use testtool because of it make you more easy to trace the error.
bash-4.0$ ./translate_res_file.pl ../../errorlogs/ch2_flexible_source_range_selection.res |grep ^ERROR -B 5 MESSAGE: Close document TEST_CASE: TEST_CASE: tDataRangeDialogWriter MESSAGE: Open test document gTestToolPath/chart2/required/input/ch_flexrange07.odt MESSAGE: Select chart using navigator ERROR: Auswahlliste : Entry number 5 is too large in Select. Max. allowed is 0 -- WARNING: BORDERWINDOW "無題 1 - OpenOffice.org Writer TTD :localhost.localdomain[12479]" geschlossen, RType = BORDERWINDOW, UId = Active TEST_CASE: TEST_CASE: tDataSeriesDialogWriter MESSAGE: Open test document gTestToolPath/chart2/required/input/ch_flexrange08.odt MESSAGE: Select chart using navigator ERROR: Auswahlliste : Entry number 5 is too large in Select. Max. allowed is 0
this log show us
- testcase [tDataRangeDialogWriter] had a error
- Open test document gTestToolPath/chart2/required/input/ch_flexrange07.odt
- Success
- Select chart using navigator
- Fail:this happen may be come up on the Navigator or near and error message help you to get truth.
- testcase [tDataSeriesDialogWriter] had a error
- Open test document gTestToolPath/chart2/required/input/ch_flexrange08.odt
- Success
- Select chart using navigator
- Fail:this happen may be come up on the Navigator or near and error message help you to get truth.
source of this log
bash-4.0$ cat ch2_flexible_source_range_selection.res |grep "tDataRangeDialogWriter\|tDataSeriesDialogWriter" -A 5 1;~chart2\required\includes\ch2_flexible_source_range_selection01.inc;438;27;61;"tDataRangeDialogWriter" 4;~chart2\required\includes\ch2_flexible_source_range_selection01.inc;443;4;65535;"Open test document gTestToolPath/chart2/required/input/ch_flexrange07.odt" 4;~chart2\tools\ch_tools_common.inc;163;4;65535;"Select chart using navigator" 2;~global\tools\includes\required\t_doc2.inc;294;48;65535;"Auswahlliste : %ResId=22021%%Arg1=Select%%Arg2=5%%Arg3=0%" 3;~global\tools\includes\required\t_doc2.inc;294;48;65535;"--/home/goodday/OOO310_m18/testautomation/global/tools/includes/required/t_doc2.inc: fSelectFirstOLE%()" 3;~chart2\tools\ch_tools_common.inc;164;4;65535;"--/home/goodday/OOO310_m18/testautomation/chart2/tools/ch_tools_common.inc: fOpenTestDocumentAndSelectDataRange(Byref sInputFile$)" 3;~chart2\required\includes\ch2_flexible_source_range_selection01.inc;294;48;65535;"--/home/goodday/OOO310_m18/testautomation/chart2/required/includes/ch2_flexible_source_range_selection01.inc: tDataRangeDialogWriter()" 3;~chart2\required\ch2_flexible_source_range_selection.bas;58;12;65535;"/home/goodday/OOO310_m18/testautomation/chart2/required/ch2_flexible_source_range_selection.bas: main()" 4;~global\system\includes\master.inc;561;12;65535;" ** Error in TestExit -Routine **" 5;~global\system\includes\master.inc;562;12;65535;"DOCKINGWINDOW "ナビゲータ" geschlossen, RType = DOCKINGWINDOW, UId = ViewNavigator" 5;~global\system\includes\master.inc;562;12;65535;"BORDERWINDOW "無題 1 - OpenOffice.org Writer TTD :localhost.localdomain[12479]" geschlossen, RType = BORDERWINDOW, UId = Active" 1;~chart2\required\includes\ch2_flexible_source_range_selection01.inc;596;222;234;"" 1;~chart2\required\includes\ch2_flexible_source_range_selection01.inc;600;28;63;"tDataSeriesDialogWriter" 4;~chart2\required\includes\ch2_flexible_source_range_selection01.inc;609;4;65535;"Open test document gTestToolPath/chart2/required/input/ch_flexrange08.odt" 4;~chart2\tools\ch_tools_common.inc;163;4;65535;"Select chart using navigator" 2;~global\tools\includes\required\t_doc2.inc;294;48;65535;"Auswahlliste : %ResId=22021%%Arg1=Select%%Arg2=5%%Arg3=0%" 3;~global\tools\includes\required\t_doc2.inc;294;48;65535;"--/home/goodday/OOO310_m18/testautomation/global/tools/includes/required/t_doc2.inc: fSelectFirstOLE%()" 3;~chart2\tools\ch_tools_common.inc;164;4;65535;"--/home/goodday/OOO310_m18/testautomation/chart2/tools/ch_tools_common.inc: fOpenTestDocumentAndSelectDataRange(Byref sInputFile$)" 3;~chart2\required\includes\ch2_flexible_source_range_selection01.inc;294;48;65535;"--/home/goodday/OOO310_m18/testautomation/chart2/required/includes/ch2_flexible_source_range_selection01.inc: tDataSeriesDialogWriter()" 3;~chart2\required\ch2_flexible_source_range_selection.bas;59;12;65535;"/home/goodday/OOO310_m18/testautomation/chart2/required/ch2_flexible_source_range_selection.bas: main()" 4;~global\system\includes\master.inc;561;12;65535;" ** Error in TestExit -Routine **" 5;~global\system\includes\master.inc;562;12;65535;"DOCKINGWINDOW "ナビゲータ" geschlossen, RType = DOCKINGWINDOW, UId = ViewNavigator" 5;~global\system\includes\master.inc;562;12;65535;"BORDERWINDOW "無題 1 - OpenOffice.org Writer TTD :localhost.localdomain[12479]" geschlossen, RType = BORDERWINDOW, UId = Active" 1;~chart2\required\includes\ch2_flexible_source_range_selection01.inc;784;222;234;""
step 4-A:See the Testcase
search the testscript which have a testcase
you can find out from log or this kind of command.
bash-4.0$grep -f *.inc |grep tDataRangeWizardWriter ch2_flexible_source_range_selection01.inc:' #1 tDataRangeWizardWriter ' Updatetest for tabpage DataRange via chart wizard ch2_flexible_source_range_selection01.inc:testcase tDataRangeWizardWriter
get test case from script
testscripts are written in OpenOffice.org Basic. so the comment for this langage is [']. if you want to know about basic you can read OpenOffice.org Basic Guide
sometimes the style of writing is deference as this sample. it depend on person who write testscript.
bash-4.0$cat ch2_flexible_source_range_selection01.inc |grep "testcase*\|'" testcase tDataRangeWizardWriter '///Open test document <i>gTestToolPath</i>/chart2/required/input/ch_flexrange05.odt '///Choose the Chart type <i>line</i> (should be the fifth in the top-down list) '/// Click Next button '///Check data range in edit field value (ChartFlexRange.A1:E5) '///Check if -Data series in rows- is enabled '///Check if -Data series in columns- is enabled and chosen '///Check if -First row as label- is enabled and checked '///Check if -First column as label- is enabled checked '///Enter invalid range ChartFlexRange.A1:E '///Check if -Data series in rows- is disabled '///Check if -Data series in columns- is disabled '///Check if -First row as label- is disabled '///Check if -First column as label- is disabled '///Check that Back, Next and Finish does not work '///Enter valid range ChartFlexRange.A1:E5 '///Check if -Data series in rows- is enabled '///Check if -Data series in columns- has been enabled and chosen '///Check if -First row as label- is enabled and checked '///Check if -First column as label- is enabled checked '///Press Cancel button '///Close document ' '------------------------------------------------------------------------- ' testcase tDataSeriesWizardWriter '///Open test document gTestToolPath/chart2/required/input/ch_flexrange06.odt '///Choose the Chart type <i>area</i> (should be the fourth in the top-down list) '/// Click Next button '/// Click Next button '///Check if the data series listbox is enabled and contains four entries '///Check if the ADD button is enabled and click the button '///Check if there are now five entries in the data series listbox '///Check if the new data series was added below the highlighted one '///Check if the REMOVE button is enabled and click the button '///Check if there are now four entries in the data series listbox '/// Select third entry '/// The Move Up button should be enabled '/// Click the Move Up button twice '/// Select second entry '/// If Move Down button is enabled '/// Click Move Down button twice '///Select first entry in Data Ranges '///Verify that the range matches the one in edit field 'GetSelText(line,column) is a new parameter since CWS:SRC680gh13 'Instead of 1,2 there must be 1,4 used becqause there are two invisible 'label-columns in that treelist-box '///Select second entry in Data Ranges '///Check that the range in the edit field has changed, too 'GetSelText(line,column) is a new parameter since CWS:SRC680gh13 'Instead of 1,2 there must be 1,4 used becqause there are two invisible 'label-columns in that treelist-box '///Change the range in edit field '///Check that the range in Data Ranges has changed, too 'GetSelText(line,column) is a new parameter since CWS:SRC680gh13 'Instead of 1,2 there must be 1,4 used becqause there are two invisible 'label-columns in that treelist-box '/// Set ChartFlexRange.A2:A4 as range for categories '///Enter invalid range ChartFlexRange.A2:A '///Check that the BACK button is disabled '///Enter valid range ChartFlexRange.A2:A5 '///Press Cancel button '///Close document ' '------------------------------------------------------------------------- ' testcase tDataRangeDialogWriter '///Open test document <i>gTestToolPath</i>/chart2/required/input/ch_flexrange07.odt '///Check data range in edit field (ChartFlexRange.A2:A3;ChartFlexRange.B1:E3) '///Check if -Data series in columns- is enabled and chosen '///Check if -Data series in rows- is enabled '///Check if -First row as label- is enabled and checked '///Check if -First column as label- is enabled and checked '///Enter invalid range ChartFlexRange.A1:E '///Check that -Data series in rows- is disabled '///Check that -Data series in columns- is disabled '///Check if -First row as label- is disabled '///Check if -First column as label- is disabled '///Check that OK button is disabled '///Enter valid range ChartFlexRange.A1:E3 '///Check if -Data series in columns- is enabled and chosen '///Check if -Data series in rows- is enabled '///Check if -First row as label- is enabled and checked '///Check if -First column as label- is enabled and checked '///Press Cancel button '///Close document ' '------------------------------------------------------------------------- ' testcase tDataSeriesDialogWriter '/// Open test document <i>gTestToolPath</i>/chart2/required/input/ch_flexrange08.odt '/// Switching to tab page Data Series '///Check if the data series listbox is enabled and contains four entries '///Check if the ADD button is enabled and click the button '///Check if there are now five entries in the data series listbox '///Check if the new data series was added below the highlighted one '///Check if the REMOVE button is enabled and click the button '///Check if there are now four entries in the data series listbox '///Use UP button and verify that it it disabled on first entry '///Use DOWN button thrice and verify that it is disabled on last entry '///Select first entry in Data Ranges '///Verify that the range matches the one in edit field 'GetSelText(line,column) is a new parameter since CWS:SRC680gh13 'Instead of 1,2 there must be 1,4 used becqause there are two invisible 'label-columns in that treelist-box '///Select second entry in Data Ranges '///Check that the range in the edit field has changed,too 'GetSelText(line,column) is a new parameter since CWS:SRC680gh13 'Instead of 1,2 there must be 1,4 used becqause there are two invisible 'label-columns in that treelist-box '///Change the range in edit field to ChartFlexRange.B2:B3 '///Check that the range in Data Ranges has changed, too 'GetSelText(line,column) is a new parameter since CWS:SRC680gh13 'Instead of 1,2 there must be 1,4 used becqause there are two invisible 'label-columns in that treelist-box '///Edit range for Categories (ChartFlexRange.A2:A3) '///Enter invalid range ChartFlexRange.A2:A '///Check that the OK button is disabled '///Enter valid range ChartFlexRange.A2:A4 '///Press Cancel button '///Close document ' '------------------------------------------------------------------------- ' '///Menu Insert Object Chart to open chart wizard
Do it by your hand and find out the reason
try this testcase by my hand. sometimes you need to see the testscript. because this comment of testscript does not have detail.
testcase tDataRangeDialogWriter '///Open test document <i>gTestToolPath</i>/chart2/required/input/ch_flexrange07.odt '///Check data range in edit field (ChartFlexRange.A2:A3;ChartFlexRange.B1:E3) '///Check if -Data series in columns- is enabled and chosen '///Check if -Data series in rows- is enabled '///Check if -First row as label- is enabled and checked '///Check if -First column as label- is enabled and checked '///Enter invalid range ChartFlexRange.A1:E '///Check that -Data series in rows- is disabled '///Check that -Data series in columns- is disabled '///Check if -First row as label- is disabled '///Check if -First column as label- is disabled '///Check that OK button is disabled '///Enter valid range ChartFlexRange.A1:E3 '///Check if -Data series in columns- is enabled and chosen '///Check if -Data series in rows- is enabled '///Check if -First row as label- is enabled and checked '///Check if -First column as label- is enabled and checked '///Press Cancel button '///Close document ' '-------------------------------------------------------------------------
step 4-B:Debag with TestTool
set the break point and find out the error. sometime it is easy way for get the reason.
- open the testtool.
- run the testscript.
- after that ,you may get error.
- click above line from error message
- set the breakpoint on the testscript.
- run the testscript again.
- it will be stop on the break point.
- you can try the testscript step by step (short cut key is "F8")
sometimes you can not see the error on debug mode.it is fine.
knowledge
ERROR: Auswahlliste : Entry number 5 is too large in Select. Max. allowed is 0
normally this error`s reason is timing. the script can not access correctly to control if the control rendering soon.
MESSAGE: Select chart using navigator ERROR: Auswahlliste : Entry number 5 is too large in Select. Max. allowed is 0
- Try it 1:Just try again this testscript.
- Try it 2:open the navigater before starting the test.
- Try it 3:modify the testscript add wait() or sleep() function.(your system might be need some waiting time.)
- Try it 4:Do the testcase by your hand
ERROR: DocumentConverter : Window/Control could not be accessed.
normally this error come from environment on automation test. the script might be get no Expected window or Control and the script could not access expected window/control.
- try it 1:Delete all of the work file ,configuration file and retest.
- try it 2:See the screen saver/power management configuration on system and retest.
- try it 3:check security software/configuration.
Glossary
- Auswahlliste(German)
- shortlist(English)
- 選択リスト(Japanese)
Any Quotation
please contact qa-dev mailing list.