AOO UX Research Surveys

< AOO User Experience Community Home

Introduction

To help make informed design and development decisions, the OpenOffice community needs user data!

We need to understand who uses OpenOffice, what they use it for, and what is important to them. Beyond roles and goals, we need to understand user attitudes and behaviour, uncover latent and emerging needs, and understand drivers and barriers of change.

Surveys are an excellent way to obtain information about our users, their goals, needs and expectations. The outcome of user survey effort is enhanced insight into the user, their context of use and criteria for success.

Surveys are an ongoing research activity, that can deliver broad-ranging or targeted user feedback and insight.

• See for a list of recent and current survey activities.
• See AOO UX Research Strategy for more information on the broader UX research effort, including metrics and reporting.

Survey Research Goals

• to characterize the current state of the product experience
• to capture data (see next section) that helps our design and development contributors make informed decisions
• establish a baseline for future comparison
• refine our understanding of our users and their characteristics
• to understand task frequency, feature importance and user satisfaction
• identify opportunities for improvement that will have the biggest positive impact

Survey Development Goals

• develop survey assets that we can re-use over time to capture longitudinal data
• develop surveys that are light-weight and do not take too much time for respondent to complete
• develop surveys that are compelling, so that people want to participate
• develop survey that are enjoyable to complete

AOO Community Design and Development Data Needs

• data that helps us understand who uses AOO
• data that helps us understand why people use AOO
• data that helps us understand what tasks people want to accomplish in AOO
• data that helps us understand how people use AOO
• data that helps us understand how people integrate AOO into their lives
  • work
  • social
• data that helps us understand which existing features are most valuable
• data that helps us understand how we might enhance/improve existing features
• data that helps us prioritize our incremental enhancement effort
• data that helps us identify new opportunities
  • new products
  • new capabilities
• data that helps us understand which technology we need to support and integrate with
  • different devices
  • different platforms
  • different integrations
• data that helps us understand how people heard about, access and consume AOO

Methodology

The user experience research effort seeks to characterize the current state of the product experience by applying a variety of qualitative and quantitative survey methods. User research insight will help us to not only understand existing usability risks but also understand the value of the product offering. Data collected through user surveys can enable the community to understand the most important user tasks and goals, assess how well products support those goals and having experienced the product, what is the user's overall impression of the product and the brand. Survey questions targeting a combination of behavioral data and attitudinal data will help us establish a baseline for future analysis.

Approach

Proposed by Kevin Grignon - 2012 06 20

Given the low cost and high value of user surveys, we should consider deploying an ongoing survey program. User surveys could be deployed to capture the following types of information:

• user profile and demographics
• task prioritization
• task satisfaction
• feature design evaluation
• brand impression

While we can harvest demographic data from our download metrics, we should also include standard

Delivery

Discuss here

Date	Name	Description	Status	Notes
Recent Surveys
2012 06 20	2012 07 - AOO - Task Prioritization Survey	To understand task prioritization for documents, spreadsheets and presentations tasks.	In design	Includes the following questions: 1) core demographics 2) task prioritization 3) standard usability score 4) net promoter score
2012 06 20	2012 07 - AOO - Task Prioritization Survey	To understand task prioritization for documents, spreadsheets and presentations tasks.	In design	Includes the following questions: 1) core demographics 2) task prioritization 3) standard usability score 4) net promoter score
Archived Surveys
2008 06	2008 06 - OOo User Survey	To understand user satisfaction with OOo	Complete - Results available	Includes the following questions: 1) core demographics 2) offering satisfaction ratings

Survey Questions Candidates

The following questions have been proposed for inclusion in a user satisfaction survey:

Proposed by Kevin Grignon, 2012 06 01

Notes: Kevin Grignon 10:00, 5 June 2012 (CST) - I would like to tweak these questions to ensure their are generic enough for home users and business users. OO.org survey below does a better job of this.

Yorick 14:04, 4 June 2012 (UTC) I like your goals above and they seem to cover all forms of survey, but without considering the logistics. We need to define the demographic, timing, means of contact, means of delivery, timespan, data analysis and publication. All of these have an affect on the style, content, size and scope of the questionnaire. Survey styles vary considerably depending just on demographic and delivery.

At first glance this one seems quite long, but I think that's just an impression. If the respondent is excluded from answering the queries relating to the applications that they don't use, then I think it would be OK.

Kevin Grignon 10:00, 5 June 2012 (CST) - Thanks for comments Yorick. I agree the length is a problem. Let's discuss how we might break up the survey to address our data needs - chunking, and progressive disclosure can help. I'd like us to capture our data goals so we can decide how to delivery this. See data goals section above.

Yorick 21:59, 4 June 2012 (UTC) You have used Symphony application branding, so I presume this is just a cut and paste from IBM survey, we should use the OpenOffice application brand names: Writer, Calc, Draw, Impress and Math (Changed)

Kevin Grignon 10:00, 5 June 2012 (CST) - Re: application names. Thanks for updating. I changed a few outliers ;)

• A1: Which version of OpenOffice do you use? [end survey if response “I don't use OpenOffice”]
  • Version 1.0, Version 1.1, Version 1.2, Version 1.3, Version 3.0+, I don't use OpenOffice, Other

• A2: Which OpenOffice applications do you use (at least a few times a month)?
  • Writer, Calc, Impress, Base, Draw, Math

• A3: Overall, how satisfied are you with OpenOffice?
  • Very satisfied, somewhat satisfied, Neutral/undecided, somewhat dissatisfied, very dissatisfied

• A4: Why are you satisfied, dissatisfied, or neutral/undecided?
  • Text field

• A5: If you answered with anything other than “Very satisfied,” what would it take to make you “Very satisfied?”
  • Text field

• A6: Which factors contribute the most to your continued use of OpenOffice?
  • Text field

• A7: Which OpenOffice application do you use the most often?
  • Writer, Calc, Impress, Base, Draw, Math

Writer

• A8: How often do you use OpenOffice Writer?
  • Multiple times a day, Once a day, Multiple times a week, Once a week, Less than once a week

• A9: In addition to OpenOffice Writer, which competitive products do you actively use?
  • List competitive products plus “other” option

• A10: Indicate if a task is important to accomplishing your work (check all that apply) and if it currently functions well in OpenOffice Writer (for those checked).
  • List key tasks

• A11: Are there any missing tasks that are both important and function poorly?
  • Text field

• A12: What issues/deficiencies have you found with OpenOffice Writer?
  • Text field

Yorick 14:04, 4 June 2012 (UTC) "biggest" is colloquial and broadly subjective. It doesn't need to be there. Personalise the question: "What issues/deficiencies have you found with OpenOffice ####?" You could even enumerate the query: "What are two...etc". The issues that are at the forefront of the users mind will automatically present themselves. Odd colloquialisms do not translate well, if you feel that there is a need for an adjective there, then "Significant" would be a better choice. Our surveys need to be translated into many languages.

Kevin Grignon 10:00, 5 June 2012 (CST) - Agreed. Question candidate updated in all app instances.

• A13: Indicate your level of agreement with the following statements. “OpenOffice Writer...”
  • Is easy to use, Offers competitive features, Is something I could recommend, Helps me get my work done, Includes most or all of the features I need, Is innovative, Is as good as or better than the competition

Calc

• A14: How often do you use OpenOffice Calc?
  • Multiple times a day, Once a day, Multiple times a week, Once a week, Less than once a week

• A15: In addition to OpenOffice Calc, which competitive products do you actively use?
  • List competitive products plus “other” option

• A16: Indicate if a task is important to accomplishing your work (check all that apply) and if it currently functions well in OpenOffice Calc (for those checked).
  • List key tasks

• A17: Are there any missing tasks that are both important and function poorly?
  • Text field

• A18: What issues/deficiencies have you found with OpenOffice Calc?
  • Text field

• A19: Indicate your level of agreement with the following statements. “OpenOffice Calc...”
  • Is easy to use, Offers competitive features, Is something I could recommend, Helps me get my work done, Includes most or all of the features I need, Is innovative, Is as good as or better than the competition

Impress

• A20: How often do you use OpenOffice Impress?
  • Multiple times a day, Once a day, Multiple times a week, Once a week, Less than once a week

• A21: In addition to OpenOffice Impress, which competitive products do you actively use?
  • List competitive products plus “other” option

• A22: Indicate if a task is important to accomplishing your work (check all that apply) and if it currently functions well in OpenOffice Impress (for those checked).
  • List key tasks

• A23: Are there any missing tasks that are both important and function poorly?
  • Text field

• A24: What issues/deficiencies have you found with OpenOffice Impress?
  • Text field

• A25: Indicate your level of agreement with the following statements. “OpenOffice Impress...”
  • Is easy to use, Offers competitive features, Is something I could recommend, Helps me get my work done, Includes most or all of the features I need, Is innovative, Is as good as or better than the competition

Draw

• A26: How often do you use OpenOffice Draw?
  • Multiple times a day, Once a day, Multiple times a week, Once a week, Less than once a week

• A27: In addition to OpenOffice Draw, which competitive products do you actively use?
  • List competitive products plus “other” option

• A28: Indicate if a task is important to accomplishing your work (check all that apply) and if it currently functions well in OpenOffice Draw (for those checked).
  • List key tasks

• A29: Are there any missing tasks that are both important and function poorly?
  • Text field

• A30: What issues/deficiencies have you found with OpenOffice Draw?
  • Text field

• A31: Indicate your level of agreement with the following statements. “OpenOffice Draw...”
  • Is easy to use, Offers competitive features, Is something I could recommend, Helps me get my work done, Includes most or all of the features I need, Is innovative, Is as good as or better than the competition

Base

• A32: How often do you use OpenOffice Base?
  • Multiple times a day, Once a day, Multiple times a week, Once a week, Less than once a week

• A33: In addition to OpenOffice Base, which competitive products do you actively use?
  • List competitive products plus “other” option

• A34: Indicate if a task is important to accomplishing your work (check all that apply) and if it currently functions well in OpenOffice Base (for those checked).
  • List key tasks

• A35: Are there any missing tasks that are both important and function poorly?
  • Text field

• A36: What issues/deficiencies have you found with OpenOffice Base ?
  • Text field

• A37: Indicate your level of agreement with the following statements. “OpenOffice Base...”
  • Is easy to use, Offers competitive features, Is something I could recommend, Helps me get my work done, Includes most or all of the features I need, Is innovative, Is as good as or better than the competition

Math

• A38: How often do you use OpenOffice Math?
  • Multiple times a day, Once a day, Multiple times a week, Once a week, Less than once a week. Yorick 14:04, 4 June 2012 (UTC) "Never (if your answer is Never, sakip to question A44)" ("Never" should always be added and instructions to skip. "Never" is a frequent answer for Math, Base, Draw and Impress.)

• A39: In addition to OpenOffice Math, which competitive products do you actively use?
  • List competitive products plus “other” option

• A40: Indicate if a task is important to accomplishing your work (check all that apply) and if it currently functions well in OpenOffice Math(for those checked).
  • List key tasks

• A41: Are there any missing tasks that are both important and function poorly?

• A42: What issues/deficiencies have you found with OpenOffice Math?

• A43: Indicate your level of agreement with the following statements. “OpenOffice Math...”
  • Is easy to use, Offers competitive features, Is something I could recommend, Helps me get my work done, Includes most or all of the features I need, Is innovative, Is as good as or better than the competition

Additional Details

• A44: How did you receive the link to this survey?

• A45: What is your relationship with Apache OpenOffice?
  • User, Contributor, Business Partner, Other

• A46: In which country do you primarily work?
  • [country list]

• A47: Which operating system do you use?
  • Windows, Mac, Linux, Other

• A48: If applicable In which industry is your company?
  • [industry list]

• A49: If applicable, what is your company size?
  • Less than 50 employees, 50-99 employees, 100-999 employees, 1,000+ employees

• A50: Which best describes your job role?
  • [roles list]

Proposed by Albino Neto, 2012 06 01

Yorick 14:04, 4 June 2012 (UTC) Can we define the goals of this query set

• How old are you ?

Regina Henschel says: Why this question?

Rob Weir says: Maybe ask sex [of user]?

• B1: What S.O you use ?
  • Linux
  • Mac
  • Windows
  • other [what]

Regina Henschel says: The user might use different OS depending where he works, for example Windows on the job and Linux at home. Do you want to know the OS of the machine the user is currently using or the machine he is mostly using?

Rob Weir says: Is this a single choice or multiple choice question? Many people use more than one. So if the intent is a single choice, you might word the question as: "what is your primary operating system"?

• B2: Use of Linux ? Wich graphic interface:
  • GNOME
  • KDE
  • Lxde
  • Others [what]

Regina Henschel says: We want to know, whether Win98 is still used. So after answer "Windows" a question for the type of Windows should follow.

• B3: Where do you use Apache OpenOffice?
  • Home
  • Office
  • Company
  • Telecenter
  • Others [what]

Regina Henschel says:What is the difference between 'Office' and 'Company'? What is a 'Telecenter'? I miss 'pulic PC in school, university, or library'. Again, do you mean the machine the user currently uses or the machine he is mostly using?

Rob Weir says: To me the choices "Office" and "Company" seem to be the same. So there is a risk in the survey that users will randomly split between those two choices. Better to have a smaller number of choices that are clearly different, like: Home, Work, School, Other.

• B4: Where to get support ?
  • Manuals
  • Mailling list
  • Search
  • Friends
  • Others [what]

Regina Henschel says: 'Forum' is missing.

• B5: Where do you want to get support ?
  • Manuals
  • Mailing list
  • Search
  • Friends
  • Others [what]

Regina Henschel says: 'Forum' is missing.

• B6: How often do you use Apahce OpenOffice ?
  • Sometimes
  • With frequency
  • Daily

Regina Henschel says: What is 'Sometimes' and 'With frequency'? What about a question "When did you use AOO the last time?'

Rob Weir says: The "daily" choice is quantifiable. "with frequency" is not. I don't know if it is more or less than "daily".

• B7: How much time you spend on the computer ?
  • 30 min to 1 hora
  • 1h to 3h
  • 3h to 5h
  • More of 8h

Regina Henschel says: There might be a large difference between workdays and weekend.

Rob Weir says: per day

• B8: As you consider using the computer ?
  • Beginner
  • intermediate
  • Advanced (expert)

• How important computer for you:
  • unimportant
  • insignificant
  • Very Important

Regina Henschel says: Both will mostly give the middle answer. Try to find four answers. What do you want to learn from these questions?

• B9: How do you consider a nice software?
  • With enough buttons
  • Buttons significant
  • Buttons simple and agile
  • Results
  • Buttons and good visual meanings.

Regina Henschel says: Why the focus on buttons? You might ask which other software the user thinks is 'nice' and why.

Rob Weir says: This might be a good place to ask a series of questions along the lines of: B10: How important are each of the following to you. Score 1-5, where 1 is "not important at all" and 5 is "very important" 1. Easy to use without reading manual 2. Interoperability with Microsoft file formats 3. Open source, built and supported by a volunteer community 4. Is available in my language 5. Speed 6. Availability of free document templates 7. Availability of extensions And so on.

The nice thing about this approach is you can then correlate the responses for each of these questions with the demographic info (how often do you use your computer, where do you use AOO, etc.)

That is pretty much the standard form of an opinion survey: 1) A section of de