Chart2/ChartAnnotation
Significance (p-Values)
-> VERY HIGH PRIORITY
One of the most useful informations present on a Chart which is way to often forgotten describes the significance of the displayed difference.
- a trivial difference (i.e. non-significant) might look frightening big through mischoosing the axes
- while a real difference might be overlooked
This conveys really additional information (unlike some other techniques, like voluminizing)!!!
It is so important, that it deserves highest priority. This feature would also force users to comply to best practice guidelines avoiding mispresentation of their data. [I do NOT mean by this to enforce p-value display, but when confronted with the possibility to automatically display the significance levels, many users will feel that they should do it.] I will describe in the next paragraphs some common scenarios.
When performing a statistical analysis we obtain a number of p-values for the various data groups. Depending on the purpose of our Chart we may choose different values to display. There are generally two approaches for displaying p-values:
- explicit: like p = 0.002 (provide actual number)
- code the p-value: NS (not significant), * (e.g. p< 0.05), ** (p< 0.01), and so on (the cutoffs are not universal, but might be changed)
Here are the various possibilities/ options that should be available:
- Display Non-Significant results: useful when one wants to highlight the absence of an effect
- display as NS
- display full p value (e.g. p = 0.10)
- display full p value only when p is marginal (e.g. p = 0.06; marginal cutoff is usually 0.05 < p < 0.10)
- Display Significant Results: (usual cutoff p < 0.05)
- explicit p: e.g. p = 0.02
- if p> 0.01, display only (round to) 2 decimal points
- if p = 0, display as p < 0.001 (set explicitly the value)
- if p<0.01:
- display as p<0.01
- display 3 decimals
- if p<0.001: display as p<0.001
- for p of the form p < 1E-8, display in scientific format (i.e. like 1E-8, or 1E-3)
- coded
- NS see above
- 0.01 < p < 0.05: code as one *
- 0.001 < p < 0.01: code as two *, (aka **); (cutoff: 0.001 or 0.005)
- p < 0.001, code as ***
- explicit p: e.g. p = 0.02
TO BE EXPANDED