Difference between revisions of "Chart2"
m (→Links) |
|||
Line 221: | Line 221: | ||
* instruct users what external programs (extensions) do exist and where to find them (provide url) | * instruct users what external programs (extensions) do exist and where to find them (provide url) | ||
* allow users to easily acces the functionality of external programs through OOo Menus (or functions) | * allow users to easily acces the functionality of external programs through OOo Menus (or functions) | ||
+ | |||
+ | ---- | ||
---- | ---- | ||
Line 239: | Line 241: | ||
** see point ''6.m'' in original list | ** see point ''6.m'' in original list | ||
** useful programs ''GRASS'' and ''R'' (including packages ''GRASS'' and ''gstat'') | ** useful programs ''GRASS'' and ''R'' (including packages ''GRASS'' and ''gstat'') | ||
+ | |||
+ | Other chart types (less priority): | ||
+ | * see also http://addictedtor.free.fr/graphiques/allgraph.php | ||
+ | * Cluster Analysis: see also stat wiki page | ||
+ | * Dendrograms | ||
+ | ** coloured: http://addictedtor.free.fr/graphiques/RGraphGallery.php?graph=79 | ||
+ | * Regressions: | ||
+ | ** Effect Plot for ''Generalized Linear Models'': http://addictedtor.free.fr/graphiques/RGraphGallery.php?graph=124 | ||
+ | ** Quantile Regression: http://addictedtor.free.fr/graphiques/RGraphGallery.php?graph=109 | ||
+ | * Various Boxplot Derivations: | ||
+ | ** Boxplots: http://addictedtor.free.fr/graphiques/RGraphGallery.php?graph=102 | ||
+ | ** Violin Plot: http://addictedtor.free.fr/graphiques/RGraphGallery.php?graph=43 | ||
+ | * Various Scatterplots | ||
+ | ** with Tufte axes: http://addictedtor.free.fr/graphiques/RGraphGallery.php?graph=81 | ||
+ | ** smoothened: http://addictedtor.free.fr/graphiques/RGraphGallery.php?graph=139 | ||
+ | ** 2D boxplot: http://addictedtor.free.fr/graphiques/RGraphGallery.php?graph=112 | ||
+ | * Climate Diagrams: | ||
+ | ** ''climate diagrams'' are brief summaries of average climatic variables and their time course | ||
+ | ** see http://www.zoolex.org/walter.html and http://addictedtor.free.fr/graphiques/RGraphGallery.php?graph=122 | ||
+ | * Others: e.g. http://addictedtor.free.fr/graphiques/RGraphGallery.php?graph=131, http://addictedtor.free.fr/graphiques/RGraphGallery.php?graph=96 | ||
== Links == | == Links == |
Revision as of 18:44, 13 October 2006
Chart2 is a sub-project of the OpenOffice.org Chart Project. Our goal is to develop a new Chart component for (presumably) OOo 2.6.
Charts are used for visualizing data sets from e.g. spreadsheets by two and three dimensional diagrams. There are a lot of different two- and three-dimensional chart types you can choose from. This page gathers information about the new chart implementation of OpenOffice.org. It is especially written to help new comers to the process of developping in the new OOo chart module.
If you would like to participate, if you have comments or questions related to the chart you are welcome on the graphics mailinglists: (users,dev,features,bugs,cvs)@graphics.openoffice.org. See also the section at the bottom of this page called [Some useful information].
Development in the New Chart
As the information on how to compile the new chart and on how to develop in this project have become quite lenghty, they can be found now on separate pages:
- FAQ about Chart API Compatibility If you have problems with Macros or extensions that use the com.sun.star.chart API you may find answers or ask questions on this page.
- Compiling the new chart module
- Developing for the new chart module
Open Technical Issues in the New Chart
There still some architectural issues left that have to be solved. This section serves for showing those problems and showing the progress in finding solutions.
Application-Framework-Related Problems
Identify the XDataProvider reliably
If the chart is embedded in a container document that provides the data for this chart, the chart contains range-strings at several places in its conten.xml stream that are understood by the container, so that the data can be located there. In Calc such a string would look like this: "Sheet1.A2:A7". Such range-strings can appear in the chart:plot-area element of a chart or at a chart:series, chart:categories or chart:domain element.
If a chart has its own data, this data is stored in a local table in the XML-file. The mentioned range-strings are then of a similar form, with the fixed table-name "local_table". But, the local table is also stored, as a kind of cache, if the data comes from outside. Therefore the existence of the local table does not imply that a chart has own data.
So, neither the existence of local data, nor the existence of range-addresses (and even the content, considering that it is allowed that a sheet may also have the name local_table) determines whether or not a chart uses own data or data from the container document.
In the old chart implementation, the chart always reads the internal data and uses it. If the data comes from the container, the container has an additional attribute that contains the complete data-range as one string. It then sets this string at the chart, which gives the chart the knowledge that data comes from outside (actually, in the old implementation, there is always local data. If data in Calc changes, the Calc prepares an SchMemChart object that contains the new local data and sets this object at the chart).
In the new implementation, we must to get rid of the badly-designed approach, as we no longer have one complete address-range for the entire chart.
Solution: We have to introduce a new XML-attribute at the chart that identifies the data provider. This may be a flag denoting "own data" or "external data", or even a URL, or the like, that uniquely identifies the data provider document.
Problem: Currently, it is necessary that when loading a chart, the container document is set as XParent before the chart itself is loaded. Otherwise there is no data provider, and the ranges are not valid for the internal data provider, and therefore are forgotten. A later setting of the data provider does not re-establish the correct range-strings.
Update of the Chart on changed Data
When a chart gets its data from the container document, and the data changes, the changes are notified via a listener mechanism. That is, when the chart queries for data at the XDataProvider, it gets XLabeledDataSequences. At those XLabeledDataSequences it starts listening for changes. If a change event is fired, the chart sets itself as modified, which forces the view to repaint, as it listens to model changes exactly via this mechanism.
However, if a Calc document containing Charts is loaded, the charts are either not loaded immediately because they are not visible, or because the contain a replacement image that shows the last rendered view.
In both cases, changes in the spreadsheet, like a change in data (which should be visible via a newly rendered view) or moving around ranges (which must change the ranges that are stored in the model), are not propagated to the chart, as it is not loaded and therefore is not listening.
In the old implementation, again, the fact that the Calc knew the data-range for each chart, solved this problem. Because the Calc could load all unloaded charts affected by the change, and then notify it. However, this requires internal knowledge of the chart to be exposed to the container document.
Solution: When an OLE-object is loaded, it only shows the replacement image. This state is called the "LOADED" state. This means only a small "stub" is loaded, not the entire object. The XModel of an object (including the entirely applied content of the XML-streams) is loaded in the "RUNNING" state. So when all charts would initially be set into the RUNNING state, this would ensure the existence of all XModels, which would allow the listening to work.
See also: Chart2 Single-Click Concept. There, the issue of loading models at the right time is also handled.
Problem:: It has to be evaluated, if loading the XModel objects of all charts in a document might take too long in loading a document, or if it consumes too much memory.
Context Menus via Framework API
The ChartController implements the XDispatchProvider that allows to find out whether menu entries are enabled or not via queryDispatch. The menu is created via an XML file that is defined in chart2/uiconfig.
The content of the context menu, of course, depends on the context, so a generic mechanism (like an xml-file) for defining it may not always be possible, but in most cases is. Anyway, it should be possible to use context menus via UNO API, and also share the XDispatchProvider to find out if certain commands are available or not. Disabled menu items are not shown in context menus, but this should be handled by the implementation of the popup-menu.
Currently, the menu shows icons next to some menu entries, but the context menu doesn't. It should not be necessary to take care of the icons in the applications. They should work also via the framework.
Solution: In the long run, the context menus should also be possible to create via xml-files. Maybe with the possibility to dynamically extend them with more entries (maybe needed for a list of corrections for auto-correction of mis-spelled words). As a short-term solution, there will be a function offered by the framework to pass an XPopupMenu and an XFrame which will take care of enabling commands via the dispatch found at the frame.
Range Selection Component
In the chart there is a dialog for assigning range-addresses to data series, categories or to the whole chart. The range-addresses must be in the correct format of the XDataProvider. Instead of typing tediously long and cryptic strings into the edit-field, a user rather wants to use the mouse (or keyboard) to select the ranges visually in the Calc.
For this purpose the XDataProvider offers a method to retrieve a sheet.XRangeSelection via the getRangeSelection() method. This component provides a little dialog that contains the range-address and the possibility to select ranges with the mouse (which are then put into the dialog's edit field). The content of the little dialog can be queried by the chart (which opened the range selection tool) and it can then put the string into the edit field in the chart's own dialog.
The problem here is, that charts are OLE-objects. The chart has to be in the UI_ACTIVE state to be able to open the dialog. When you press the button to open the range selection tool and click into the Calc spreadsheet, this action results in an automatic deactivation of the chart to the RUNNING state (which closes the dialog?) and activation of the Calc, which means the Calc is now UI_ACTIVE.
To be UI_ACTIVE means also that the content of the menu bar as well as the existence and content of toolbars changes. In the case of the toolbars, this may also lead to a shift of the visual area of the document, and therefore a shift of the chart object.
What we need here is a way to stay active in some sort in the chart (maybe ACTIVE instead of UI_ACTIVE) and be still able to select a range of cells in the Calc.
Calc- and Writer-Related Problems
The com.sun.star.table.XTableCharts object is based on the old chart
When you want to insert a new chart into Calc or Writer, this can be done via the com.sun.star.table.XTableChartSupplier interface that returns an XTableCharts container. At this container there is a method:
void addNewByName( [in] string aName, [in] ::com::sun::star::awt::Rectangle aRect, [in] sequence< CellRangeAddress > aRanges, [in] boolean bColumnHeaders, [in] boolean bRowHeaders );
You need a sequence of CellRangeAddresses to give a range and the parameters whether to use the first column or row for labels. In comparison to the XDataProvider method:
XDataSource createDataSource( [in] sequence< ::com::sun::star::beans::PropertyValue > aArguments );
the parameters in aArguments differ in the following way: instead of sequence< CellRangeAddress >, we only have one string. This is no problem, as there exists a conversion. But the parameter "DataRowSource" is missing. So you can only say if the first column or row is used for labels, but not if the data comes from rows or columns. Without this information, the other information is also useless. This renders the entire interface useless.
As a work-around, you can create the chart with some dummy data, and then attach new data using the data provider interface. However, a new interface, probably one that only gets the name and the rectangle would be more appropriate than the work-around.
Live-Preview when inserting a new Chart
In the old implementation, when a chart was inserted into Calc or Writer, you got a dialog that had its own preview in a little window. After pressing OK, the chart was created and inserted into the document.
In the new implementation, the container inserts a new chart first, then creates the wizard dialog (via UNO) and shows all changes directly in the newly inserted chart. When a user presses OK, everything is fine. But when a user presses Cancel, the inserted chart has to be removed again.
This approach is new to the applications. You insert an object (modifying the document), change it, and after pressing Cancel, you might get the unmodified document without the chart you had before. Currently, the implementations create Undo-Actions for inserting a chart. After pressing Cancel, you might still have an Undo or Redo action to get the chart back. That is not what you would expect at this place, although the undo-action might be the correct object to use when you want to make the chart insertion undone. Maybe, it is sufficient here to remove the undo/redo action from the stack of the undo-manager.
Drag & Drop in Calc: Identify Drag-Source
If you want to drag some selected cells in a spreadsheet and drop them on a chart, so that the chart uses this new data, the chart will get an XTransferable. The format that is contained in the XTransferable is currently the DDE format. The chart has to make sure that the dropped content comes from the same document or the same XDataProvider in particular. To do that, there must be a possibility to find out if an XTransferable belongs to the XParent of the current chart, or the XDataProvider which is attached.
One idea was to compare the "Topic" part of the DDE message with the title of the Calc document's window. Currently this is the same. At least during Drag & Drop this should not change either. So, we would need a way to query the title of the parent XModel in exactly the format that is contained in the DDE topic (for an unsaved document the name is Untitled1, Untitled2, ...).
Maybe there is a better approach. Something like
XDataProvider::isValidTransferable( [in] XTransferable xObject )
This way, the data provider can implement what fits best to find out if the XTransferable comes from the same document.
Drag & Drop works only in ACTIVE mode
Drag & Drop only works when the chart is always ACTIVE. So, unless charts will be always ACTIVE, drag & drop does not work, as the drop target is not the chart but the surrounding embedded object. Alternatively, we need a mechanism that forwards the drop-request to the underlying OLE-object, which would require activation at least in this moment. This could be flag-driven, to avoid loading all OLE objects when hovering over them with drop content.
Graphic-Framework-Related Problems
The type of the com.sun.star.drawing.BitmapTable
For gradients, transparency-gradients, hatches, line-dashes and bitmaps there are tables in the chart model that contain those elements together with names. E.g., you may have a gradient with the name "My Gradient". When you set the "FillGradientName" Property of an object to "My Gradient", the object will display the corresponding gradient found in this table. The same holds for fill-bitmaps.
However, there is a problem as the table for bitmaps maps a name to a URL rather than to some real graphic object. So, when you add an element to this table, the graphic itself isn't used anywhere in the document at this time. As we have a graphic-manager that deals with all graphics, that keeps graphics only if they are used (if there is a refcount > 0), the grapghics are dropped in this case.
Solution: Change this map to a mapping from names (strings) to objects of type com.sun.star.graphic.XGraphic. As the existence of an XGraphic object ensures that the graphic is available in the graphic-manager, this will solve this problem. Only in the moment an element in this list is used in no model object and also removed from the list, the underlying graphic will be dropped by the graphic-manager.
- see Issue 66558. Kai Ahrens works on this now. He will change the type of the bitmap table for all applications, as we now have the XGraphic API.
Chart Problems
Autoscaling of Text relative to Diagram Size
In the old chart implementation, when auto-scaling was on, a resize of a chart OLE object or the resize of the Window of a standalone chart resulted in a resize of the actual font sizes in the model, i.e. the model was changed. In the case of an OLE object this is ok, as this also changes the visual area, which is also a model property. However, resizing a document window should not modify the model.
To overcome this bad design, in the new chart we decided to add a reference size to font sizes in the model. So when auto-scaling should be on for an object, the size of the page (the whole chart, which equals the visual area size) is set as ReferencePageSize along with the font. So if the chart is resized, the view can calculate a new font size by comparing the new size of the page with this reference size and adapt the stored font size. So the font becomes bigger without having to change the model (apart from the fact that a resize of the OLE object modifies the model, because the visual area size is changed). If a font should not be scaled, the reference size is simply set to void. So far this concept is very good. It also solves the problem of getting the exact same font size after resizing the chart to an extreme (e.g. very small) and back to the original size again, which was a mess in the old chart.
However, there are objects where the fonts are supposed to scale with the diagram size rather than the page size, e.g. the axes. The problem here is that the diagram size may be void, which means the view has to calculate it by itself. The actual size of the diagram is therefore not known unless a view is created that does the auto-calculation. So, if you want to enable auto-scaling for an axis in a chart, you would have to set the actual diagram size as refrerence size at the model axis object. However, you don't know it when the diagram has an auto-size.
In the file-format we should also have the currently visible font size plus a boolean flag saying whether an object should auto-scale or not. So, here we have a similar problem. We would need to have a view that calculates the size of the diagram (if it has auto-size), after a chart was loaded, in order to translate the boolean flag into a fitting diagram reference size. And also for saving, we would need the current diagram size for calculating the currently visible font size.
So, for case that the diagram is auto-sized, to avoid having a view for switching auto-scale on for diagram-dependent font sizes, as well as saving and loading of files we need a solution here.
Solution ideas:
- In case the diagram is auto-sized, set the page size as reference size. Unless the diagram is resized, the mechanism would then work for resizes of the whole chart like before. The font size written to the file would be the real font size in the model, which is ok. The view would have to use the page size for comparison when it finds out that the diagram's model has no size.
- What happens if a user resizes the diagram? In this scenario we would need a mechanism that sets the currently calculated auto-size of the diagram (before the resize) as a fixed size to all model objects that use a diagram reference size and have the page size set as reference. Then it has to set the new diagram size at the diagram. This is error prone, as you might forget objects. Also, comparing the current reference size with the page size is also not very elegant.
- Drop the feature of diagram-related font sizes. Use only the page as reference size. This would simplify things but would break the feature of getting smaller fonts when you resize the diagram inside a chart document. I currently prefer this solution for different reasons.
- First, we only have one concept: a reference page size, not reference sizes that refer to different objects.
- Second, we always know the page size, as it is equal to the visual area size of the model. Apart from that an auto-page size wouldn't make much sense. So, we solve this problem.
- Third, the down-side of doing this is, that resizing the diagram would not scale the fonts at axes, (axis titles?) and data points. But on the other hand, a chart document should layout the chart in a nice way, so that a user should not have the need to resize the diagram. If a user still wants to change the diagram size, he can change the font sizes manually. I think this is a valid compromise, as I think this does not happen too often. Also, for small diagram size changes a non-scaling font doesn't matter too much.
- It simply does not make sense to set a reference size that is not found at the correspodning object (the diagram) in the model, but can only be obtained by a view. If you replaced your view implementation or you changed the auto-calculation of the view, you will get different results, so it is not reliable.
- On resize change the model.
- Setting re-scaled fonts is a bad idea as the old chart showed. You get rounding errors with sometimes drastic results.
- It would be ok for OLE objects, as they are modified anyway (the visual area size)
- If we would introduce a stand-alone chart some day, we would have to change the behaviour similar to draw or writer: The chart has a fixed page size and resizes of the window only change the visual area of the document. However, this conflicts with the use of the visual area as page size in the OLE scenario, where we definitely want to have the page size equal to the visual area size.
- Therefore, this is seems like a bad idea.
- Do not allow an empty diagram size if there are objects with DiagramReferenceSizes.
- This is also no good idea, as the diagram would need knowledge about all objects with diagram reference sizes, to behave accordingly on setting the size.
- When we want auto-resize as default, this would also mean the default would be to have a concrete diagram size, which is not reasonable.
So, summarizing, dropping the feature of fonts scaling with the diagram seems the most appropriate thing to do. Of course we will lose a feature and have to decide if it is ok for users to no longer have this. As I said, I think resizing the diagram to something very much smaller than the one you get with auto-calculation is not a very common or typical scenario.
(article by Bm@openoffice.org)
Chart Types
This section will describe the various chart types and also provide possible solutions for implementation. It will also try to establish a realistic priority list.
This list is based primarily on the List of wished enhancements for Charts (see http://graphics.openoffice.org/files/documents/12/3278/List_of_whished_enhancements_for_Charts.odt). I will also present some entries not covered in the original list. Some other issues are detailed on the statistics wiki page, too (see http://wiki.services.openoffice.org/wiki/Statistical_Data_Analysis_Tool).
Intro: External Programs
Implementing all these issues directly into OOo will be both difficult und unnecessary. It is not always pertinent to reinvent the wheel and I will present later a better solution.
Why not implement everything in OOo?
DISADVANTAGES
- many resources are needed (coders, people to test, even financial constraints and time delays)
- code becomes more and more complex, more difficult to understand by new developers, more testing time needed
- most experienced coders work already on other open source projects; less availability for a new project
ALTERNATIVE
- a very good alternative is to do all these advanced things with dedicated external software, when such free alternatives exist, e.g.: (see also Links section below)
- gnuplot
- R (with over 500 packages), octave and many more, each suitable for some specific task.
ADVANTAGE
- once the external program is embedded into OOo, one can easily implement many more features (from the specific program) with minimal effort
- dedicated programs offer more advanced solutions than any OOo implementation (or even competitive software)
- less resources needed:
- programs are developed by their own groups,
- tested by appropriate folks and therefore
- NO major delays, less propensity for bugs
- new features are implemented more easily (we at OOo need only make a new Menu/Gui and paste the correct syntax/call to the external program)
- smaller code, smaller program
- only users that need that option will run it (see also the note below)
- most other users will not have it installed, therefore OOo should also run faster; not so many resources loaded
Another strong reason why external software is a better alternative:
If the coders aren't working with people that know how to do numerical methods then what are the odds that it'll even come out correctly [quote from a user]
A special note:
- because some licenses may not be compatible with OOo (although they are still open source), I DO NOT MEAN including the code in OOo
- what I mean is a general mechanism in place that allows OOo to communicate (bidirectioanly) with the external software
- instruct users what external programs (extensions) do exist and where to find them (provide url)
- allow users to easily acces the functionality of external programs through OOo Menus (or functions)
Chart Types
These are mainly covered in the List of wished enhancements for Charts (see points 6.a through 6.m). There are a number of other chart-types NOT covered in that document, and I will briefly describe those which are critical:
- Histograms: high priority; this is further discussed on the stat wiki page;
- Overlapping Data: Venn Diagrams;
- high priority;
- While there is a workaround for many current issues, there is NO method to display overlapping data in OOo;
- see stat wiki page and wikipedia (http://en.wikipedia.org/wiki/Venn_diagrams)
- Gantt Charts:
- see point 6.l in original list and also http://en.wikipedia.org/wiki/Gantt (including the External links - Software section)
- R-packages: plotrix; Gantt Project: http://ganttproject.sourceforge.net/
- Geographical Maps and Map Data:
- see point 6.m in original list
- useful programs GRASS and R (including packages GRASS and gstat)
Other chart types (less priority):
- see also http://addictedtor.free.fr/graphiques/allgraph.php
- Cluster Analysis: see also stat wiki page
- Dendrograms
- Regressions:
- Effect Plot for Generalized Linear Models: http://addictedtor.free.fr/graphiques/RGraphGallery.php?graph=124
- Quantile Regression: http://addictedtor.free.fr/graphiques/RGraphGallery.php?graph=109
- Various Boxplot Derivations:
- Various Scatterplots
- Climate Diagrams:
- climate diagrams are brief summaries of average climatic variables and their time course
- see http://www.zoolex.org/walter.html and http://addictedtor.free.fr/graphiques/RGraphGallery.php?graph=122
- Others: e.g. http://addictedtor.free.fr/graphiques/RGraphGallery.php?graph=131, http://addictedtor.free.fr/graphiques/RGraphGallery.php?graph=96
Links
see also:
- statistical wiki page: http://wiki.services.openoffice.org/wiki/Statistical_Data_Analysis_Tool
- original list: http://graphics.openoffice.org/files/documents/12/3278/List_of_whished_enhancements_for_Charts.odt
External Software
- R-project: http://www.R-project.org/
- gnuplot: http://www.gnuplot.org/
- GRASS: http://grass.itc.it/
- gstat: http://www.gstat.org/index.html
External Chart Types examples
Documentation
Some useful information
- Main Chart Project Page: http://graphics.openoffice.org/chart/chart.html
- Chart Specifications: http://specs.openoffice.org/chart/index.html
- Current CWS Name: chart2mst3
- IRC channel: #chart2.openoffice.org on freenode
- Björn's Weblog (which also shows up at Planet OpenOffice.org)