Mac OS X Porting - OpenGL transitions

OpenGL transitions on Mac OS X

Description

In Impress, there are transitions between slides, with or without funny 3D effects. The problem is, those nice and Funny effects are power consuming, and without hardware acceleration – means software rendered –, extremely slow, what is not acceptable by users.

This task aims to replace the software rendered transitions with hardware accelerated transitions on Mac OS X.

We'll use OpenGL, the API Mac OS X does natively use

Below the technical description, the involved code, the plan, and so on. Feel free to add information if you estimate it usefull for the task.

Thanks in advance :-)

Goal

Make OpenGL transitions work in Impress, on Mac OS X Aqua (on both PowerPC and Intel architectures)

Timeline

Expected timeline for the cws to be ready for QA : ~~October / November 2008~~ May 2009

Current status

9th March 2009

new subdir scheme, where major OS's will be separated :

slideshow/source/engine/OGLTrans/mac 
slideshow/source/engine/OGLTrans/win
slideshow/source/engine/OGLTrans/unx

ogltrans4mac has been rebased with DEV300_m42
transogl03 (integrated in DEV300_m42 ) has been integrated. Everything is broken, and needs to be evaluated.

18th January 2009

catching with color space issue ...
build m39 + the new pen + cairo. Found color space and bad views areas issues with cairo. The rest seems to work

16th January 2009

extracted the pixel shaders patch from Radek Dulik/ Thorsten Behrens go-oo patches, will apply them to the cws
extracted the fixes for the slides Radek Dulik/ Thorsten Behrens go-oo patches, will apply them to the cws

26th September

One issue verified on 42" screen : during OpenGL transitions, the view does not use the fullscreen area, but the Windowed view area (needs to check the exact area) -> work in progress, + some limitations (unimplemented features), explained below.

7th September

Excepted the flicker (caused be the slideshow itself) the transitions work. We have two working solutions, and a choice for the implementation will probably be done soon ( waiting from ssa, who will decide what is the best )

Remaining tasks:

build issue in officecfg: the .xcs and .xcu files have to be modified when ENABLE_OPENGL is true
check for memory leaks
check for stability

31st August

FIXED: Fixed the colorspace issue: was the pDetectedFormat

TODO:

Cleanup in void OGLTransitionerImpl::GLInitSlides()
Continue to search what happens with the frame (why this sort of "refresh ?" )
Work in progress: started learning the slideshow

30th August

The issue with flicker seems to be caused by the slideshow itself. I'll add symbols in libsd, and I'll try to see what happens.

concerned code is in: sd/source/ui/slideshow

The color issue seems to be caused by some color/alpha channel ( RGBA .. RGAB ) or something like that issue.

Shane M. Mathews (I'm in contact with him) suggested several possibilites. I'll give them a try.

Issue 0 was not an issue. OpenGL version detected is at least 1.3, but can be more.

TODO: test on Leopard, just in case the issue is Tiger only (that would be a bad news)

29th August

Re-reading OpenGL Programming Guide. Tested a lot of possibilities with NSOpenGLContext, unsuccessfully.

Radek Dulik adviced me to check whether the context is really Double buffered (was an issue on Linux). Maybe a missing makeCurrentcontext somewhere (where ? )

Analyzed the known issues:

0) solved: OpenGL 1.2 or 1.3 detected (2.0 should work imho)

1) bad colors (bad OpenGL initialization?) Looks like the complementary color appears, of red effect added. Needs more work

2) offset in window mode, caused by scrollbar not included in the slide position computation (the frame dimensions are different)

3) flicker

the problem is (using ssa words):

the slideshow displays the slide first
then an opengl view is painted on top of it and plays the slide transition
once it is removed the old background appears again still containing the first frame of the animation
and after a very short time the slideshow engine draws the final frame,
which is visible as a flicker

4) optimization / compatibility?

28th August

Works (experimental and hacked builds are available at Laurent Buisson site).

There are currently no crash, but it needs improvement:

bad colors
flickering at the end (will need precise and non trivial work)
no optimization

other tests in progress: replace the NSOpenGLView, with a customized NSView ( sublassed as NSOpenGLView, with more possibilities)

cws state changed for new

added tasks and modules to the cws. First commits to come soon.

=> Important: don't forget to run autoconf in config_office, to regenerate configure (configure.in has been modified)

27th August

cws ogltrans4mac has been created (planned)

Stephan Schaefer helped me ( he made the NSOpenGLView work some weeks ago), and including all the changes, I was able to package a working version!

Created the patches concerning:

scp2,
config_office (René confirmed it is ok to bypass OpenGL / gl.h, glut.h headers and GLUT tests, and declare ENABLE_OPENGL=TRUE as default on Mac OS X),
officecfg,
sd,
slideshow

Waiting for more information, about how proceed in vcl (I now know it works, but a big issue remains to be fixed :-) ).

26th August

Found why the libOGLTrans was not loaded: the registration as UNO_COMPONENT was not done.

Other issues : the new transitions effects are not listed, because not in the .xcu nor any .xml

I finally found where: in sd/xml! -> transitions-ogl (who is a plain .xml file) has to be

renammed into transition-ogl.xml
copied into the solver (accordingly to th esd/prj/d.lst file)

Consequence : some changes are needed in officecfg too:

Existing transitions are put in officecfg/registry/data/org/openoffice/Office/UI/Effects.xcu

-> new entries, matching with transitions-ogl.xml have to be entered (probably around line 1910, after "Random Transition" effect)

FIXME : localisation of new transitions names?

25th August

Reading documentation (vcl code, mostly about outdev and co)

Defined the implementation plan (working with DEV300_m29)

24th August

Did some tests, using the first aquaOpenGLView I created. Lot of crashes.

23rd August

Added other ideas

22nd August

Started the wiki page and the implementation in parallel

PLEASE DO NOT REMOVE THE CREATION DATE

Page created Ericb 11:55, 22 August 2008 (CEST)

Concerned domains

Build :

conditional activation of ENABLE_OPENGL in configure
addition of NSOpenGLView class in vcl
conditional build of libOGLTrans.dylib in slideshow
conditional packaging of ligOGLTrans.dylib in scp2
conditional addition of new transitions in the officecfg schema/UI files
conditional registration of libOGLTrans.dylib at packaging time

Core :

NSOpenGLView implementation in vcl
OpenGL initialization and context creation in the slideshow
load the libOGLTrans.dylib

User Interface :

add the list of the new transitions in the list
use them

Concerned modules

config_office: make OpenGL default on Mac OS X Aqua => ENABLE_OPENGL=TRUE
slideshow: where the lib is built. This lib is linked with libvcl (vcl must be built before slideshow)
scp2: to add the new lib in the package
sd : rename the transitions-ogl in transitions-ogl.xml and deliver it
officecfg : add the new transitions names in the list, to make them appear in the UI + add the transitions-ogl.xml file name in the matching Effects.xcs schema

Examples

To illustrate the current work in progress, some experimental screenshots are available => CLICK ME

Feedback

The page below, is the place for user experience comments, suggestions and feddback.

PLEASE use it and contribute! click me

Modifications

Existing (historical implementation)

Common:

vcl/source/opengl.cxx, implements:

functions pointers (most important gl functions typedef'ed)
+ corresponding macro for functions initialization
+ impl functions getters
Ctor: parameter pOutDev is a pointer on a OutputDevice*, using mpOutDev
Dtor: deletes mpOGL
usual methods ... and so on

vcl/source/salogl.cxx:

Static members: oslModule (himpOGLLib == libgl.dylib), NSOpenGLContext* .. (bool) state
Macros
Callbacks for CreateContext, DeleteContext, MakeCurrent, GetCurrentContext
OpenGLWndProc (what's that ?)

vcl/source/gdi/outdev.cxx ->there is a very interesting OpenGL* OutputDevice::GetOpenGL()

FIXME: To be continued

Concerned files / ideas

Ideas

27th August

Created the patches concerning:

scp2,
config_office (René confirmed it is ok to bypass OpenGL / gl.h, glut.h headers and GLUT tests, and declare ENABLE_OPENGL=TRUE as default on Mac OS X),
officecfg,
sd,
slideshow

Should be the only concerned modules. I used the OpenGL transitions listed in sd/xml/transitions-ogl.xml, but there are obvisouly more in slideshow/source/engine/transitions.

Waiting for more information, about how proceed in vcl (I now know it works, but a big issue remains to be fixed :-) ).

26th August:

I finally found why the libOGLTrans was not loaded: the registration as UNO_COMPONENT was not done.

Other issues : the new transitions effects are not listed, because not in the .xcu nor any .xml

I finally found where: in sd/xml! -> transitions-ogl (who is a plain .xml file) has to be

renammed into transition-ogl.xml
copied into the solver (accordingly to th esd/prj/d.lst file)

Consequence: some changes are needed in officecfg too:

Existing transitions are put in officecfg/registry/data/org/openoffice/Office/UI/Effects.xcu

-> new entries, matching with transitions-ogl.xml have to be entered (probably around line 1910, after "Random Transition" effect)

FIXME : localisation of new transitions names?

25th august:

After reading a lot of code (mostly vcl), it appears this is not possible (excepted doing hacks), to use OpenGL without use outdev thing (got systematic crashes using NSOpenGLView directly, and outdev complains about lost events and pending pointers)

The consequence is, the slideshow calls must be abstraction of CreateContext, MakeCurrentContext, ReleaseContext ..etc.

And the real implementation must go in vcl/aqua/source/gdi/salogl.cxx, respecting vcl/source/gdi/opengl.cxx virtual classes.

The plan (to be confirmed seems to be):

design the abstraction we'll use, and prepare the OGLTrans_TransitionerImpl.cxx like it is for Linux, but adapted to Mac OS X
implement NSOpenGLView* pOpenGLView in vcl, guessing it will be sufficient (else, we'll have to implement a new class, but things will become complicated)
define the NSView containing the scene, instantiate an OpenGLNSView using the same frame, and use addSubview. Current choosen Place to do that: OGLTrans_TransitionerImpl.cxx
implement (using existing Windows and Linux implementation) the same classes / macros ( :-/ ) methods in salogl.cxx
add the mac osx cases in outdev and anywhere else it is needed

Retry with the coming DEV300_m30

Possibilities:

Everything is based on implement an OpenGLView adding a new NSOpenGLView* pOpenGLView object in sysdata.hxx, or an NSView drawn as a subview of the (NSView * type) pView.

TODO: investigate about ImplOutdev and co

Try 1: either implement (NSView * type) pOpenGLView in vcl, more precisely, modify:

vcl/inc/vcl/sysdata.hxx

vcl/aqua/inc/salframe.h

vcl/aqua/inc/salframeview.h,

vcl/aqua/source/window/salframe.cxx

vcl/aqua/source/window/salobj.cxx

+ the makefile

and in slideshow:

slideshow/source/engine/OGLTrans/OGLTrans_TransitionImpl.hxx

slideshow/source/engine/OGLTrans/OGLTrans_TransitionImpl.cxx

slideshow/source/engine/OGLTrans/OGLTrans_TransitionerImpl.cxx

Issue is at buildtime:

Pb: visibility of the vcl/aqua/inc headers from slideshow at buildtime

Update: predeclaration in vcl/inc/vcl/sysdata.hxx will help a lot.

Main issue is, how to explain the compiler NSView* pOpenGLView has new methods (OpenGL specific), the NSView* pView has not ?

More precisely, there are warnings because the pOpenGLView is supposed receive no answer from the OpenGL specific methods used in OGLTrans_TransitionerImpl.cxx. In theory, it should work out of the box, but the code aims to be warning free; so this solution is not acceptable.

Workaround could be implement the new methods directly in the NSView, or delegate. Just wild guess [FIXME]

Try 2 or

(maybe completely wrong, but worth a try)

Implement the OpenGL methods directly in the SalframeView. just two methods have the same names/parameters numbers, so just rename they properly to avoid clash. Important: initWithFrame for OpenGL has two parameters, while initWithFrame for normal NSView has only one.

Try 3 or

Add an NSOpenGLView * pOpenGLView in the sysdata.hxx, and modify everything in vcl accordingly.

==> click here to have a look at NSOpenGLView available methods

Thus, in OGLTrans_TransitionerImpl.cxx instanciate

an NSView who is a pointer on the normal NSView, and an NSOpenGLView, who is a pointer

or

create the AquaOpenGLView object directly in slideshow, modify the other slideshow files mentioned above (changes are similar)

Result

Using the second solution, the subview seems to -somewhat- show something.

Test: a right click in the transition demo, using the presentation wizard, give the same contextual menu we have in fullscreen (yes, this is weird, but this is encouraging for the future).

Known issue: there is no communication between the subView and the NSApplication, events are lost, timeout and chaos in event queue

-> deadlock and crash.

TODO: Second test will consist in delegate the notifications send for the refresh, to the NSApp.

It is expected, once this NSOpenGL view will work, this should be no problem to use it from the slideshow.

Communication between slideshow object and the NSApplication (customized NSView, itself inheriting of NSWindow) will use the notification center as proxy (what is extremely fast in theory).

Tasks and concerned files

Make ENABLE_OPENGL default on Mac OS X

config_office/configure.in + run autoconf Add the info in the cws comments for the RE !

Result: ENABLE_OPENGL=TRUE on Mac OS X Aqua // OK

Task: build the lib

slideshow/source/engine/OGLTrans/makefile.mk // OK

add the case Aqua, modify CFLAGSCXX + add fremaworks OpenGL, GLUT and Cocoa for linking

Task: Deliver the libs / includes:

slideshow/prj/d.lst // OK

deliver the new lib on Aqua (libOGLTrans.dylib)

Issues attached to the task

issue 93013

TODO (draft)

[DONE] build the OpenGL trans lib, warning free, on Mac OS X
- [DONE] fix warnings and breakages, with empty implementation
- [DONE] fix linking
- [Done but needs work] create customized NSView (skeleton, and pre-implementation)
[DONE] add the lib to the package
[DONE] register the lib
[DONE] rename the transitions-ogl in transitions-ogl.xml in sd and deliver it properly in the solver
in officecfg, add the new transitions names in the list, to make them appear in the UI
[DONE] in officecfg, add the transitions-ogl.xml file name in the matching Effects.xcs schema
[DONE] Design the first implementation
- design the abstraction used in slideshow
[DONE] implement NSOpenGLView or customize an NSView [Current plan: try using NSOpenGLView first]
- manage OpenGL context
- manage OpenGL pixels mapping
- manage colors
- manage events and notifications critical
- initialize the view properly in splitted view
- manage geometry
- [WORKS] implement the fullscreen mode
finalize the transitions integration in Aqua bundle
[DONE] make transitions work
work on the flickering issue with last image
test
integrate

LINKS

Below a now exhaustive list of links. Feel free to add interesting links.

Original Shane M. Mathews OpenGL transitions implementation (GSoC 2007, mentor: Thorsten Behrens)

Apple OpenGL Documentation

Cocoa OpenGL

Custom GLView

NSView + OpenGLView in Apple database

NSOpenGLView initialisation issue

Name	OOo Nickname	Role
Eric Bachard	ericb2	Development, Documentation
Stephan Schaefer	ssa	Development
xxxx	xxxx	User Experience
xxxxxx	xxxxxx	Documentation

COMPLETE ME