Difference between revisions of "User:TerryE/phpBB3.0.5 Migration/Follow-up Issues and Actions"

From Apache OpenOffice Wiki
Jump to: navigation, search
(Automated Unactivated Users Prune)
(Caching Setup and Policy)
Line 28: Line 28:
 
== Caching Setup and Policy ==
 
== Caching Setup and Policy ==
  
:''Main Article'': [[User:TerryE/phpBB3.0.4_Migration_—_APC_Tuning|APC Tuning]]
 
 
I have enabled the APC (Another Php Cache) cache for the site.
 
I have enabled the APC (Another Php Cache) cache for the site.
  
Line 45: Line 44:
 
  apc.enable_cli=1
 
  apc.enable_cli=1
  
APC seems to be working fine apart from the setting ''apc.stat=1'' doesn't seem to work in CSK 1.3.1It worked fine in 1.3.  It now seems to be testing the ctime rather than the mtime (See [http://forums.sun.com/thread.jspa?threadID=5387577 APC change of behaviour on CSK upgrade] for more details)The phpBB cache works <sup>(+)</sup> because it deletes old files and the writes new ones, so the ctime is updated as well as the mtime.
+
APC seems to be working well apart (from one gremlin)You can see how effective this is by executing a temporary APC query [http://user.services.openoffice.org/apc.php?files=yes usOOO &mdash; APC Cache Summary].  I just ran this report and on a 47hr window since last refresh we are running at around 170K cache reference/hr and of these we have a 99.4% cache hit rate.  The figure is even bigger when weighted by code size.
  
You can see how effective this is by executing a temporary APC query [http://user.services.openoffice.org/apc.php?files=yes usOOO &mdash; APC Cache Summary].  I just ran this report and on a 47hr window since last refresh we are running at around 170K cache reference/hr and of these we have a 99.4% cache hit rate.  The figure is even bigger when weighted by code size.
+
The gremlin is that ''apc.stat=1'' doesn't seem to work in CSK 1.3.1, though it did work fine in 1.2. (See [http://forums.sun.com/thread.jspa?threadID=5387577 APC change of behaviour on CSK upgrade] for more details)The phpBB cache functionality ''is'' impacted by this because opening an existing file for write does not update the inode timestamp  works, because it deletes old files and the writes new ones, so the ctime is updated as well as the mtime.  This has triggered me to look at more detail at the whole caching design in phpBB.  I am believe that some of the funnies we are seeing which cache persitence are an artefact of this. Need to do some more checking
  
: (+) We are seeing some funnies which I am beginning to suspect is an artefact of this. Need to do some checking
+
:''Main Article'': [[User:TerryE/phpBB3.0.4_Migration_—_Cache_Tuning|Cache Tuning]]
  
 
== Forum Permissions and Access ==
 
== Forum Permissions and Access ==

Revision as of 07:12, 22 May 2009

Follow-up Issues and Actions

I have included the list of issue in alphabetic order below.

Attachment Quotas (Closed)

I set the attachment quotas on the EN and FR boards as 300Mb and 500Mb respectively, with the other boards are set at 100Mb for now. Unfortunately I used the values 300Mb where this should have been a number which resulted in attachments being disabled. The first time I fixed it, I had finger trouble on the update (accidentally typing a ";" before the where clause in a update has a disastrous result :lol:).

"Oups, seems there is a problem in the counts bottom of page: all are set to 314,872,800 topics/posts/users!"

Another hour extracting and restoring phpbb_XX_config tables from the last backup and rolling forward the changes that I made.

Automated Mailbox Robot

Main Article: Automated Mailbox Robot

I now have this working for each forum, subject to limitation that the Admin Mailbox Configuration post must be correctly configured. There seems to be something wrong with the forum permissions propagations. I have used the workaround of using my administrator powers to adopt a moderators permissions so I have now (as at UTC 02:08, 18 May 2009) set up all of the forum mailboxes and tested each by sending a test message to each. The NL admins mostly have some more playing/tweaking of the configs but the robot is broadly working.

ACP Actions Hangs

This was raised by Acknak:

I've noticed a consistent glitch with banning users. Once I fill out the form and click "Submit" I never get a response back. The browser just sits, with status "Waiting for user.services.openoffice.org". I've let it sit that way for several minutes, but never see the "success" page come back. However, if I cancel the page load at that point, go back and re-load the banning form, te ban list has been updated with the user ID that I submitted, so I think the ban is actually successful.

I have also noticed with the delete Inactive accounts function.

Automated Unactivated Users Prune

This is another batch script that I have still in development. It automates a tedious twice weekly task. Low priority until the issues which are impacting the NL Administrators are sorted.

Caching Setup and Policy

I have enabled the APC (Another Php Cache) cache for the site.

apc.enabled=1
apc.shm_segments=1
apc.cache_by_default=1
apc.optimization=0
apc.shm_size=64
apc.ttl=240
apc.user_ttl=240
apc.gc_ttl=120
apc.stat=1
apc.num_files_hint=1024
apc.user_entries_hint=200
apc.mmap_file_mask=/dev/zero
apc.enable_cli=1

APC seems to be working well apart (from one gremlin). You can see how effective this is by executing a temporary APC query usOOO — APC Cache Summary. I just ran this report and on a 47hr window since last refresh we are running at around 170K cache reference/hr and of these we have a 99.4% cache hit rate. The figure is even bigger when weighted by code size.

The gremlin is that apc.stat=1 doesn't seem to work in CSK 1.3.1, though it did work fine in 1.2. (See APC change of behaviour on CSK upgrade for more details). The phpBB cache functionality is impacted by this because opening an existing file for write does not update the inode timestamp works, because it deletes old files and the writes new ones, so the ctime is updated as well as the mtime. This has triggered me to look at more detail at the whole caching design in phpBB. I am believe that some of the funnies we are seeing which cache persitence are an artefact of this. Need to do some more checking

Main Article: Cache Tuning

Forum Permissions and Access

I am an admin on all forums and should therefore have edit access to all forums. If I go into the ACP->Manage Groups->Group Forum Permissions and check, the Admin group has full permission on all forums. However in some forums, e.g. es and fr, I don't have edit edit access to topics but I do on some such as en. I haven't bottomed out what the difference is yet. The relevant code fragment from viewtopic.php (prettied for readability) is:

  'U_EDIT'       => (!$user->data['is_registered']) ?  : 
                       ( ( ( $user->data['user_id'] == $poster_id && 
                             $auth->acl_get('f_edit', $forum_id) && 
                             ( $row['post_time'] > time() - ( $config['edit_time'] * 60 ) || 
                               !$config['edit_time']
                             )
                           ) || $auth->acl_get('m_edit', $forum_id)
                         ) ? append_sid("{$phpbb_root_path}posting.$phpEx", 
                                        "mode=edit&f=$forum_id&p={$row['post_id']}") :
                             ),

that is for other than the O/P, acl_get must return the m_edit privilege for that forum.

Until I can work this one out the work around is to pick a moderator and test his permissions.

Language Localisation

The out-of-the-box language packs contain some mistranslations, A number of the NL Administrators have posted changes asking me to correct various miswordings in their translations. It's a little error-prone for me to do as I usually don't understand the nuances of the text and its context. There is a feature (now working) whereby the administrators can now do these changes themselves through the ACP if that have founding administrator rights in you forum, but as most of you don't have this on your own database:

  • You log onto /forum/test database as TestFounder (pwd see closed post) and then through the ACP through the link at the foot of the page which is visible when you are logged in as an administrator.
  • You navigate to ACP->System->Language Packs. This lists the Language packs installed in your NL Forum. Each of the named packs is a URI; click on the pack you want to edit.
  • Under the Language entries there is a drop down allowing you to pick the file that you want to review / change. There are Language Files which relate 1-1 with the page URIs (e.g. posting.php is all of the NL strings specific to the posting form) and a common one for NL strings that are used on multiple forums. There are also templates for emails and registration. Make the changes that you need. One great advantage of doing it this way is that the encoding is UTF.
  • Unfortunately, you need to be a Founding Administrator to have these right, and the NL admins do'nt have this, so I have added an account to the /test/forum instance (name and password provided on our private forum). Go into this test forum to do this work. Since all the NL instances use the same code and language packs, you can do this work here.
  • When you are done click the "submit and download" button. This saves a copy on the server and downloads a copy to your PC which you can then review and edit (in a UTF8 enabled editor) at your leisure or pass to colleagues for them to update. For all but the simplest edits it is safe to do this download, because if you leave and back to the form you will start again with the installed version. (In fact you may just prefer to use an editor rather than the form interface in which case you select the file and click submit and download straight-away.)
  • When you are happy with the changes then email the files to me (TerryE at the mail domain OpenOffice.org) and I will install them for you pretty much as soon as I pick up the email.

As an aside, I had real troubles getting this to work  :crazy: It was a bastard to debug. (See phpBB — (ACP) Unable to submit large forms).

Given that I am the resource bottleneck at the moment, can I ask all NL administrators which outstanding translation change requests to use this mechanism.

Language Pack Inconsistencies

Even though the Langauge packs are shared, the corresponding language tables in the forums were brought across from the old databases and are therefore inconsistent. This doesn't impact phpBB at an applications level, but it would make sense to standardise these as a one off. (The only difference should be whether the individual packs are enabled for a given forum.)

Mail subscriptions aren't working (Closed)

You are supposed to be able to subscribe to a forum or post and this will trigger automatic mail notifications. This wasn't working. I set up a test harness using the test NL forum on the server to try this out. After a lot of tedious debugging, I discovered two issue, both of which I really regard as bugs in phpBB:

  • I had set up the forums so that the ACP->Email Settings->E-mail package size to 50. OK, the comment says "This is the number of maximum e-mails sent out in one package. This setting is applied to the internal message queue; set this value to 0 if you have problems with non-delivered notification e-mails." (My italics) However, when I drill through all the code the queuing and dequeuing functionality is implemented, but the dequeue is triggered by a cron.php hook which was never called.
  • The reason for this is that the common.php includes the following conditional which causes the cron.php task to fail in our shared code implementation, since this causes it to attempt to resolve the cache directory in /var/www/phpBB-common/cache which doesn't exist. What the logic for doing this, heaven knows, but my work-around is to reset $phpbb_root_path back after including this in the cron.php routine.
if (defined('IN_CRON'))
{
    $phpbb_root_path = dirname(__FILE__) . DIRECTORY_SEPARATOR;
}

The way the cron hook is executed is that the footer template includes the hook:

 <!-- IF not S_IS_BOT -->{RUN_CRON_TASK}<!-- ENDIF -->

The page_footer setup function has an optional parameter $run_cron which defaults to true, and in fact is always called page_footer(), so this cron path is executed on every page generation. The cron.php is a pseudo-cron task. This executes a logic cascade including the triggering of the processing email queue every 'queue_interval'. If it needed, it includes a image link to a dummy image which is one blank pixel, but in reality this triggers a callback from the browser to initiate the cleanup function. As most browsers implement a lazy image load during the page load, this doesn't impair the user experience.

I have put in this fix, but also set the email_package_size parameter to zero for now. I will review this for tuning and optimisation later, but for now this works.

More Information: Cron Tasks

Code Expand / Contract

Three minor known issues:

  • A forum user (see Code expand/collapse behaviour) has pointed out that we need a tweak to the client-side javascript to preserve the current focus when doing an expand / contract:
 <a onclick="selectCode(this); return false;" href="#">Select all</a>
 <a class="codebox" onclick="expandCode(this, true); return false;" href="#">Expand view</a>
 <a class="codeboxexp" onclick="expandCode(this, false); return false;" href="#">Collapse view</a>
  • The code is not picking up NL context for "SELECT ALL / EXPAND VIEW / COMPRESS VIEW
  • For this mode to latch, I need to refresh the style through the ACP.

Review of Apache Logs

I went through the Apache error logs to see what avoidable errors could be addressed:

  • The robots.txt file needed updating [Done]
  • The favicon.ico needs an explicit meta tag in the standard header [TBD]
  • The fr rss.php is generating a lot of misses [TBD]
  • The fr ci-joint database is generating a lot of hits to files of the format fichier/YYYY/MM/0000NNNNN_KKKKK.tempMMMM.htm which don't exist. (The directories fichier/YYYY/MM exist and contain files like 07-105956.ods but no tempMMMM.htm files) [for Bidouille to resolve]
  • Entering the adm links to the wrong path relative style instead of styles. A symlink from adm/styles to styles will fix this. [Done]
  • There were a couple of usernames as forum icons which I removed (an artifact of Ff field defaults.) [Done]
  • One of the bg.gifs was missing. [Done]

I will look at the error logs again in a week to see what more needs to be done.

In terms of the access logs, I was using the lines

<IfModule logio_module>
  # You need to enable mod_logio.c to use %I and %O
  LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
</IfModule>

to define a combinedio alias, but I had forgot to enable mod_logio and thus the alias wasn't defined, so for the last week, we've had "combinedio" logged for every access. I've fixed this. I will look at this again in a few days time.

RSS Feed

A number of the power users on the fr forum used the previous RSS feed. This functionality has been temporarily dropped and needs reinstituted. The current implementation adopts the Simple RSS mod for phpBB3. However, this mod has significant limitations that I can't ignore. Most importantly it bypasses the phpBB access control system, allowing unregistered users to browse closed forums.

I need to discuss this further with the fr proponents of this mod.

Site Home Page (Closed)

I hadn't migrated this one so for a day, the home page as It works!. Oops. A few people raised this so I hacked quick-and-dirty home page based on the main OOo homepage and style. Since I didn't change the CSS, we've got a couple of wobbly columns. See user.services.OpenOffice.org for this example.

Drew is going to reimplement a proper home page.

I also had to add redirectors from /en to /en/forum, etc. Again symlink to a shared redirector the /var/www/phpBB-common folder.

View New Post (Count)

This count is no longer being updated during the session.

Warning user flag on viewtopics and viewforum (Closed)

This was previously implemented on the fr forum. This functionality was been temporarily dropped and has now been reinstituted.

Personal tools