Revision as of 09:51, 19 May 2009

Follow-up Issues and Actions

I have included the list of issue in alphabetic order below.

Attachment Quotas (Closed)

I set the attachment quotas on the EN and FR boards as 300Mb and 500Mb respectively, with the other boards are set at 100Mb for now. Unfortunately I used the values 300Mb where this should have been a number which resulted in attachments being disabled. The first time I fixed it, I had finger trouble on the update (accidentally typing a ";" before the where clause in a update has a disastrous result :lol:).

"Oups, seems there is a problem in the counts bottom of page: all are set to 314,872,800 topics/posts/users!"

Another hour extracting and restoring phpbb_XX_config tables from the last backup and rolling forward the changes that I made.

Automated Mailbox Robot

I now have this working for each forum, subject to limitation that the Admin Mailbox Configuration post must be correctly configured. There seems to be something wrong with the forum permissions propagations. I have used the workaround of using my administrator powers to adopt a moderators permissions so I have now (as at UTC 02:08, 18 May 2009) set up all of the forum mailboxes and tested each by sending a test message to each. The NL admins mostly have some more playing/tweaking of the configs but the robot is broadly working.

Automated Unactivated Users Prune

Banned user flag on viewtopics and viewforum

This was previously implemented on the fr forum. This functionality has been temporarily dropped and needs reinstituted.

Caching Setup and Policy

I have enabled the APC (Another Php Cache) cache for the site.

apc.enabled=1
apc.shm_segments=1
apc.cache_by_default=1
apc.optimization=0
apc.shm_size=64
apc.ttl=240
apc.user_ttl=240
apc.gc_ttl=120
apc.stat=1
apc.num_files_hint=1024
apc.user_entries_hint=200
apc.mmap_file_mask=/dev/zero
apc.enable_cli=1

APC seems to work fine apart from the setting apc.stat=1 doesn't seem to work in CSK 1.3.1. It worked fine in 1.3. This was a case of two wrongs making a right. If you edit a file and save it back with vi, this updates the existing file leaving its ctime unchanged. This setting does what it says. To be sensitive to edits you need to set apc.stat_ctime=1. I'll try this out today.

You can see how effective this is by executing a temporary APC query usOOO &mash; APC Cache Summary. I just ran this report and on a 47hr window since last refresh we are running at around 170K cache reference/hr and of these we have a 99.4% cache hit rate. The figure is even bigger when weighted by code size.

Cache Follow-up

Any time phpBB executes a SQL query, style template, or dataset retrieval, it creates an md5 of the sql query / tempate and stores this under the name sql_<md5Value> in the cache (or euqivalent for the other types). Before executing the query, it does an existence check for the file andexecutes an @include it to evaluate it. PHP loades the cached bytecode version instead. Here is a typical example

      <?php
       /* SELECT COUNT(DISTINCT s.session_ip) as num_guests FROM phpbb_en_sessions s \
          WHEREs.session_user_id = 1 AND s.session_time >= 1242322740 AND s.session_forum_id = 61   */
       $expired = (time() > 1242323134) ? true : false;
       if ($expired) { return; }
       $this->sql_rowset[$query_id] = unserialize('a:1:{i:0;a:1:{s:10:"num_guests";s:1:"0";}}');
       ?>

This code sets the boolean $expired and if false then also returns the results set. If true or the file doesn't exist then the execution does the SQL query then creates this file (but only for what the application deems as reusable queries). With this phpBB cache the results are latched for a specific time (in this example the expiry time was some 6 mins after the start of the session) even if the underlying DB data has changed. Thanks to the APC, the code for this is also cached to executing this is quick. This approach is different to the MySQL cache, where the execution plan for each SQL queries is cached in an LRU buffer. This saves parse costs, but the query is still re-evaluated each call.

Forum Permissions and Access

I am an admin on all forums and should therefore have edit access to all forums. If I go into the ACP->Manage Groups->Group Forum Permissions and check, the Admin group has full permission on all forums. However in some forums, e.g. es and fr, I don't have edit edit access to topics but I do on some such as en. I haven't bottomed out what the difference is yet. The relevant code fragment from viewtopic.php (prettied for readability) is:

  'U_EDIT'       => (!$user->data['is_registered']) ?  : 
                       ( ( ( $user->data['user_id'] == $poster_id && 
                             $auth->acl_get('f_edit', $forum_id) && 
                             ( $row['post_time'] > time() - ( $config['edit_time'] * 60 ) || 
                               !$config['edit_time']
                             )
                           ) || $auth->acl_get('m_edit', $forum_id)
                         ) ? append_sid("{$phpbb_root_path}posting.$phpEx", 
                                        "mode=edit&f=$forum_id&p={$row['post_id']}") :
                             ),

that is for other than the O/P, acl_get must return the m_edit privilege for that forum.

Until I can work this one out the work around is to pick a moderator and test his permissions.

Language Localisation

The out-of-the-box language packs contain some mistranslations, A number of the NL Administrators have posted changes asking me to correct various miswordings in their translations. It's a little error-prone for me to do as I usually don't understand the nuances of the text and its context. There is a feature (now working) whereby the administrators can now do these changes themselves through the ACP if that have founding administrator rights in you forum, but as most of you don't have this on your own database:

• You log onto /forum/test database as TestFounder (pwd openofficetest) and then through the ACP through the link at the foot of the page which is visible when you are logged in as an administrator.
• You navigate to ACP->System->Language Packs. This lists the Language packs installed in your NL Forum. Each of the named packs is a URI; click on the pack you want to edit.
• Under the Language entries there is a drop down allowing you to pick the file that you want to review / change. There are Language Files which relate 1-1 with the page URIs (e.g. posting.php is all of the NL strings specific to the posting form) and a common one for NL strings that are used on multiple forums. There are also templates for emails and registration. Make the changes that you need. One great advantage of doing it this way is that the encoding is UTF.
• Unfortunately, you need to be a Founding Administrator to have these right, and the NL admins do'nt have this, so I have added an account to the /test/forum instance (name and password provided on our private forum). Go into this test forum to do this work. Since all the NL instances use the same code and language packs, you can do this work here.
• When you are done click the "submit and download" button. This saves a copy on the server and downloads a copy to your PC which you can then review and edit (in a UTF8 enabled editor) at your leisure or pass to colleagues for them to update. For all but the simplest edits it is safe to do this download, because if you leave and back to the form you will start again with the installed version. (In fact you may just prefer to use an editor rather than the form interface in which case you select the file and click submit and download straight-away.)
• When you are happy with the changes then email the files to me (TerryE at the mail domain OpenOffice.org) and I will install them for you pretty much as soon as I pick up the email.

As an aside, I had real troubles getting this to work  :crazy: It was a bastard to debug. (See phpBB — (ACP) Unable to submit large forms).

Given that I am the resource bottleneck at the moment, can I ask all NL administrators which outstanding translation change requests to use this mechanism.

Language Pack Inconsistencies

Even though the Langauge packs are shared, the corresponding language tables in the forums were brought across from the old databases and are therefore inconsistent. This doesn't impact phpBB at an applications level, but it would make sense to standardise these as a one off. (The only difference should be whether the individual packs are enabled for a given forum.)

Mail subscriptions aren't working (Closed)

You are supposed to be able to subscribe to a forum or post and this will trigger automatic mail notifications. This wasn't working. I set up a test harness using the test NL forum on the server to try this out. After a lot of tedious debugging, I discovered two issue, both of which I really regard as bugs in phpBB:

• I had set up the forums so that the ACP->Email Settings->E-mail package size to 50. OK, the comment says "This is the number of maximum e-mails sent out in one package. This setting is applied to the internal message queue; set this value to 0 if you have problems with non-delivered notification e-mails." (My italics) However, when I drill through all the code the queuing and dequeuing functionality is implemented, but the dequeue is triggered by a cron.php hook which was never called.
• The reason for this is that the common.php includes the following conditional which causes the cron.php task to fail in our shared code implementation, since this causes it to attempt to resolve the cache directory in /var/www/phpBB-common/cache which doesn't exist. What the logic for doing this, heaven knows, but my work-around is to reset $phpbb_root_path back after including this in the cron.php routine.

if (defined('IN_CRON'))
{
    $phpbb_root_path = dirname(__FILE__) . DIRECTORY_SEPARATOR;
}

The way the cron hook is executed is that the footer template includes the hook:

 <!-- IF not S_IS_BOT -->{RUN_CRON_TASK}<!-- ENDIF -->

The page_footer setup function has an optional parameter $run_cron which defaults to true, and in fact is always called page_footer(), so this cron path is executed on every page generation. The code is (I've compressed its layout to save space):

 $cron_type = ;
 if ( time() - $config['queue_interval'] > $config['last_queue_run'] && 
      !defined('IN_ADMIN') && 
      file_exists($phpbb_root_path . 'cache/queue.' . $phpEx))             { // Process email queue
     $cron_type = 'queue';
 } else if (method_exists($cache, 'tidy') && 
            time() - $config['cache_gc'] > $config['cache_last_gc'])       { // Tidy the cache
     $cron_type = 'tidy_cache';
 } else if (time() - $config['warnings_gc'] > $config['warnings_last_gc']) {
     $cron_type = 'tidy_warnings';
 } else if (time() - $config['database_gc'] > $config['database_last_gc']) { // Tidy the database
     $cron_type = 'tidy_database';
 } else if (time() - $config['search_gc'] > $config['search_last_gc'])     { // Tidy the search
     $cron_type = 'tidy_search';
 } else if (time() - $config['session_gc'] > $config['session_last_gc'])   {
     $cron_type = 'tidy_sessions';
 }
 if ($cron_type) {
     $template->assign_var('RUN_CRON_TASK',
                           '<img src="' . append_sid($phpbb_root_path . 'cron.' . 
                                                     $phpEx,'cron_type=' . $cron_type) . 
                           '" width="1" height="1" alt="cron" />');
 }

What this ladder does is that when one of these thresholds has passed the page includes a image link to a dummy image which is one blank pixel, but in reality this triggers the cleanup function. This trigger occurs at page load but most browsers implement a lazy image load so this doesn't impair the user experience.

I have put in this fix, but also set the email_package_size parameter to zero for now. I will review this for tuning and optimisation later, but for now this works.

Review of Apache Error Logs

I went through the Apache error logs to see what avoidable errors could be addressed:

• The robots.txt file needed updating [Done]
• The favicon.ico needs an explicit meta tag in the standard header [TBD]
• The fr rss.php is generating a lot of misses [TBD]
• The fr ci-joint database is generating a lot of hits to files of the format fichier/YYYY/MM/0000NNNNN_KKKKK.tempMMMM.htm which don't exist. (The directories fichier/YYYY/MM exist and contain files like 07-105956.ods but no tempMMMM.htm files) [for Bidouille to resolve]
• Entering the adm links to the wrong path relative style instead of styles. A symlink from adm/styles to styles will fix this. [Done]
• There were a couple of usernames as forum icons which I removed (an artifact of Ff field defaults.) [Done]
• One of the bg.gifs was missing. [Done]

RSS Feed

A number of the power users on the fr forum used the previous RSS feed. This functionality has been temporarily dropped and needs reinstituted.

Site Home Page (Closed)

I hadn't migrated this one so for a day, the home page as It works!. Oops. A few people raised this so I hacked quick-and-dirty home page based on the main OOo homepage and style. Since I didn't change the CSS, we've got a couple of wobbly columns. See user.services.OpenOffice.org for this example.

Drew is going to reimplement a proper home page.

I also had to add redirectors from /en to /en/forum, etc. Again symlink to a shared redirector the /var/www/phpBB-common folder.

View New Post (Count)

This count is no longer being updated during the session.