Item11752: DBCache fails to build cache silently

pencil
Priority: Normal
Current State: Closed
Released In: n/a
Target Release: n/a
Applies To: Extension
Component: DBCacheContrib, DBCachePlugin
Branches: trunk
Reported By: LynnwoodBrown
Waiting For:
Last Change By: MichaelDaum
I was working with a rather large web (~3600 topics) and could not get DBCacheContrib to complete rebuilding of the cache. I would try deleting the cache and then rebuilding it using the rest call. During this process, some error messages would show up in /woring/logs/error.log but there was nothing that I found helpful in identifying which topic it failed on. Subsequent called to the rest call simple had no affect. The cache did not change in size and no further error messages were recorded.

Talking with MichaelDaum on irc, he pointed me to look for form field field references that had mal-formed topic references that were left over from moving some topics between webs. This took a very long time but I knew I was making progress because after searching down and fixing some such topics and rebuilding the cache again, the cache would get bigger. Finally, it appeared that the cache was completely rebuilt and all topics were showing up in DBQUERYs.

It would be great if DBCacheContrib was more fault tolerant in dealing with this kind of issue (people do move topics between webs) but at the least, it would be much improved if error messages generated 1) when it can not complete building the cache initially, and 2) if a partial cache is not being further added to. At the least, it would be useful to have it list what topic it fails on. Otherwise, it is extremely difficult to find where a fix is needed.

-- LynnwoodBrown - 12 Apr 2012

Hi Lynnwood. It doesn't seem to be the amount of topics being cached rather than one specific malformed one. Are you able to handcraft a txt file that brings down the indexer so that I can reproduce the problem?

-- MichaelDaum - 13 Apr 2012

Thanks for the suggestion. I'll give it a try. Right now, although the cache appears to be complete, it is not winning awards for speed. A page that searches questions (the bulk of the topics) takes over 2 minutes to load and ofter times out. frown, sad smile No errors in apache of foswiki logs. BTW, the server is dual quad core Xeon CPUs and a Gigabit network card with 6G of ram (which according to top are idling most of the time).

So i don't know if the issue mal-formed topic creating trouble for the cache of something else. I'll try to create a bad topic and see if i can make it fail to rebuild cache. I spent a lot of time trying to search down and remove any mis-placed cross-web references in field names or values.

-- LynnwoodBrown - 13 Apr 2012

The error is in DBCacheContrib::Map::set() where it blindly tries to build new maps when the formfield name has got dots in it. Normally set() isn't called with parameters containing a dot. Most probably this isn't covered by unit tests either. Here's a patch which cured the problem for me while not removing this hidden feature all together:

--- lib/Foswiki/Contrib/DBCacheContrib/Map.pm   (revision 14949)
+++ lib/Foswiki/Contrib/DBCacheContrib/Map.pm   (working copy)
@@ -226,10 +226,13 @@
     if ( $attr =~ m/^(\w+)\.(.*)$/o ) {
         $attr = $1;
         my $field = $2;
-        if ( !defined( $this->FETCH($attr) ) ) {
-            $this->STORE( $attr, $this->{archivist}->newMap() );
+        my $map   = $this->FETCH($attr);
+        if ( defined($map) ) {
+            $map->set( $field, $val );
         }
-        $this->FETCH($attr)->set( $field, $val );
+        else {
+            print STDERR "ERROR: no map found to store $attr.$field\n";
+        }
     }
     else {
         $this->STORE( $attr, $val );

-- MichaelDaum - 05 Jun 2012

At long last, I have had to the opportunity to test the patch which Michael outlines above and can confirm that it did work in my case. I'd recommend committing this.

-- LynnwoodBrown - 13 Nov 2012
 
Topic revision: r7 - 19 Nov 2012, MichaelDaum
The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License    Legal Imprint    Privacy Policy