You are here: Foswiki>Tasks Web>Item14329 (31 Jan 2018, GeorgeClark)Edit Attach

Item14329: Memory leaks in Foswiki

pencil
Priority: Normal
Current State: Waiting for Release
Released In: 2.2.0
Target Release: minor
Applies To: Engine
Component: Performance
Branches: master Item14288 Item14380 Item14537
Reported By: GeorgeClark
Waiting For:
Last Change By: GeorgeClark
We are seeing memory leaks on Foswiki.org, evident with long running fcgi handlers. We've not been restarting them automatically.

Local testing, on a lightly used system I watched fcgi memory growth following two transactions:
  • ..../System/WebHome?refresh=cache;cover=print
  • ..../System/WebChanges?refresh=cache;cover=print

The page without the search seeing very minor memory growth. However the SEARCH containing was growing by 1000K or more for 10 requests.

Running MemoryCycleTests, with Foswiki::finish() commented out, shows 102 circular references. The only baseline is in the code, "(56 are found as of Jun2006)"

Following patch might clean up a few of them. It reduces it to 99. The bulk of the references are due to many objects locally storing a {session} reference.

Given the growth on SEARCH pages, the MetaCache might be a cause of the leak. The fix adds a call to finish() the TopicObject during cleanup. rather than just undefining the containing stucture.

diff --git a/core/lib/Foswiki.pm b/core/lib/Foswiki.pm
index 90e23da..776864f 100644
--- a/core/lib/Foswiki.pm
+++ b/core/lib/Foswiki.pm
@@ -2579,6 +2579,8 @@ sub finish {
     undef $this->{requestedWebName};    # Web name before renaming
     undef $this->{scriptUrlPath};
     undef $this->{user};
+    $this->{users}->finish() if $this->{users};
+    undef $this->{users};
     undef $this->{_INCLUDES};
     undef $this->{response};
     undef $this->{evaluating_if};
@@ -2586,6 +2588,10 @@ sub finish {
     undef $this->{sandbox};
     undef $this->{evaluatingEval};
     undef $this->{_ffCache};
+    $this->{zones}->finish() if $this->{zones};
+    undef $this->{zones};
+    $this->{renderer}->finish() if $this->{renderer};
+    undef $this->{renderer};
 
     undef $this->{DebugVerificationCode};    # from Foswiki::UI::Register
     if (SINGLE_SINGLETONS_TRACE) {
diff --git a/core/lib/Foswiki/Meta.pm b/core/lib/Foswiki/Meta.pm
index 1d20dba..483df1c 100644
--- a/core/lib/Foswiki/Meta.pm
+++ b/core/lib/Foswiki/Meta.pm
@@ -534,6 +534,13 @@ sub finish {
     undef $this->{_web};
     undef $this->{_topic};
     undef $this->{_session};
+    undef $this->{_loadedRev};
+    undef $this->{_latestIsLoaded};
+    undef $this->{_text};
+    undef $this->{_preferences};
+    undef $this->{_indices};
+    undef $this->{FILEATTACHMENT};
+
     if (DEBUG) {
 
     #someone keeps adding random references to Meta so to shake them out..
diff --git a/core/lib/Foswiki/MetaCache.pm b/core/lib/Foswiki/MetaCache.pm
index 68e347d..42cd46e 100644
--- a/core/lib/Foswiki/MetaCache.pm
+++ b/core/lib/Foswiki/MetaCache.pm
@@ -76,12 +76,12 @@ Break circular references.
 
 sub finish {
     my $this = shift;
-    undef $this->{session};
 
     #must clear cache every request until the cache is hooked up to Store's save
     foreach my $cuid ( keys( %{ $this->{cache} } ) ) {
         foreach my $web ( keys( %{ $this->{cache}->{$cuid} } ) ) {
             foreach my $topic ( keys( %{ $this->{cache}->{$cuid}->{$web} } ) ) {
+                $this->{cache}->{$cuid}{$web}{$topic}{tom}->finish();
                 undef $this->{cache}->{$cuid}{$web}{$topic};
                 $this->{undef_count}++;
             }
@@ -89,7 +89,10 @@ sub finish {
         }
         undef $this->{cache}->{$cuid};
     }
+
+    undef $this->{session};
     undef $this->{cache};
+    undef $this->{meta_cache_session_user};
 
     if (TRACE) {
         print STDERR
diff --git a/core/lib/Foswiki/Render.pm b/core/lib/Foswiki/Render.pm
index 104ee57..21d2dd1 100644
--- a/core/lib/Foswiki/Render.pm
+++ b/core/lib/Foswiki/Render.pm
@@ -105,6 +105,7 @@ Break circular references.
 # documentation" of the live fields in the object.
 sub finish {
     my $this = shift;
+    undef $this->{_anchorNames};
     undef $this->{NEWLINKFORMAT};
     undef $this->{LINKTOOLTIPINFO};
     undef $this->{LIST};
diff --git a/core/lib/Foswiki/Render/Zones.pm b/core/lib/Foswiki/Render/Zones.pm
index 4f27419..016cfbd 100644
--- a/core/lib/Foswiki/Render/Zones.pm
+++ b/core/lib/Foswiki/Render/Zones.pm
@@ -174,7 +174,9 @@ sub _renderZoneById {
     my $zone        = $params->{_DEFAULT} || $params->{zone};
 
 #return "\n<!--ZONE $id-->\n" . _renderZone( $this, $zone, $params, $topicObject ) . "\n<!--ENDZONE $id-->\n" ;
-    return _renderZone( $this, $zone, $params, $topicObject );
+    my $result = _renderZone( $this, $zone, $params, $topicObject );
+    $this->{_renderZonePlaceholder}{$id} = undef;
+    return $result;
 }
 
 # This private function is used in ZoneTests
diff --git a/core/tools/MemoryCycleTests.pl b/core/tools/MemoryCycleTests.pl
index b10ba67..e4d189d 100755
--- a/core/tools/MemoryCycleTests.pl
+++ b/core/tools/MemoryCycleTests.pl
@@ -45,12 +45,13 @@ use Foswiki::UI::View;
 #NOTE that Foswiki::finish() is hiding many circular references by foricbly clearing
 #them with the %$this = (); its worth uncommenting this line once in a while to
 #see if its gettign worse (56 are found as of Jun2006)
-#*Foswiki::finish = sub {};
+*Foswiki::finish = sub {};
 
     $Foswiki::Plugins::SESSION = $session;
     monitor( 'Foswiki' => \$Foswiki::Plugins::SESSION );
 
-    Foswiki::UI::run( \&Foswiki::UI::View::view );
+    #Foswiki::UI::run( \&Foswiki::UI::View::view );
+    $Foswiki::engine->run();
 
     print_circular_ref( \$Foswiki::Plugins::SESSION );
 }

-- GeorgeClark - 26 Feb 2017

 
Topic revision: r7 - 31 Jan 2018, GeorgeClark
The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License    Legal Imprint    Privacy Policy