You are here: Foswiki>Tasks Web>Item11326 (05 Jul 2015, GeorgeClark)Edit Attach

Item11326: SubscribePlugin is extremely slow with large complex WebNotify topics.

pencil
Priority: Urgent
Current State: Closed
Released In: 2.0.0
Target Release: major
Applies To: Extension
Component: MailerContrib
Branches: trunk
Reported By: GeorgeClark
Waiting For:
Last Change By: GeorgeClark
Topic view for logged-in users on Tasks web is extremely slow.

  • View any task or non-search topic as WikiGuest ~ 1 second to return the main document
  • Log in - same view takes 13-18 seconds to retrieve the base document.

Other factors:

cover=print Fast
skin=print Fast
skin=plain Fast
skin=default SLOW
raw=on or raw=debug SLOW
raw=text or raw=all Fast

  • Running ./view -user GeorgeClark -topic Tasks.Item11051 is Fast
  • Checked with the release01x00 vhost - seemed faster, but still very slow
  • Cloned Tasks web to another server. Seems fast in all conditions.
  • Logged in as admin superuser. Still slow.

-- GeorgeClark - 04 Dec 2011

The problem is the SubscribePlugin. It is adding 10-20 seconds to the rendering time. Disabled for now.

-- GeorgeClark - 04 Dec 2011

I feel as if this is a recent thing. And trunk is much faster. I've reproduced locally.

-- PaulHarvey - 04 Dec 2011

It happens on trunk code, something to do with Main web, I think.

-- PaulHarvey - 04 Dec 2011

I've tried versions of MailerContrib back to distro:9c88747740c3, but it's all still terribly slow..

-- PaulHarvey - 04 Dec 2011

Support web recommends that users should subscribe to their own question topics, but we've got SubscribePlugin turned off, so they can't.

Added WebSiteTasks to reflect this

I changed target to patch, given that we need to fix this for foswiki.org

-- PaulHarvey - 03 Mar 2012

The workaround for this is to set DISABLEDPLUGINS = SubscribePlugin in Tasks.WebPreferences (and in any other webs with exceedingly large WebNotify topics.

I added a config setting - {SubscribePlugin}{AvailableWebs}, and planned to handle the issue that way, but just disabling the plugin is just as effective, provided that it's not overridden in a topic. It might be best to add this feature anyway, as it gives a bit more control to the administrator.

I'm opening a separate task to add that feature, and will re-title this to deal with the performance issue directly.

Setting this task to be a release blocker for 1.2. The performance issues are too extreme to enable this as a default plugin.

-- GeorgeClark - 03 Mar 2012

the problem is due to the fact that the code that adds entries to a subscriber set attempts to optimise the set each time an entry is added. This is an O(N^2) process, but guarantees a fast lookup when searching for matches in the set. So there's a comromise here; by skipping the optimisation on the read we accept there may be some suboptimal entries. Optimising when a new webnotify is written means we get the best of both worlds.

Not quite finished here yet; the load is still too slow, need to take a closer look at that.

-- CrawfordCurrie - 22 Jan 2013

OK, I'm done. It should be useable on f.o again now. No changes to the SubscribePlugin, all changes were in MailerContrib.

-- CrawfordCurrie - 23 Jan 2013

 

  • nytprof.tar.gz: NYTProf output using trunk code and foswiki.org's data
Topic attachments
I Attachment Action Size Date Who Comment
nytprof.tar.gzgz nytprof.tar.gz manage 7 MB 04 Dec 2011 - 06:28 PaulHarvey NYTProf output using trunk code and foswiki.org's data
Topic revision: r14 - 05 Jul 2015, GeorgeClark - This page was cached on 01 Jun 2016 - 07:20.

The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License