Item9375: DBCacheContrib poor performace

Hi everyone,

I having really performances issues with my foswiki instanace. My foswiki is running under:

• OS: Virtual Windows Server 2008
  • Host: Windows Server 2008
  • Virtual machine engine: hyper-V
• RAM: 2GB
• CPUs: Intel XEON Quad core 2.0 Ghz

• Apache2(xampp) - with mod_perl
• perl5.10(xampp)

I have the following plugins installed:

• TWikiCompatibilityPlugin (Foswiki-1.0, $Rev: 1776 (2009-01-06) $): add TWiki personality to Foswiki
• SpreadSheetPlugin (10 Nov 2009, $Rev: 5484 (2009-11-10) $): Add spreadsheet calculations like "$SUM($ABOVE())" to Foswiki tables and other topic text
• CommentPlugin (12 Sep 2009, $Rev: 4843 (2009-09-11) $): Quickly post comments to a page without an edit/preview/save cycle
• DBCachePlugin (3.40, $Rev: 6290 (2010-02-12) $): Lightweighted frontend to the DBCacheContrib
• EditChapterPlugin (2.12, $Rev: 6337 (2010-02-15) $): An easy sectional edit facility
• EditTablePlugin (4.33, $Rev: 4950 (2009-09-16) $): Edit tables using edit fields, date pickers and drop down boxes
• FormQueryPlugin (17 Nov 2009, $Rev: 5542 (2009-11-17) $): Provides query capabilities across a database defined using forms and embedded tables in Foswiki topics.
• InterwikiPlugin (20 Sep 2009, $Rev: 5022 (2009-09-20) $): Link ExternalSite?:Page text to external sites based on aliases defined in a rules topic
• JQueryPlugin (3.11, $Rev: 20090710 (2009-07-10) $): jQuery JavaScript library for Foswiki
• PreferencesPlugin (20 Sep 2009, $Rev: 5037 (2009-09-20) $): Allows editing of preferences using fields predefined in a form
• RevCommentPlugin (22 Jan 2009, $Rev: 2125 (22 Jan 2009) $): Allows a short summary of changes to be entered for a new revision.
• SlideShowPlugin (02 Aug 2008, $Rev: 2742 (2009-02-26) $): Create web based presentations based on topics with headings.
• SmiliesPlugin (20 Sep 2009, $Rev: 5046 (2009-09-20) $): Render smilies like smile as icons
• TablePlugin (1.043, $Rev: 5078 (2009-09-21) $): Control attributes of tables and sorting of table columns
• TinyMCEPlugin (26 Fennt t b 2010, $Rev: 6527 (2010-02-26) $): Integration of the Tiny MCE WYSIWYG Editor
• TwistyPlugin (1.5.3, $Rev: 4751 (2009-09-02) $): Twisty section Javascript library to open/close content dynamically
• VarCachePlugin (29 Jan 2009, $Rev: 2215 (29 Jan 2009) $): Cache Foswiki macros in selected topics for faster page rendering
• WorkflowPlugin (03 Feb 2010, $Rev: 6211 (2010-02-03) $): Supports work flows associated with topics
• WysiwygPlugin (18 Sep 2009, $Rev: 5356 (2009-10-22) $): Translator framework for Wysiwyg editors
• ZonePlugin (1.2, $Rev: 6437 (2010-02-19) $): Gather content of a page in named zones while rendering it

And, ModPerlContrib, DBCacheContrib.

the foswiki is configured to used ModPerlContrib, and all search topic are using DBQUERY.

My data directory has 13382 files(topics),where about 4000 are topics with forms all of them have to be reach by DBQUERY search.

I only have 300 users, and the application is unusable with 40 useres(takes around 2 min to login and view a text only topic) so i having really performance issues.(I have a performance test but is in spanish, the test show that only the CPU is 100%, I/O and RAM are OK).

So, how do i have to configure my Foswiki to be able to support my 300 users?

Thnaks,

Pleace Help!!

-- JuanPabloGaviria - 22 Jul 2010

When I tested foswiki on windows, I noticed the decrease in performance, but far from making it unusable...

• Have you made some kind of benchmark of disk access where foswiki files are? * Have you made some kind of file system tuning? (see this and this)
• Is your $Foswiki::cfg{Sessions}{ExpireAfter} setting adjusted to a negative value? (and is tick_foswiki.pl scheduled to run?)
• How many session files do you have? (working/tmp folder)
• How is your $Foswiki::cfg{RCS}{SearchAlgorithm} adjusted?
  • What is the impact of using PurePerl, Forking and NativeSearchContrib? ( ModPerlEngineContrib doesn't work with the last one )
• What is the impact of turning ModPerlEngineContrib on and off?

I'd appreciate to see your performance test, even if it's in spanish

-- GilmarSantosJr - 23 Jul 2010

Background: Support.Question544

-- PaulHarvey - 23 Jul 2010

I don't really understand why this is "Urgent" against the core - it appears to relate to DBCacheContrib, from the support question. However, it's waiting for feedback from the reporter, so marking as such. Without meaningful benchmarks, there's nothing we can do.

-- CrawfordCurrie - 23 Jul 2010

Changed the title to make it DBCacheContrib specific. Crawford, any additional thoughts on the findings at Support.Question544 ?

-- MichaelDaum - 23 Jul 2010

No. I'm afraid I have too many demands on my time at the moment to give it the attention it deserves.

-- CrawfordCurrie - 23 Jul 2010

Hi,

Thanks for all the answers. The Support.Question544 is related to a local test(my localhost) and the target was to test te performace for a foswiki with thousauns(350000) only with 1 user and running in Ubuntu-i386-9.10.

This time i'm runnig foswiki on Win-2008. I used the foswiki windows installer at the first time, so the foswiki was ruuning with: * Apache2 * perl(strawberry) * the same hardware: * OS: Virtual Windows Server 2008 * Host: Windows Server 2008 * Virtual machine engine: hyper-V * RAM: 2GB * CPUs: Intel XEON Quad core 2.0 Ghz

then, i developed 3 foswiki-applications: 1. For bussiness process documentation(utilitu to create sub-webs with specifyc web-templete like bussines-process-template, bussines-subprocess-template, etc) 1. Corporate events application(simple dataforms with simple events search using %SEARCH%) 1. Corporate Address book(simple dataforms with simple people, clientes, parnets, etc search using %SEARCH%)

Then, we copied all excel-word docs to this 3 foswiki-applications(around 13000 topics), then we make the performance test, and the result was:

• with 10 concurrent users:
  • Too many httpd process using 100% CPU
  • the response time to view a topic was about 1min.
  • the topic-search result takes about 2min

Unfortunatelly we don't have that data, so we deside to make all the tunning apache with mod_perl(we could't get fastcgi on win), foswiki whit ModPerlEngineContrib, DBCacheContrib, DBCachePlugin, change all the %SEARCH% for %DBQUERY%, INCLUDE for DBCALL. And run test algo to the follow thinks:

• With 50 concurente users:
  • login and view the Main(23 sec)
  • Then go to specific topic, text only(23 sec)
  • Then go to specific topic, with %SEARCH% because it has to search a pattern text in a specific topic and DBCachePlugin doesn't support that kind of search(45 sec)
  • Then go to specific topic, text only(20 sec)
  • The logout(20 sec)

The only thing i haven't done is the thing about sessions, an i'm about to do it.

So this the story of the problema, and i don't thing that it related with the DBCachePlugin, or DBCacheContrib. And i don't know what the actual problema is? so sorry if i create the item as Urgent and wothout exact "applies to".

I'm going to try the sessions thing and then post the results and the performance test, for now if someone has a clue about this pleace help.

Thanks,

NOTE: we are using viewfile to ensure the acces control the attachmente, excluding System and Main webs

-- JuanPabloGaviria - 23 Jul 2010

Are these results os using the Storable backend of DBCacheContrib?

Unfortunately, FastCGI is very difficult to setup on windows. I achieved it once, but I didn't document it and never got it again

-- GilmarSantosJr - 23 Jul 2010

Hi Gilmar,

Yes, i'm using Storable as backend, in the Support.Question544 i try BerkleyDB and everything was wrong.

-- JuanPabloGaviria - 25 Jul 2010

Improved default settings according to the documented practices in above support question.

Crawford, may I upload a new release (-> coord with author)?

-- MichaelDaum - 28 Mar 2011

Updated settings for latest release.

-- MichaelDaum - 06 Apr 2011