Item1382: Small speed optimisation of search code

Priority: Enhancement
Current State: Closed
Released In: 1.1.0
Target Release: minor
Applies To: Engine
Reported By: SvenDowideit
Waiting For:
Last Change By: KennethLavrsen
Meta::putKeyed uses a surprising amount of cpu (using NYTProf) looping through 'all' the items in a 'type'

(I was profiling AdminToolsCategory)

by replacing this loop with a set of hash indexes, I shaved off ~4seconds from the 11-12second run time - NYTProf slows thigs down smile

I'm going to commit it to trunk today, and see about release branch tomorrow.

-- SvenDowideit - 27 Mar 2009

I have a suspicion that its due to the DocumentGraphics topic, which someone has added the attachment META to for all ~260 images

and so there's another issue - as it appears that topic might be loaded and parsed more than once., which i'll poke tomorrow.

-- SvenDowideit - 27 Mar 2009

Interesting observation

We have to be careful not to break something in release branch but the observation that DocumentGraphics attachment meta can slow down things is very interesting and worth investigating. I will try and benchmark the difference with and without. There is no functional need for the icons to be formally attached.

And people can still see the images in the topic as long as they are in the pub dir

-- KennethLavrsen - 27 Mar 2009

well. I backed out the INDEX hash, as it causes unit test failures, and I get the feeling that the {_loadedRev} change has more real effects.

Looking around, there seem to be alot of ways to get Foswiki to re-re-reload files - using YuiMenuContrib (which has one tmpl file containing one TMPL:DEF seems to cause Foswiki to load 165 tmpl files (lots of reloads) vs 36! in just pattern.

so there's quite alot of code analysis that should be done - in many cases a print STDERR wherever there's an open(), and in others NTYProf.

I'm going to leave it for a little, as i really need to be elsewhere smile

-- SvenDowideit - 28 Mar 2009

Interesting enough that I picked this up and did it. All tests pass. Sven, take a look at what I did; I didn't use your code, exactly, but I think I followed your intent accurately. If I did then set this "Waiting for Release".

-- CrawfordCurrie - 23 May 2009

i recon it looks good smile

-- SvenDowideit - 24 May 2009

ItemTemplate edit

Summary Small speed optimisation of search code
ReportedBy SvenDowideit
Codebase trunk
SVN Range Foswiki-1.0.0, Thu, 08 Jan 2009, build 1878
AppliesTo Engine
Priority Enhancement
CurrentState Closed
Checkins distro:57e7d85a8c36 distro:9f1e5b1c6daf distro:3cc0692b6de8 distro:8bda099a0255
TargetRelease minor
ReleasedIn 1.1.0
Topic revision: r13 - 04 Oct 2010, KennethLavrsen
The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License    Legal Imprint    Privacy Policy