You are here: Foswiki>Tasks Web>Item13820 (21 Jan 2016, MichaelDaum)Edit Attach

Item13820: The putBackBlocks and _putBackProtected routines are extremely inefficient.

Priority: Enhancement
Current State: Closed
Released In: 2.0.3
Target Release: patch
Applies To: Engine
Component: FoswikiRender, Performance
Branches: master
Reported By: GeorgeClark
Waiting For:
Last Change By: MichaelDaum
The routines that restore removed blocks of text during the rendering process are extremely inefficient.

  • It repeatedly re-scans the entire topic text for markers using a regex. 1000's of markers in some cases.
  • For each hit, it expands the string in-place requiring a string copy operation.
  • It processes the markers in completely random order due to perl hash reordering.

Tables, especially with EditRowPlugin requires 1000's of markers. The time in Render::_putBackProtected() was approx. 4.5 seconds. Changing the routine to re-build the page using concatenation brought the time down to 63ms. The following changes were made:

  • Drive the replacement process by a scan of the topic text for markers, instead of a search of the hash keys.
  • Scan using index() to locate each marker sequentially in a single pass of the data.
  • Build a new page by concatenating the original text segments with the marker replacements.

-- GeorgeClark - 15 Oct 2015

Results rendering page with 400 table rows, and EditRowPlugin active:
Before: # spent 4.41s (10.1ms+4.40) within Foswiki::Render::_putBackProtected which was called 45 times, avg 97.9ms/call: 

After: # spent 62.9ms within Foswiki::Render::_putBackProtected which was called 45 times, avg 1.40ms/call:

-- GeorgeClark - 15 Oct 2015

Excellent optimization achieved.

Regexes used to be fast, alas they aren't anymore on unicode text.

I recently investigated a considerable slow-down upgrading from Foswiki 1.1.9 to 2.0.3. It turned out that the system's default perl 5.18.2 was the problem. Regexing unicode strings was back to normal speed on a more recent perl, in this case installing a local 5.22.1 did the trick.

But looking at the patch I get the impression that the new code is more efficient not only because of avoiding regexes.

Avoiding regexes is not something we should encourage as that's not the mindset when programming perl. However we have to face the situation of linux distributions that currently almost all ship a perl (mostly 5.18.2) that is not able to regex unicode text flawlessly. Foswiki will inevitably be slower by a factor of 5-20 (see also perlunicode). This also affects a lot simpler code than putBackBlocks.

-- MichaelDaum - 21 Jan 2016

ItemTemplate edit

Summary The putBackBlocks and _putBackProtected routines are extremely inefficient.
ReportedBy GeorgeClark
Codebase 2.0.2, 2.0.1, 2.0.0
SVN Range
AppliesTo Engine
Component FoswikiRender, Performance
Priority Enhancement
CurrentState Closed
Checkins distro:f379321b1f13
TargetRelease patch
ReleasedIn 2.0.3
CheckinsOnBranches master
masterCheckins distro:f379321b1f13
Topic revision: r4 - 21 Jan 2016, MichaelDaum
The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License    Legal Imprint    Privacy Policy