You are here: Foswiki>Tasks Web>Item10222 (07 Jun 2012, MichaelDaum)Edit Attach

Item10222: bin/rename across webs with many topics is extremely slow

pencil
Priority: Urgent
Current State: Duplicate
Released In: n/a
Target Release: n/a
Applies To: Engine
Component: FoswikiUIRename, Performance
Branches:
Reported By: KiltBear
Waiting For:
Last Change By: MichaelDaum
Using the standard renaming process of going to More topic actions... both 1.1.2 released version of Rename.pm and the 1.1.3 checked in version (installed on my 1.1.2 wiki) seem to work, but with either version of Rename.pm it takes about 2-3 minutes for the redirectto page to finally show if one of the webs involved has many topics.

Example: I have 3 webs I am working between: Quest (238 topics), Sandbox (26 topics), Science (5373 topics). If I rename my test topic between Quest & Sandbox, the renaming process performs in an acceptable amount of time. However, if I try to go to or from the Science web it takes about 2 minutes before the redirectto the resultant page shows up. So... I think even though there is no updating of topics going on, the Rename.pm function is some how processing through all topics (though not changing them.)

It is not an issue when renaming within a web.

-- KiltBear - 03 Jan 2011

I tried to reproduce on my own site in comparison with 1.0.9, but had inconclusive results. In fact 1.0.9 was 5x slower, but I use VMs and who knows, maybe that server had high load (hard to tell without VPN'ing in). Also my trunk site uses MongoDBPlugin, but I don't think that helps in a rename.

However, on trunk.foswiki.org/Tasks, which has 6286 topics, I got 27.26s moving from Tasks to Sandbox, and on foswiki.org which is still 1.0.9, I got 21.45s.

Of course, I don't think ~20s rename pauses are acceptable to any user, but you didn't really say that this was worse than 1.0.x - is that the case? Or did your web only become this slow at renaming after upgrading to 1.1?

-- PaulHarvey - 07 Jan 2011

Also, are you using FastCGIEngineContrib or ModPerlEngineContrib?

-- PaulHarvey - 07 Jan 2011

Hi Paul, I am not using FastCGIEngineContrib or ModPerlEngineContrib. I've not come across this before because I have not moved things between webs much, and never to and from this "large" Science web. Between the two smaller webs it's about an 8 second wait time.

What is really strange here, though is that within a web, the rename is very fast in comparison (3 seconds). It only happens when you cross webs, which makes it sound like something really wonky is going on.

-- KiltBear - 07 Jan 2011

I was thinking that this might be "by design" (we know backlinks/renames needs a rethink), because the inter-web move might make the backlink search scan more webs, whereas the intraweb move doesn't (unless you tell it to).

But if a rename within science web is only 3 seconds, but in/out takes 2+ minutes, there should be a possible fix.

-- PaulHarvey - 08 Jan 2011

Hi Paul, Sorry for my lack of clarity. The delay in the rename happens AFTER the searching for backlinks. I've done the oopsmore page, I get to the rename page, it's done all of the searching for referring pages.

Even if there are no referring pages, it's from that point where I hit the rename button and it starts the copy of the topic to the new web that it takes two minutes.

Additionally, the topic.txt and topic.txt,v are indeed copied over within the first few seconds. (watching on the linux server side I see the move happen) It just sites there for the next two minutes chunking through things it doesn't need to chunk through.

As an aside: When I moved a DataForm topic that was attached to hundreds of topics across the web Science web to the Quest web, the rename and the modification of hundreds of topics to point to the DataForm on the new web happened in about that same two minutes of time.

-- KiltBear - 08 Jan 2011

That's okay, I knew what you meant in my last comment - I suspect that even though you've already done the backlink search and made a selection on which topics to update, then actual rename does the backlink search again.

-- PaulHarvey - 09 Jan 2011

The back link search doesn't take two minutes though. It happens within seconds, so I would guess it's doing more than just searching for them again.

-- KiltBear - 09 Jan 2011

I agree. Set back to confirmed.

-- PaulHarvey - 11 Jan 2011

Although this is a very unfortunate defect, we have more important release blockers that need resolution. I don't think this should block 1.1.3

-- PaulHarvey - 14 Jan 2011

Agree. It is more a fundamental performance issue than an urgent bug for next patch release.

-- KennethLavrsen - 01 Mar 2011

Why do renames have to be synchronous? Maybe it should be a high-priority housekeeping task that will be done sooner than later, but doesn't hold up the GUI.

-- BobBagwill - 20 May 2011

The initial backlink search is a simple grep regex which accepts a lot of false positive results and only looks for a simple match in every topic. The actual rename uses the list of "selected" topic posted with the request, but uses a much more complex regex to do the substitutions in each topic. It applies much more precise rules for linking references, quoted names in macros, template references, web-qualified links for cross-web references, squabbed links [[]] in noautolink sections, explicit URL references, etc. So this also depends upon the size/complexity of the topics being scanned for the backlinks.

Unfortunately we've probably made this worse as bugs have been opened/fixed for missed backlinks. Template references were recent additions, as were quoted wikiwords for macro parameters, and italic wikiwords. All of these were missed on 1.0.x.

-- GeorgeClark - 20 May 2011

the problem is, the initial backlink search returns no backlinks. There are no backlinks to fix. There are no topics selected to be fixed. It seems like it is scrubbing through all existing topics even though none have been highlighted to be adjusted.

-- KiltBear - 20 May 2011

Something strange is going on. With no topics selected, the rename should be really fast..no more searching going onm. This needs some tracing.

-- GeorgeClark - 21 May 2011

I'm missing something here in setup to recreate this on Foswiki 1.1.3.. I created 5123 topics in a Sandbox web. FillerTopic1 -> FillerTopic5123. Then created SomeOtherTopicName. Renamed it to the Main web, and then renamed it back to the Sandbox web. Renames only took a couple of seconds - no significant delays.

What am I missing. Do the filler topics need to be more complex than simple "This is a fillter topic". Does it need any parent or metadata associated?

-- GeorgeClark - 23 May 2011

See findings at Item11929

-- MichaelDaum - 07 Jun 2012
 

ItemTemplate edit

Summary bin/rename across webs with many topics is extremely slow
ReportedBy KiltBear
Codebase 1.1.5, 1.1.5 RC2, 1.1.5 RC1, 1.1.4, 1.1.4 RC2, 1.1.4 RC1, 1.1.4 beta2, 1.1.4 beta1, 1.1.3, 1.1.3 RC1, 1.1.3 beta1, 1.1.2, 1.1.1, trunk
SVN Range
AppliesTo Engine
Component FoswikiUIRename, Performance
Priority Urgent
CurrentState Duplicate
WaitingFor
Checkins
TargetRelease n/a
ReleasedIn n/a
CheckinsOnBranches
trunkCheckins
Release01x01Checkins
Topic revision: r17 - 07 Jun 2012, MichaelDaum
The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License    Legal Imprint    Privacy Policy