Item9838: WikiGroups runs into "premature end" error if you have many groups

Priority:	CurrentState:	AppliesTo:	Component:	WaitingFor:
Urgent	Closed	Engine		Main.GeorgeClark

Reported by GuentherFischer via mailing list.

I copied most parts of my old Main-Web like WikUsers, *Group SitePreferences etc. and have now a big problem with WikiGroups. The listing of the Groups goes to Error: Premature end of script headers: view

I nearly have 120 Grroups.

Then I delete all my own Groups and add only Groups step by step and test the WikiGroups-page. It works with 3...10 Groups but gets slower and slower till the error ocours ...

-- GuentherFischer - 18 Oct 2010

Can you please c'n'p the whole error message (from apaches errorlog) and tell me your LoginManager, PasswordManager and UserMapper settings?

-- OliverKrueger - 18 Oct 2010

Hi Oliver,

[Mon Oct 18 11:20:14 2010] [error] [client 134.109.201.15] Premature end of script headers: view, referer: https://twiki-test.hrz.tu-chemnitz.de/bin/view/Main/WebHome

$Foswiki::cfg{LoginManager} = 'Foswiki::LoginManager::TemplateLogin'; $Foswiki::cfg{PasswordManager} = 'Foswiki::Users::HtPasswdUser'; $Foswiki::cfg{UserMappingManager} = 'Foswiki::Users::TopicUserMapping';

I uncomment the line as you write amail before:

Simply uncomment the return in $this->{CACHED} in http://trac.foswiki.org/browser/trunk/TopicUserMappingContrib/lib/Foswiki/Users/TopicUserMapping.pm#L1529

and it seems to solve the problem for now ... a litle bit frustrated about such a magic BUG? in the new major release but also happy about the good and quick support.

-- GuentherFischer - 18 Oct 2010

If you can reproduce the bug by creating some more then 10 Groups there should be a patch release in a short time. If I can help with more testing let me know.

-- GuentherFischer - 18 Oct 2010

-- GuentherFischer - 18 Oct 2010

Hi again, I do some more tests ... I does copy some more Groups to Foswiki-1.1.0 and see now:

• before uncomment the line above I has an upgrade button in the GroupsList for all old Group's
• now this botton is missing

So I think it helps me to go forward, but it is not the real solution ?! Is there a script to convert old Groups to new (all data in the META)?

-- GuentherFischer - 18 Oct 2010

On my server it takes a little more than 30 seconds to show WikiGroups with 220 groups.

I have attached a zip with 220 or so groups and a couple of users. This can help Oliver and others reproducing and now I have created them, it can save others time doing the same.

-- KennethLavrsen - 18 Oct 2010

Oliver, I've checked in the one-line fix reactivating the cache.

-- GeorgeClark - 18 Oct 2010

I have checked the time it takes to render WikiGroups with or without the cache and it still takes 33 seconds

If I view the topic as guest so I have no access to changing the groups the same topic loads in 10 seconds

I think the problem is the way we have built the user interface. We end up adding 220 Twisties in this case and 220 buttons

I think we may need to rethink this.

I reuploaded the zip with the many groups . the first had access rights for viewing the topic which was a hack I had tried earlier to root cause another bug

-- KennethLavrsen - 18 Oct 2010

I've committed a small change to GroupViewTemplate and WikiGroups. This adds another parameter %maint%. If maint is set to off, all of the twisties and individual group maintenance forms are not included. On my test system, for the 220 groups, it reduced page load time from 23 seconds to approx. 4 seconds. (from yslow).

-- GeorgeClark - 19 Oct 2010

I take the changed files TopicUserMapping.pm, GroupViewTemplate and WikiGroups from trunk/ and it works for me. I see all the upgrade buttons and the add/remove buttons in the upgraded groups. The time needed for WikiGroups is 10-12 seconds. In this context: I have TWiki/Foswiki since 2003. So there are different lines in ther WikiUsers like

• ThomasJohn - ThomasJohn - 16 May 2008
• ThomasKademann - tkad - 01 Mar 2007
• ThomasHering - 02 Mar 2004

The first a Regsitration for extern users (htpasswd), the second local user with Shiboleth Auth and the third old Regstration (htpasswd). I use "$Foswiki::cfg{Register}{AllowLoginName} = 1;" since Foswiki and ShibLoginManager. Could this make your Algorithm slow? Should I consolidate ...

-- GuentherFischer - 19 Oct 2010

Guenther - if you are down at 10-12 seconds with 200 groups and still having the twisties for adding and removing group members visible, then I guess it was the re-enabling of the cache that did the trick for you.

With respect to the extra setting to turn off the UI for adding/removing users from groups, I see it a bit as an emergency hack and I hope UI innovators like ArthurClemens may come up with a new idea. It could be that instead of 200 twisties we could have a small piece of JS that calls up the UI when needed via rest or similar. There must be other ways that do not put such a load on server and browser.

Guenther - you can always try to manually update your WikiUsers topic but I doubt this was the root cause because the groups consists of WikiNames so there should be no translation needed between canonical/login IDs to WikiNames in the process of listing group members.

-- KennethLavrsen - 19 Oct 2010

Kenneth, George, Oliver, Olivier - thank you for your help. I hope the patch will be visible for other Foswiki-1.1.0 downloaders in a short time ...

BTW - with enabling cache the WikiGroups with less time (2 sec or so) if no group has changed ...

-- GuentherFischer - 19 Oct 2010

I have to add another problem.

• I see duplicate username in the groups - I think there is another Task !?
• I can't remove users from the group:
• I test to add -> it works
• I test to remove this user it works

If I look in the real text file with vim, I see the names in form of Main.UserName. Then I remove all Main. pattern in the setting. After that I can remove the usernames from the group. But I think the form Main.UserName should work ...

-- GuentherFischer - 19 Oct 2010

It is very important for us to track issues that each bug has its own bug report.

So we need a new bug report for the above.

-- KennethLavrsen - 19 Oct 2010

I am closing this now as Waiting For Release.

I have done many fixes that may also fix the problem Guenther added. The duplicate users should be fixed I believe.

Guenther try and apply the fixes for Item9848 and see if you still have problem. If so open new bug report.

-- KennethLavrsen - 20 Oct 2010