Item2282: Statistics script not working

The statistics script is not working. When I try to force an update the script goes on indefinitely.

Here are the details of my Foswiki installation-

• Server - Apache 2.2.9
• OS - Windows 2003 Server Edition
• Foswiki - 1.0.7
• Strawberry Perl 5.10

For authentication, I am using the mod_auth_sspi module of Apache in order to authenticate against the Active Directory in my intranet.

---

I just tested this on a random web on:

• Apache 2.2.6
• Windows XP pro
• Foswiki 1.0.7
• Active State perl

Works perfectly. Which suggests that there is something weird about your installation. Can you provide any more information? For example, are there any messages in any of the Foswiki or apache logs that might suggest a problem?

-- CrawfordCurrie - 23 Oct 2009

GuruprasadIyer we need an update on this bug which is marked urgent for 1.0.8 release or I will need to No Action it and regard it as a support question.

-- KennethLavrsen - 04 Nov 2009

---

There are no messages in the Foswiki log file. However the Apache log file has a status code of 401 which indicates some issue with authentication. Also, after forcing an update I get an "Internal Server Error" page after a long time.

-- GuruprasadIyer - 05 Nov 2009

It is important that we get the exact full error text from the Apache log. Even if it looks cryptic it does contain information that helps us understanding what it is.

Also we need all the information we can get on how your setup with mod_auth_sspi is done. If we cannot reproduce this we cannot solve it.

An additional thing you can provide is to edit your ..../bin/LocalLib.cfg file and change

#$ENV{FOSWIKI_ASSERTS} = 1;

to

$ENV{FOSWIKI_ASSERTS} = 1;

This will make Foswiki crash with a lot of error messages in both Apache log and in the browser. These messages will help us a lot.

-- KennethLavrsen - 05 Nov 2009

---

In .../data/debug.txt, these are the lines-

| 2009-11-05T08:38:02Z debug | WebStatistics: Bad logfile line  |

| 2009-11-05T08:38:02Z debug | WebStatistics: Bad logfile line 10.118.71.26 |

Here is the .../apache/logs/access.log.

Also, once I enabled the ASSERTS setting in .../bin/LocalLib.cfg I got some errors about the use of uninitialized variables in .../lib/Foswiki/Users.pm. I had to make some changes to correct it.

Now what happens is the statistics script when forced runs as usual and displays the usage data but for only the user who is running the script, if I run it it only shows the edits I have done, if my colleague runs it then only the edits done by him show up.

-- GuruprasadIyer - 05 Nov 2009

---

I wanted to note that I am also having the exact same statistics issue on a Solaris 10 platform:

• Apache/2.0.59 (Unix) DAV/2 mod_perl/2.0.4
• Solaris 10
• Foswiki 1.0.7
• Perl/v5.8.4

Specifically, my issues are this:

• When running via cron, I get a "0" for topic saves and file uploads (which is wrong) and no one is listed under "Top Contributors".
• When running via my id (member of AdminGroup), I get a count for saves and uploads and my Id is the only one under the "Top Contributors".

I am not not 100% sure what should the correct Views, Saves, and Upload counts should be. But, I do know "0" is wrong. Also, for "Top Contributors", I know I am not the only one contributing.

Also, the above "DEBUG LOG" is exactly the same Errors I get running Statistics.

If this information needed to be under a separate bug, please let me know and I can create one. Thanks, Adam

-- AdamRoybal - 18 Nov 2009

We are not getting much help from you guys that can lead to a fix.

We need the error_log from Apache. Not the access log.

When you run with ASSERTS on we need the exact copy/paste of the errors you see in the browser. It does not help that you tell us "I got some errors about the use of uninitialized variables in .../lib/Foswiki/Users.pm". We need the exact error text. We need the whole error text. It tells which lines in the code that fails and it also tells from where the line was called. This information is vital in helping us fixing the problem.

You also say "I had to make some changes to correct it". What are "some changes"?

I am going to release 1.0.8 this weekend and we would need all the feedback requested to include a fix in this release.

-- KennethLavrsen - 25 Nov 2009

---

I have turned ASSERT on and also changed the log level in httpd.conf to debug, however there is no error mesg. in error.log of Apache. The debug.txt file in Foswiki's data folder has the same error mesg. as stated above in the earlier post by me.

I have made changes to some files in order to remove the Main. prefix from the usernames since we are not using the wiki authentication mechanism.

-- GuruprasadIyer - 25 Nov 2009

When you run with ASSERT on AND you end up with a message that contains "use of uninitialized variables in .../lib/Foswiki/Users.pm" then you have a screen full of messages. Please give us the entire message. All the 5-20 lines of it.

If you have "use of uninitialized variables in .../lib/Foswiki/Users.pm" you also have messages in the Apache error_log. We need that too.

And please give us the exact modifications to the code you have made.

-- KennethLavrsen - 25 Nov 2009

I did a fresh installation of Foswiki without any of the modifications. This time I don't get any errors after I turn on ASSERT. However, the issues is as is and there are no entries in error.log of Apache.

-- GuruprasadIyer - 26 Nov 2009

OK. This is even more of a mystery then.

I do not have a Windows 2003 server or a Strawberry perl installation so I cannot help any further on this. I will have to defer this one till after 1.0.8

-- KennethLavrsen - 29 Nov 2009

Unfortunately the apache access log is useless. We would need the Foswiki .changes log, which is stored in the 'data' directory for each web.

I just repeated the exercise of taking a clean install, configuring it on XP Home and running statistics; works perfectly. We really need more information from the reporters to progress this any further, so I'm dropping it to "Normal" status and marking for feedback.

-- CrawfordCurrie - 29 Nov 2009

More than a month with no feedback.

I am putting No action on this. Reopen anyone that has an interest in this and can provide the information that enable us to reproduce it.

-- KennethLavrsen - 09 Jan 2010

There is a line in lib/Foswiki/UI/Statistics.pm, in the subroutine _collectLogData:

$logFileUserName = Foswiki::Func::getCanonicalUserID($logFileUserName);

When I comment out this line then the statistics are reported. However, I feel there are still some anomalies in the data generated; I am looking into them.

-- GuruprasadIyer - 14 Jan 2010

---

I have an update on my Statistics issue which has been Resolved. But first, let me add a little more concerning my Foswiki set-up:

• No user registration allowed and User Topics are not required.
• The Foswiki login page is not used. Access is via a SSO token created by logging into the Sun JES Portal/Identity manager.
• Apache Web server has the "Sun JES Policy Agent" installed which restricts updates and "protected" topics unless authenticated.

With that said, my configuration file and my LocalLib.cfg were a 1-off from the standard vanilla install. I made the following changes to my configuration in order to be more in line with using Topic User Mapping.

1. I removed the ChangeAuthenticationVariable TWiki hack I used from a couple of years ago. Now, my /bin/!LocalLib.cfg does not have non-standard updates in it.
2. I re-read the configuration page and I made the following two changes (first one to get Topic User Mapping working correctly and the other to force non-mapped users to Unknown).:

   {Register}{AllowLoginName}
   $Foswiki::cfg{Register}{AllowLoginName} = 1;
   {RenderLoggedInButUnknownUsers}
   $Foswiki::cfg{RenderLoggedInButUnknownUsers} = 1

3. Lastly, I ran some LDAP queries that formatted the WikiUsers topic which I suspect must be present now for Statistics to work correctly. My old TWiki install did not need this page.

After all three of these changes were made, my statistics are now working as expected from both the ../statistics/... url or from the crontab script.

I HTHs with GuruprasadIyer's issue. Thanks.

-- AdamRoybal - 15 Feb 2010