Duplicate notifications from MailerContrib for some, but not all users. Why?

Question

Duplicate notifications from MailerContrib for some, but not all users. Why?

[Detail not relevant to the solution removed in final update.]

Crontab:
7 0 * * * /var/www/twiki/tools/runmailnewsnotify
7 0 * * * /var/www/twiki/tools/runmailwebnotify

cat /var/www/twiki/tools/runmailnewsnotify
#!/bin/bash
cd /var/www//twiki && /usr/bin/perl -I bin tools/mailnotify -q -news '*'

cat /var/www/twiki/tools/runmailwebnotify
#!/bin/bash
cd /var/www//twiki && /usr/bin/perl -I bin tools/mailnotify -q '*'

Note: MailerContrib was updated to latest version 01-Apr-2008; no change.

Any clues would be appreciated...

Environment

TWiki version:	TWikiRelease04x01x02	edit
TWiki plugins:	CalendarPlugin, CleanupRemoteUserPlugin, CommentPlugin, EditTablePlugin, ImageGalleryPlugin, InterwikiPlugin, PreferencesPlugin, RenderListPlugin, SlideShowPlugin, SmiliesPlugin, SpreadSheetPlugin, SubscribePlugin, TablePlugin, TwistyPlugin	edit
Server OS:	Fedora Core 6 - 2.6.22.14-72	edit
Web server:	Apache	edit
Perl version:	5.8.8	edit
Client OS:	XP SP2	edit
Web Browser:	IE7	edit
Categories:	Plugins %SUPPORTCATEGORIES%	edit

-- TimotheLitt - 02 Apr 2008

Answer

%STATUS_REMINDER%

Try this: Please do not run both commands at the same time. According to your crontab, runmailwebnotify is invoked while runmailnewsnotify is still running.

In otherwords, crontab should be one line:

7 0 * * * /var/www/twiki/tools/runmailnotify 

And runmailnotify should be

#!/bin/bash /var/www/twiki/tools/runmailnewsnotify /var/www/twiki/tools/runmailwebnotify 

Or even,

#!/bin/bash  cd /var/www//twiki && /usr/bin/perl -I bin tools/mailnotify -q -news '*'  cd /var/www//twiki && /usr/bin/perl -I bin tools/mailnotify -q '*'  

-- GilbertHerschberger - 20 May 2008

Thanks for the hint!

This restriction wasn't obvious from the documentation - in fact, the installation instructions have a heading talking about "creating your cron job(s)".

Seems like an interlock is missing in the code - shouldn't be the user's job to manage synchronization.

Here is the final resolution. I consolidated all the maintenance into a single cron job/script. This eliminated the duplicate notifications.

The script can be run several times a day if frequent notifications are desired; it will run weekly processing only once /wk and daily processing only once a day - on selectable day/time. (I run the old wrapper scripts for convenience when manually forcing an update; one could in-line them if desired.)

You might want to use this as an example in the documention or include it in the distribution. Nonetheless, the lack of synchronization seems like a bug.

crontab

# # Daily wiki maintenance - 0007 #   -d debug, Day of week for weekly activities, hour for daily # 7 0 * * * /var/www/twiki/tools/twiki_maintenance Mon 00 

twiki_maintenance

#!/bin/sh
#
# Run TWiki maintenance tasks sequentially
#
# Independent crontab entries may run in parallel
# and produce duplicate notifications per
# http://twiki.org/cgi-bin/view/Support/DuplicateNotificationsFromMailerCon
#
# One crontab entry runs this periodically
#
PG=`basename $0`
TOOLS=`dirname $0`
NOW=`date +"%a|%H"`
TODAY=`echo $NOW  | sed -e 's/^\(.*\)|\(.*\)$/\1/'`
HR_NOW=`echo $NOW | sed -e 's/^\(.*\)|\(.*\)$/\2/'`
#
# -d = enable debug messages
#
if [ "$1" = "-d" ]; then
    DEBUG='Y'
    shift
fi
#
# $1 = Day of week (Mon ... Sun) to do weekly processing
#
WKLY_DAY=$1
#
if [ -z "$WKLY_DAY" ]; then WKLY_DAY="$TODAY" ; fi
#
# $2 = Hour (00-23) to do daily & weekly processing
#      (Other hours do hourly only)
#
WKLY_HR=$2
if [ -z "$WKLY_HR" ]; then WKLY_HR="$HR_NOW" ; fi
#
# Output debug message
#
function Debug () {
    if [ -n "$DEBUG" ]; then
        echo "`date` [$PG] $*"
    fi
}
#
#
#
Debug "Running on $TODAY @ $HR_NOW:xx"
#
# Stuff to do every day
#
if [ "$HR_NOW" -eq "$WKLY_HR" ]; then
    #
    # Daily at specified hour
    #
    Debug "Running statistics"
    $TOOLS/runstatistics


    if [ "$TODAY" = "$WKLY_DAY" ]; then
        #
        # Weekly processing
        #
        Debug "Running ticktwiki"
        $TOOLS/runticktwiki
    fi
fi
#
# Every hour that cron runs this script
#
Debug "Running newsnotify"
$TOOLS/runmailnewsnotify


Debug "Running webnotify"
$TOOLS/runmailwebnotify


# [End of file]

-- TimotheLitt - 14 Jul 2008

Making a lock may be a good idea but implemented the wrong way it a crash of code can mean that nothing will work again in a dead lock so who ever decides to make a lock must make it fool proof.

-- KennethLavrsen - 14 Jul 2008

Upgrading to new twki and realized that my example had hardcoded the tools directory. Updated to figure it out at runtime. Perhaps it will help someone else.

-- TimotheLitt - 14 Oct 2008

I filed Tasks.Item2460 - our mailnotify script really shouldn't need a wrapper script - rather than making locks, its actually simpler to make queues and to process sequentially..

No such template def 'PROMPT:supportquery'