Item10826: Upgrade to foswiki 1.1 breaks Working together with CommentPlugin to create HolidaylistPlugin entries

We are in the upgrading process from foswiki 1.0.10 to 1.1.3.

The upgrade entirely breaks our applications which rely on the interplay of FormPlugin and CommentPlugin to generate content for ActionTrackerPlugin and HolidaylistPlugin. For details of our use case, compare Item9082, where we explain which input data we need to post-process with FormPlugin before writing the data with CommentPlugin.

As a simple test case, just use the example in FormPluginExamples, as it was given in foswiki 1.0. It has been removed in the meantime; I really wonder why...

========================== START CODE =================================

---++ Working together with <nop>CommentPlugin to create <nop>HolidaylistPlugin entries

We want to create a form to enter holiday data: a 'from' date and a 'until' date. The default date format from <nop>HolidaylistPlugin is:
<verbatim>
   * from_date - until_date - wikiusername
</verbatim>

We want an easy entry form, but the data must not get corrupted. So:
   * The 'From' field must be filled in
   * The 'Until' field will not be displayed if not filled in, so no dash will be displayed
   * Entering data is only possible if logged in, otherwise we would get <nop>WikiGuest entries
   
<verbatim style="display:none;">
%TMPL:DEF{PROMPT:holiday_data}%
<div class="foswikiFormSteps">
<div class="foswikiFormStep">
%STARTFORM{
name="holiday_data"
action="save"
method="POST"
topic="%WEB%.%TOPIC%"
anchor="NewData"
}%
<input type="hidden" name="comment_action" value="save" />
<input type="hidden" name="comment_type" value="holiday_data" />
<input type="hidden" name="comment_templatetopic" value="%WEB%.%TOPIC%"  />
%FORMELEMENT{
name="From"
type="date"
dateformat="%Y/%m/%d"
size="9"
title="I am on holidays from:"
disabled="%IF{"$WIKINAME='WikiGuest'" then="on" else="off"}%"
validate="nonempty"
}%
%FORMELEMENT{
name="Until"
type="date"
dateformat="%Y/%m/%d"
size="9"
title="Until:"
disabled="%IF{"$WIKINAME='WikiGuest'" then="on" else="off"}%"
}%
%FORMELEMENT{
name="comment_from_date"
type="hidden"
default="$From"
condition="$From=nonempty"
}%
%FORMELEMENT{
name="comment_to_date"
type="hidden"
default=" - $Until"
condition="$Until=nonempty"
}%
%FORMELEMENT{
name="addmydates"
type="submit"
buttonlabel="Add my dates"
disabled="%IF{"$WIKINAME='WikiGuest'" then="on" else="off"}%"
}% %IF{"$WIKINAME='WikiGuest'" then="<em>Log in to add your dates</em>"}%
%ENDFORM%
</div>
</div>
%TMPL:END%
%TMPL:DEF{OUTPUT:holiday_data}%   * %URLPARAM{"comment_from_date"}%%URLPARAM{"comment_to_date"}% - %WIKIUSERNAME%
%POS:AFTER%%TMPL:END%
</verbatim>

Try it out:
#NewData

%COMMENT{type="holiday_data" noform="on" templatetopic="%WEB%.%TOPIC%"}%

========================== END CODE =================================

[The above code was already modified since now it seems, that suddenly the missing name in the submit FORMELEMENT produced an error, and so did the missing disabled option for else ...]

In my eyes it is essential, that things which were formerly advertised in code examples, won't stop working with an upgrade.

Any workaround is welcome, but a fix would be most appreciated!

-- PhilippLeufke - 01 Jun 2011

The name parameter is a different issue, it is mandatory for a longer time.

I removed the example because I couldn't get FormPlugin work together with CommentPlugin, even after hours of work. Something complex is going on behind the scenes.

-- ArthurClemens - 01 Jun 2011

Wow, this is really scary and depressing. I'm pretty upset, since this was/is a well-known use case, even explained in the example topic.

We really depend on the interplay of CommentPlugin and FormPlugin! And on the other hand we cannot just stay with foswiki 1.0.10, as in the meantime a lot of important (security-) bugs have been fixed, as you know...

Adding Crawford to the WaitingFor field, hoping that the bug can be fixed with combined efforts.

-- PhilippLeufke - 02 Jun 2011

Item9082 says it's fixed. Is it? If not, what is the problem? Can you generate a cut down, simple testcase?

-- CrawfordCurrie - 06 Jun 2011

Thanks for your reply, Crawford.

Well, yes, Item9082 was solved. If it still is, I can't say, as the bug reported here in this topic prevents CommentPlugin from writing any text to the specified target topic. I just mentioned Item9082 for clarifying our use case and why we need depend on a working interplay of CommentPlugin and FormPlugin. Maybe I should have just repeated the description of our use case instead of referencing Item9082. Sorry for the confusion...

A simple test case? Yes. Just take the code block above; it is a copy of an example which was originally delivered with Foswiki 1.0.10 (and some versions below). I just added/edited a few things to fix some obvious bugs.

-- PhilippLeufke - 06 Jun 2011

[Edit: Changed Summary, as the " -characters broke the tasks summary table: last edit was claimed to be 41 years ago, but this bug I will post another time...]

-- PhilippLeufke - 06 Jun 2011

I haven't nailed this down - FormPlugin is unknown territory for me - but the first, most obvious thing is that your form has a "Add my dates" button that has the name "submit". When JS comes to submit that form, it picks up the "submit" field instead of the "submit" function.

-- CrawfordCurrie - 07 Jun 2011

Thanks, Crawford, that was scatterbrained mistake I made for the sake of providing a short code example, as I had to add the name field, which was not present in Foswiki 1.0.10's FormPluginExamples. I fixed it now.

But unfortunately, the issue is still unsolved.

Just copy-paste the above code example to a sandbox topic and save it. Enter two dates into the date fields and submit. The data should then be written to the same topic at the NewData anchor. But no data gets written at all. No errors in the apache log as well. Just no effect.

Pasting the same code into a Foswiki 1.0.10 environment -- and removing all else="off" statements to make the date selectors work --, yields the expected result. I just checked it again.

What else can I try for debugging?

-- PhilippLeufke - 07 Jun 2011

Part of the story: for some reason the comment plugin generated hidden fields are rendered at the very end of TMPL:DEF, so after ENDFORM. So these fields are outside the form.

So you need now to move the STARTFORM / ENDFORM macros around the COMMENT:

%STARTFORM{
   name="holiday_data"
   action="save"
   topic="%WEB%.%TOPIC%"
   anchor="NewData"
}%
%COMMENT{type="holiday_data" noform="on" templatetopic="%WEB%.%TOPIC%"}%
%ENDFORM%

This does not solve the issue yet though.

I also don't get it to work on FW 1.0.10.

And please don't add a COMMENT to this topic - the comments will end up in the verbatim block

-- ArthurClemens - 07 Jun 2011

Try version 2.0.3, just uploaded.

Not a complete fix yet, because now condition is clearly not parsed correctly.

-- ArthurClemens - 07 Jun 2011

OK, I'm trying to boil the problem down by reducing the code required to reproduce the bug. So I left away some formatting as well as the post-processing of input data for now.

This is the boiled-down code that works in Foswiki 1.0.10:

<verbatim style="display:none;">
%TMPL:DEF{PROMPT:holiday_data}%
%STARTFORM{
name="holiday_data"
action="save"
method="POST"
topic="%WEB%.%TOPIC%"
}%
<input type="hidden" name="comment_action" value="save" />
<input type="hidden" name="comment_type" value="holiday_data" />
<input type="hidden" name="comment_templatetopic" value="%WEB%.%TOPIC%"  />
%FORMELEMENT{
name="mydate"
type="date"
dateformat="%Y/%m/%d"
size="9"
title="I am on holidays on:"
validate="nonempty"
}%
%FORMELEMENT{
name="addmydates"
type="submit"
buttonlabel="Add my dates"
}%
%ENDFORM%
%TMPL:END%
%TMPL:DEF{OUTPUT:holiday_data}%   * %URLPARAM{"mydate"}% - %WIKIUSERNAME%
%POS:AFTER%%TMPL:END%
</verbatim>

%COMMENT{type="holiday_data" noform="on" templatetopic="%WEB%.%TOPIC%"}%

Unfortunately I didn't succeed in converting this to make it work with Foswiki 1.1.3 and FormPlugin 2.0.3. I tried to move the COMMENT statement up, just before ENDFORM, but then I got in trouble with the verbatim block which hides the form- and template definition from the user, which is also important, of course, but I dropped the verbatim tags for now...

If I then fill out the form and submit it, the whole form definition then gets appended to the topic, but this time with the date I chose. The URL of the topic then features the anchor #FormPluginNotification. Why?

This is the code I tried in Foswiki 1.1.3:

%TMPL:DEF{PROMPT:holiday_data}%
%STARTFORM{
name="holiday_data"
action="save"
method="POST"
topic="%WEB%.%TOPIC%"
}%
<input type="hidden" name="comment_action" value="save" />
<input type="hidden" name="comment_type" value="holiday_data" />
<input type="hidden" name="comment_templatetopic" value="%WEB%.%TOPIC%"  />
%FORMELEMENT{
name="mydate"
type="date"
dateformat="%Y/%m/%d"
size="9"
title="I am on holidays on:"
validate="nonempty"
}%
%FORMELEMENT{
name="addmydates"
type="submit"
buttonlabel="Add my dates"
}%

%COMMENT{type="holiday_data" noform="on" templatetopic="%WEB%.%TOPIC%"}%

%ENDFORM%
%TMPL:END%
%TMPL:DEF{OUTPUT:holiday_data}%   * %URLPARAM{"mydate"}% - %WIKIUSERNAME%
%POS:AFTER%%TMPL:END%

-- PhilippLeufke - 08 Jun 2011

Have a look at the updated examples in Sandbox.FormPluginExamples.

-- ArthurClemens - 08 Jun 2011

Thanks, that's already a big help. It doesn't write at the NewData anchor position, though...

• No, they seem to be written inside the form, at the end. -- AC - 08 Jun 2011

And beforeclick options are displayed in the form, but don't get used as default when not modified.

• This uses placeholder attribute with the placeholder plugin for JQueryPlugin. -- AC - 08 Jun 2011

-- PhilippLeufke - 08 Jun 2011

With the most recent version of the plugin, the error is back again
All other plugins are also up-to-date.

How to reproduce: Fill your dates in Sandbox/FormPluginExamples#Working_together_with_CommentPlugin_to_create_HolidaylistPlugin_entries, submit and confirm, that the data is written nowhere. Unfortunately the apache2 error.log is silent about this as well.

This is really bad for us. How can I try to debug? Quick help is highly appreciated...

-- PhilippLeufke - 11 Aug 2011

As I suffer from the same problem, I just noticed an error message from firebug. When I submit the form the message form.submit is not a function gets displayed. This function is called in a javascript in the topics header:

<script type="text/javascript">
//<![CDATA[
....
      submitHandler: function(form) {
         $(':submit', 'form[name=box_shedule]').addClass('foswikiSubmitDisabled');
         substituteFieldTokens(form);
         form.submit();
      }
....
//]]>
</script><!--FormPlugin_box_shedule_validation-->

-- HolgerHain - 12 Aug 2011

That happens if your submit action is called "submit". Call it "action" for example:

%FORMELEMENT{
   name="action"
   type="submit"
   buttonlabel="Save"
}%

@Philipp: I must admit I am struggling with the redirect code. Some actions require to "repaint" with print "Status: 307\nLocation: $url\n\n"; after a redirect, and some give wrong results. In the meantime you can install the previous version again.

-- ArthurClemens - 12 Aug 2011

I reverted to version 2.0.4 and everything seems fine again. Waiting for the fixed recent version then...

-- PhilippLeufke - 12 Aug 2011

Should be fixed in version 2.2.0. Took me a lot of rewriting and testing, and I am still not sure if I have covered all cases.

Please note that I rewrote the HolidaylistPlugin example to make it work.

-- ArthurClemens - 14 Aug 2011

Thanks for the quick reaction. I confirm, that FormPluginExamples#Working_together_with_CommentPlugin_to_create_HolidaylistPlugin_entries works now.

But with our modified (extended) version of it, I get the following error report rendered above the form:

Some required information is missing or incomplete:
Occasion: this field has not been passed when constructing this form - use FORMELEMENT instead of HTML markup or set strictverification="off".
action33: this field has not been passed when constructing this form - use FORMELEMENT instead of HTML markup or set strictverification="off".
comment_occasion: this field has not been passed when constructing this form - use FORMELEMENT instead of HTML markup or set strictverification="off".
comment_action: this field has not been passed when constructing this form - use FORMELEMENT instead of HTML markup or set strictverification="off".
Until: this field has not been passed when constructing this form - use FORMELEMENT instead of HTML markup or set strictverification="off".
comment_who: this field has not been passed when constructing this form - use FORMELEMENT instead of HTML markup or set strictverification="off".
comment_type: this field has not been passed when constructing this form - use FORMELEMENT instead of HTML markup or set strictverification="off".
comment_from_date: this field has not been passed when constructing this form - use FORMELEMENT instead of HTML markup or set strictverification="off".
comment_templatetopic: this field has not been passed when constructing this form - use FORMELEMENT instead of HTML markup or set strictverification="off".
Icon: this field has not been passed when constructing this form - use FORMELEMENT instead of HTML markup or set strictverification="off".
Who: this field has not been passed when constructing this form - use FORMELEMENT instead of HTML markup or set strictverification="off".
comment_to_date: this field has not been passed when constructing this form - use FORMELEMENT instead of HTML markup or set strictverification="off".
comment_index: this field has not been passed when constructing this form - use FORMELEMENT instead of HTML markup or set strictverification="off".
comment_icon: this field has not been passed when constructing this form - use FORMELEMENT instead of HTML markup or set strictverification="off".
From: this field has not been passed when constructing this form - use FORMELEMENT instead of HTML markup or set strictverification="off".

The code we use for this is:

<verbatim style="display: none;">
%TMPL:DEF{PROMPT:holiday_data}%
<div class="foswikiFormSteps">
<div class="foswikiFormStep">
<input type="hidden" name="comment_action" value="save" />
<input type="hidden" name="comment_type" value="holiday_data" />
<input type="hidden" name="comment_templatetopic" value="%WEB%.%TOPIC%"  />
%FORMELEMENT{
   name="Who"
   type="text"
   title="Who:"
   validate="nonempty"
   default="%FORMFIELD{"FirstName" topic="%WIKIUSERNAME%"}%"
   mandatory="on"
   hint="First name or like written in HolidayList"
}%
%FORMELEMENT{
   name="From"
   type="date"
   dateformat="%d %b %Y"
   size="9"
   title="Start Date:"
   disabled="%IF{"$WIKINAME='WikiGuest'" then="on" else="off"}%"
   validate="nonempty"
   mandatory="on"
}%
%FORMELEMENT{
   name="Until"
   type="date"
   dateformat="%d %b %Y"
   size="9"
   title="End Date:"
   disabled="%IF{"$WIKINAME='WikiGuest'" then="on" else="off"}%"
}%
%FORMELEMENT{
   name="Occasion"
   type="text"
   title="Occasion for holidays (optional):"
   beforeclick="Holidays"
   default="Holidays"
   validate="nonempty"
}%
%FORMELEMENT{
   name="Icon"
   type="text"
   beforeclick="8-)"
   default="8-)"
   hint="Enter a [[System.SmiliesPlugin][Smilie]], like e.g. =:-)= or an [[System/DocumentGraphics][Icon]], like e.g. % ICON{wip} % (without the spaces)"
   title="Smilie or Icon (optional):"
}%
%FORMELEMENT{
   name="comment_from_date"
   type="hidden"
   default="$From"
   condition="$From=nonempty"
}%
%FORMELEMENT{
   name="comment_to_date"
   type="hidden"
   default=" - $Until"
   condition="$Until=nonempty"
}%
%FORMELEMENT{
   name="comment_who"
   type="hidden"
   default=" - $Who"
   condition="$Who=nonempty"
}%
%FORMELEMENT{
   name="comment_occasion"
   type="hidden"
   default=" - $Occasion"
   condition="$Occasion=nonempty"
}%
%FORMELEMENT{
   name="comment_icon"
   type="hidden"
   default=" - $Icon"
   condition="$Icon=nonempty"
}%
%FORMELEMENT{
   name="action33"
   type="submit"
   buttonlabel="Add my dates"
   disabled="%IF{"$WIKINAME='WikiGuest'" then="on" else="off"}%"
}% %IF{"$WIKINAME='WikiGuest'" then="<em>Log in to add your dates</em>"}%
</div>
</div>
%TMPL:END%
%TMPL:DEF{OUTPUT:holiday_data}%   * %URLPARAM{"comment_from_date"}%%URLPARAM{"comment_to_date"}%%URLPARAM{"comment_who"}%%URLPARAM{"comment_occasion"}%%URLPARAM{"comment_icon"}%
%POS:AFTER%%TMPL:END%
</verbatim>

---+++ New Entry
%STARTFORM{
name="holiday_data"
action="save"
topic="%WEB%.%TOPIC%"
anchor="NewData"
}%
%COMMENT{type="holiday_data" noform="on" templatetopic="%WEB%.%TOPIC%"}% %STARTINCLUDE% 

%ENDFORM%
#NewData

The same applies to our form which we use in combination with CommentPlugin and ActionTrackerPlugin.

Although version 2.0.4 is working nicely, I'd rather prefer to have our forms compatible with the most recent plugin version for the future...

-- PhilippLeufke - 15 Aug 2011

This is in the release notes. But you can also do what the warning says: either add strictverification="off" to STARTFORM, or change the HTML input fiels to FORMELEMENTS. Like the updated example in FormPluginExamples.

-- ArthurClemens - 15 Aug 2011

Well, right, I missed that. Sorry...

Everything seems fine now; thanks for your efforts, Arthur!

-- PhilippLeufke - 15 Aug 2011