Feature Proposal: Add +"" and key+"" option to Foswiki::Attrs

Motivation

Makes writing readable and maintainable macros very easy.

It also improves spotting differences between revisions.

The final option to allow key+"value" to extend the value of any named key rather than merely the previous one allows for even more maintainable macro expressions.

Description and Documentation

Allow a special case of +"more" to concatenate successive values together. This allows long parameters to be broken with newlines and formatted for easy reading while still being able to control whitespace. Indeed you can break up parameters wherever it makes sense.

I was able to make the code change and update topic within an hour. Crucially, the changes were easy and in particular the formatting much easier to follow: 1st time around I had added a '|' messing up the table layout but it was much easier to spot.

Another hour required for the key+"value" option.

In earlier revisions I considered += as an alternative to a simple +. After some experience using the new syntax, I found + the better option.

The community is still welcome to suggest += or other options, especially if objections can be found for +.

Examples

Before:
 format="| [[$topic][$formfield(BatchMagCode) $formfield(BatchCageDate) $formfield(BatchNumber) $formfield(BatchLocation)]]%BR%$percntBATCHDESC{\"$formfield(BatchMagCode)\" num=\"$formfield(BatchNumber)\"}$percnt | $formfield(BatchValue) | $formfield(BatchCurrencies) | $formfield(BatchSizes) | $formfield(BatchCount) | $formfield(BatchToQueue) | $formfield(BatchKilled) | $formfield(BatchIgnored) | <div style='background-color:#$percntCALC{$SET(dat,$formfield(BatchCageDate))$SET(pdat,$formfield(BatchProcessDate))$SET(age,$EVAL($percnt$formfield(BatchMagCode)_days$percnt))$IF($EXACT($percntWORKFLOWSTATE{\"$topic\"}$percnt,CHECKED),FAF0D4,$IF($GET(age)>=0,ff3300,$IF($GET(age)>=-2,ff9933,66cc33)))}$percnt'>$formfield(BatchErrors)&nbsp;</div> | $formfield(BatchOperators) | $formfield(BatchActivities) | <form name=\"Edit\" action=\"%SCRIPTURLPATH{edit}%/%WEB%/\"> <input type=\"hidden\" name=\"action\" value=\"form\"/> <input type=\"hidden\" name=\"topic\" value=\"$topic\"/> <input type=\"hidden\" name=\"redirectto\" value=\"%TOPIC%?%QUERYSTRING{encode="url"}%\"/> <input type=\"image\" src=\"%ICONURL{pencil}%\" alt=\"Edit Form\" /> </form> $formfield(BatchNotes) | $percntWORKFLOWHISTORY{\"$topic\"}$percnt | $percntWORKFLOWTRANSITION{\"$topic\" redirectto=\"%TOPIC%?%QUERYSTRING{encode="url"}%\"}$percnt |"

After:
format="| [[$topic][$formfield(BatchMagCode) $formfield(BatchCageDate) $formfield(BatchNumber) $formfield(BatchLocation)]]"
 +="%BR%$percntBATCHDESC{\"$formfield(BatchMagCode)\" num=\"$formfield(BatchNumber)\"}$percnt "
 +="| $formfield(BatchValue) "
 +="| $formfield(BatchCurrencies) "
 +="| $formfield(BatchSizes) "
 +="| $formfield(BatchCount) "
 +="| $formfield(BatchToQueue) "
 +="| $formfield(BatchKilled) "
 +="| $formfield(BatchIgnored) "
 +="| <div style='background-color:#"
 +="$percntCALC{"
 +="$SET(dat,$formfield(BatchCageDate))"
 +="$SET(pdat,$formfield(BatchProcessDate))"
 +="$SET(age,$EVAL($percnt$formfield(BatchMagCode)_days$percnt))"
 +="$IF($EXACT($percntWORKFLOWSTATE{\"$topic\"}$percnt,CHECKED),FAF0D4,$IF($GET(age)>=0,ff3300,$IF($GET(age)>=-2,ff9933,66cc33)))}"
 +="$percnt'>$formfield(BatchErrors)&nbsp;</div> "
 +="| $formfield(BatchOperators) "
 +="| $formfield(BatchActivities) "
 +="| <form name=\"Edit\" action=\"%SCRIPTURLPATH{edit}%/%WEB%/\">"
 +="<input type=\"hidden\" name=\"action\" value=\"form\"/>"
 +="<input type=\"hidden\" name=\"topic\" value=\"$topic\"/>"
 +="<input type=\"hidden\" name=\"redirectto\" value=\"%TOPIC%?%QUERYSTRING{encode="url"}%\"/>"
 +="<input type=\"image\" src=\"%ICONURL{pencil}%\" alt=\"Edit Form\" /> </form> $formfield(BatchNotes) "
 +="| $percntWORKFLOWHISTORY{\"$topic\"}$percnt "
 +="| $percntWORKFLOWTRANSITION{\"$topic\" redirectto=\"%TOPIC%?%QUERYSTRING{encode="url"}%\"}$percnt |"

Before:
 header="|*Mag Code*|*Batch Value*|*Currencies*|*Sizes*|*Count*|*To Queue*|*Killed*|*Ignored*|*Errors*|*Notes*|*History*|*Flow*|
 format="| [[$topic][$formfield(BatchMagCode) $formfield(BatchCageDate) $formfield(BatchNumber) $formfield(BatchLocation)]]%BR%$percntBATCHDESC{\"$formfield(BatchMagCode)\" num=\"$formfield(BatchNumber)\"}$percnt | $formfield(BatchValue) | $formfield(BatchCurrencies) | $formfield(BatchSizes) | $formfield(BatchCount) | $formfield(BatchToQueue) | $formfield(BatchKilled) | $formfield(BatchIgnored) | <div style='background-color:#$percntCALC{$SET(dat,$formfield(BatchCageDate))$SET(pdat,$formfield(BatchProcessDate))$SET(age,$EVAL($percnt$formfield(BatchMagCode)_days$percnt))$IF($EXACT($percntWORKFLOWSTATE{\"$topic\"}$percnt,CHECKED),FAF0D4,$IF($GET(age)>=0,ff3300,$IF($GET(age)>=-2,ff9933,66cc33)))}$percnt'>$formfield(BatchErrors)&nbsp;</div> | $formfield(BatchOperators) | $formfield(BatchActivities) | <form name=\"Edit\" action=\"%SCRIPTURLPATH{edit}%/%WEB%/\"> <input type=\"hidden\" name=\"action\" value=\"form\"/> <input type=\"hidden\" name=\"topic\" value=\"$topic\"/> <input type=\"hidden\" name=\"redirectto\" value=\"%TOPIC%?%QUERYSTRING{encode="url"}%\"/> <input type=\"image\" src=\"%ICONURL{pencil}%\" alt=\"Edit Form\" /> </form> $formfield(BatchNotes) | $percntWORKFLOWHISTORY{\"$topic\"}$percnt | $percntWORKFLOWTRANSITION{\"$topic\" redirectto=\"%TOPIC%?%QUERYSTRING{encode="url"}%\"}$percnt |"

After:
  header+"|*Control record*"
  format+"| <span style=\"display:none;\">"
        +"$formfield(BatchNumber) $formfield(BatchLocation) "
        +"$percntCALC{$SUBSTRING($topic,13,3)}$percnt $formfield(BatchCageDate) "
        +"!$formfield(BatchMagCode)"
        +"</span>"
        +"<a id=\"$topic\"/>[[$topic]]"
        +"%BR%$percntBATCHDESC{\"$formfield(BatchMagCode)\" num=\"$formfield(BatchNumber)\"}$percnt "

  header+"|*Batch Value*"
  format+"|  $formfield(BatchValue) "

  header+"|*Batch Curr- encies*"
  format+"| $formfield(BatchCurrencies) "

  header+"|*Success*"
  format+"|  $formfield(BatchCount) "

  header+"|*To Queue*"
  format+"|  $formfield(BatchToQueue) "

  header+"|*Killed*"
  format+"|  $formfield(BatchKilled) "

  header+"|*Ignored*"
  format+"|  $formfield(BatchIgnored) "

  header+"|*Batch Size*"
  format+"|  $formfield(BatchSizes) "

  header+"|*Batch Errors*"
  format+"| <div style='background-color:#"
         +"$percntCALC{$SET(dat,$formfield(BatchCageDate))"
         +"$SET(pdat,$formfield(BatchProcessDate))"
         +"$SET(age,$EVAL($percnt$formfield(BatchMagCode)_days$percnt))"
         +"$IF($EXACT($percntWORKFLOWSTATE{\"$topic\"}$percnt,CHECKED),FAF0D4,"
         +"$IF($GET(age)>=0,ff3300,$IF($GET(age)>=-2,ff9933,66cc33)))}$percnt'>"
         +"$formfield(BatchErrors) </div> "

  header+"|*Batch Oper- ators*"
  format+"| $formfield(BatchOperators) "

  header+"|*Batch Acti- vities*"
  format+"| $formfield(BatchActivities) "

  header+"|*Batch Notes*"
  format+"| <form name=\"Edit\" action=\"%SCRIPTURLPATH{edit}%/%WEB%/\">"
         +"<input type=\"hidden\" name=\"action\" value=\"form\"/>"
         +"<input type=\"hidden\" name=\"topic\" value=\"$topic\"/>"
         +"<input type=\"hidden\" name=\"redirectto\" value=\"%TOPIC%?%QUERYSTRING%#$topic\"/>"
         +"<input type=\"image\" src=\"%ICONURL{pencil}%\" alt=\"Edit Form\" /> </form> $formfield(BatchNotes)"

  header+"|*Status History*"
  format+"| $percntWORKFLOWHISTORY{\"$topic\"}$percnt "

  header+"|*Change Status*"
  format+"| $percntWORKFLOWTRANSITION{\"$topic\" redirectto=\"%TOPIC%?%QUERYSTRING%#$topic\"}$percnt "

  header+"|"
  format+"|"

Note how this enables you to better manage table layouts keeping column coding together and in turn the easy ability to move them around.

The 1st setting of a key is relaxed to allow key+"value" and not insist on key="value".

Impact

%WHATDOESITAFFECT%
edit

Implementation

See attached Attrs.zip

-- Contributors: JulianLevens - 05 Nov 2013

Discussion

Try this first:

 format="| [[$topic][$formfield(BatchMagCode) $formfield(BatchCageDate) $formfield(BatchNumber) $formfield(BatchLocation)]]%BR%\
           $percntBATCHDESC{\"$formfield(BatchMagCode)\" num=\"$formfield(BatchNumber)\"}$percnt \
         | $formfield(BatchValue) \
         | $formfield(BatchCurrencies) \
         | $formfield(BatchSizes) \
         | $formfield(BatchCount) \
         | $formfield(BatchToQueue) \
         | $formfield(BatchKilled) \
         | $formfield(BatchIgnored) \
         | <div style='background-color:#$percntCALCULATE{
              \"$SET(dat,$formfield(BatchCageDate))
               $SET(pdat,$formfield(BatchProcessDate))
               $SET(age,$EVAL($percnt$formfield(BatchMagCode)_days$percnt))
               $IF($EXACT($percntWORKFLOWSTATE{\"$topic\"}$percnt,CHECKED),FAF0D4,$IF($GET(age)>=0,ff3300, $IF($GET(age)>=-2,ff9933,66cc33)))\"
             }$percnt'>$formfield(BatchErrors)&nbsp;</div> \
         | $formfield(BatchOperators) \
         | $formfield(BatchActivities) \
         | <form name='Edit' action='%SCRIPTURLPATH{edit}%/%WEB%/'> \
             <input type='hidden' name='action' value='form' /> \
             <input type='hidden' name='topic' value='$topic'/> \
             <input type='hidden' name='redirectto' value='%TOPIC%?%QUERYSTRING{encode="url"}% '/> \
             <input type='image' src='%ICONURL{pencil}%' alt='Edit Form' /> \
           </form> $formfield(BatchNotes) \
         | $percntWORKFLOWHISTORY{\"$topic\"}$percnt \
         | $percntWORKFLOWTRANSITION{\"$topic\" redirectto=\"%TOPIC%?%QUERYSTRING{encode="url"}%\"}$percnt \
         |"

It looks even better than your += approach in my book.

-- MichaelDaum - 05 Nov 2013

I agree that it looks better - it just doesn't quite work frown, sad smile

The trailing space is lost changing the table formatting. The += approach leaves me in total control.

Of course I'm testing on 1.0.8, so maybe this has changed.

BTW where is this '\' continuation documented. I knew about it but thought it was a crude concatenation of the whole following line without trimming leading space. On irc, jast did not even know about it. I searched for 'continuation' 'multi line', 'multi-line' and came up blank. That suggest a possible documentation issue at least.

-- JulianLevens - 05 Nov 2013

Try this:

format="| left \
        | left \
        |  center  \
        |  center  \
        |  right |\
           right |"

-- MichaelDaum - 08 Nov 2013

MichaelDaum as discussed at the FoswikiCamp2014 you conceded that this was not an issue after all.

However before removing you concern, please review again as I have added some further ideas.

-- JulianLevens - 28 Mar 2014

I think this is an excellent idea. Definitely + and not += which comes preloaded with semantics. I like:

%MIDDAY_SUN{out="mad" + "dogs" + "englishmen"}%

Not so keen on

%MIDDAY_SUN{out="mad" out+="dogs" out+="englishmen"}%

because it implies an ordering over what is currently an unordered set of parameters.

-- CrawfordCurrie - 28 Mar 2014

Strictly the parameters are still unordered, but the values can be split up into fragments and joined up. The fragments do indeed have order, but the keys can still be in any order and overridden by later values (fred="default" +"value" fred="final (maybe)").

I have changed a complex topic to something simple with these ideas. Being able to keep all my 'columns' was an additional bonus contributing 20% ish to the overall benefit of this proposal.

I'll attach an alternative Attrs that makes all of this work. Anyone interested can then experiment with it in earnest it and get a feel for the pros and cons (although I think they mostly jump out at you).

It can be tweaked based on feedback before a final version is prepared for committing back to FW.

At that point I'll complete the docs and unit tests. That's where much more work is required.

-- JulianLevens - 28 Mar 2014

See attachment and try it out.

I've implemented based on +"". +="" is still possible if that's the community's preference, but we should choose and not have an either/or situation (it's more testing and support at the very least).

This changes the standard syntax as all the hacks were only made to the _unfriendly sub. The _friendly sub will need similar hacks once we have a consensus. (Where is this syntax used BTW?)

There's a test topic that you play with as well. It uses the FAQ topics in System so the required topics are readily avaialble.

  • Attrs2.zip: Attrs.pm with changes to demonstrate feature

-- JulianLevens - 29 Mar 2014

Doesn't both make sense: + for concatenating strings, and += to incrementally add to an existing parameter.

%MACRO{
  param1="hello"
  param2="foo" + "bar"
  param1+=" world"
}%

-- MichaelDaum - 29 Mar 2014

It's a choice, the attached code allows something very similar:

%MACRO{
  param1="hello"
  param2="foo" + "bar"
  param1+" world"
}%

-- JulianLevens - 29 Mar 2014

The ability to specify related parts of different parameters together would make macros much easier to understand and it would make them far less error-prone. I particularly like JulianLevens' example of grouping a column's header and format together. To me, that is more than 50% of the benefit of this proposal!.

With that in mind... Using both + and += gives you more expressive power. It allows you to differentiate between "append to previous parameter" and "append to abitrary parameter". When you make that distinction, you can append to the unnamed parameter.

%MACRO{
  param1="foo"
  "hello"
  param2="tooth" 
        +"paste"
  +=" world"
  param1="bar"
}%
would be equivalent to
%MACRO{"hello world" param2="toothpaste" param1="bar"}%
rather than
%MACRO{"hello" param2="toothpaste world" param1="bar"}%

It could still be possible to append to the unnamed parameter with only +, if you name the unnamed parameter:
%MACRO{
  param1="foo"
  "hello"
  param2="tooth" 
        +"paste"
  _DEFAULT+" world"
  param1="bar"
}%

That seems like I nasty hack. I would therefore prefer to have:
+"more"
append to the previous parameter
key+="more"
append to arbitrary named parameter
+="more"
append to the default parameter

-- MichaelTempest - 30 Mar 2014

The unnamed parameter is already named, internally. But the name is uber-geeky.

Imagine for a moment you are a manager, and you want to write a macro. You last attempted to program in 1990, and you coded in Basic. '+' is reasonably intention-revealing - it leaps out at you and says "add to the last parameter" (what's not so clear is what param1="1"+"2" means, however. Which is why many languages (including perl) don't overload the '+' operator.)

. '+=' is not intention-revealing. It's one of those things that only experienced programmers have encountered and used. And if '+=', why not '=+'? And why do you need it anyway?

I'm starting to dislike '+=' enough that I'm raising a concern. I really can't see a strong case for it. Advocates, show me one case where '+=' (or '=+') is required because you can't do what's needed using simple '+', please.

-- CrawfordCurrie - 30 Mar 2014

I see two distinct mechanisms in this proposal:
  1. Another mechanism for splitting a parameter into consecutive parts
    • %MACRO{p="a" +"b" +"c"}% becomes equivalent to %MACRO{p="abc"}%
    • There are already mechanisms for doing this. I find the existing mechanisms error-prone, so I support this new mechanism.
  2. A mechanism for splitting a parameter into non-consecutive parts
    • %MACRO{p1="a" p2="A" p1+"b" p2+"B"}% becomes equivalent to %MACRO{p1="ab" p2="AB"}%
    • I am not aware of any existing mechanisms for doing this.
    • For this mechanism, I find + and += equally geeky and non-intention-revealing
    • The problem with just + is that could be ambiguity with respect to the default parameter. How do you split up %MACRO{"ab" p2="AB"}% with just +? In conjunction with the first mechanism:
      • %MACRO{"a" p2="A" +"b" p2+"B"}% becomes equivalent to %MACRO{"a" p2="AbB"}%, whereas
      • %MACRO{"a" p2="A" +="b" p2+="B"}% becomes equivalent to %MACRO{"ab" p2="AB"}%
    • If you dispense with the first mechanism and only use + for splitting parameters into non-consecutive parts, you end up with something well-defined but potentially confusing (I certainly find it non-intuitive):
      • %MACRO{"a" p2="A" +"b" p2+"B"}% becomes equivalent to %MACRO{"ab" p2="AB"}%
    • This is the most valuable aspect of this proposal to me and I think it would be very helpful for non-programmers.
    • I would like to be able to split up the default parameter and I would like to use the same syntax for named parameters and the default parameter. These are less important to me.

I don't mind what syntax we use.

-- MichaelTempest - 30 Mar 2014

The problem being highlighted is the inconsistent treatment of the _DEFAULT parameter.

There is already an inconsistency in _DEFAULT attr treatment by Foswiki

  • %<nopSEARCH{"Banana" type="query" "Cherry" type="regex"}%
which is treated as:
  • %<nopSEARCH{"Banana" type="regex"}%

You can override the type parameter but the default remains "Banana" (there is explicit code like: $default = $value unless $default; I would assume there's a legacy reason for that).

This can be circumvented by:

  • %<nopSEARCH{"Banana" type="query" _DEFAULT="Cherry" type="regex"}%
which is treated as:
  • %<nopSEARCH{"Cherry" type="regex"}%

But as has been noted that's somewhat geeky.

This proposal brings that inconsistency into sharper relief and that's the extra problem now trying to be solved.

It's not strictly an issue of this proposal and I would hope we can implement this proposal and address the _DEFAULT inconsistencies separately.

However, let's discuss the alternatives:

  1. Carefully document _DEFAULT
    • Geeky, but works now just be explicit about this in the Docs
  2. Come up with a synonym for _DEFAULT (i.e. typeglob $alias to $default)
    • If we use a simple alphanumeric
    • The alternative to _DEFAULT needs to be intention revealing
    • My best though so far is base anybody inspired with something better
    • Risk of $alias clashing with existing Macro parameter names
  3. Something in the syntax to be explicit that you are referring to the default
    • You gain control, but this is actually still inconsistent: one syntax for non-default and another for default
    • This is why some people have suggested += as an alternative but:
      • There must be more-revealing/less-confusing suggestions than += to mean add to the default
      • Still need another syntax to say override the default

I note that %SEARCH{search="Banana" is possible and I believe some macros do provide an alternative named parameter for the default. However, they are still not the same beast. %SEARCH{"Cherry" search="Banana"}% will be treated as "Cherry" because the _DEFAULT parameter will trump the search parameter (%SEARCH prerogative).

Now just thinking out load (alternatives in pairs to appending default and overriding default)
%SEARCH{"Banana" type="regex" base+" split"}%
%SEARCH{"Banana" type="regex" base="Cherry"}%
%SEARCH{"Banana" type="regex" _base+" split"}%
%SEARCH{"Banana" type="regex" _base="Cherry"}%
%SEARCH{"Banana" type="regex" @base+" split"}%
%SEARCH{"Banana" type="regex" @base="Cherry"}%
%SEARCH{"Banana" type="regex" @+" split"}%
%SEARCH{"Banana" type="regex" @="Cherry"}%
%SEARCH{"Banana" type="regex" *+" split"}%
%SEARCH{"Banana" type="regex" *="Cherry"}%
%SEARCH{"Banana" type="regex" ++" split"}%
%SEARCH{"Banana" type="regex" =="Cherry Pie"}%

Note that any user writing a %SEARCH macro has started on the path to geekdom and we cannot come up with a geek free option. So while we should not unnecessarily geek it up, let's not over stress this point.

Comments and alternative suggestions welcome.

I note separately the point raised by CrawfordCurrie:
  • %MACRO{param="1" + "2"}%
is no worse than
  • %MACRO{param="1+2"}%

in terms of possible user confusion, does of course need Docs to explicitly state that "1"+"2" gives "12" not "3".

-- JulianLevens - 30 Mar 2014

I'm still no nearer understanding why I need to be able to modify the _DEFAULT parameter, or any other parameter for that matter, out of sequence. Real world examples where it makes something possible, or significantly simpler, than with the status quo plus '+', please

-- CrawfordCurrie - 30 Mar 2014

Julian's example shows how "append to arbitrary parameter" would make things significantly simpler for complex search format and header parameters.

-- MichaelTempest - 30 Mar 2014

You can add footer to the format and header parameters that would gain from this.

These parameters turn up in many places not just search. LdapNgPlugin has these parameters. In addition, it offers clear which might also be useful to extend per column entry (add the $tokens to clear as used in each column).

I even wonder about the possibility of query search parameter extended per column e.g.
  search+"form.name~'*ControlForm' "

  search+"and fields[name='BatchProcessDate' AND value>='%fdat%' AND value<='%tdat%'] "
  header+"|*Control record*"
  format+"| <span style=\"display:none;\">"
        +"$formfield(BatchNumber) $formfield(BatchLocation) "
        +"$percntCALC{$SUBSTRING($topic,13,3)}$percnt $formfield(BatchCageDate) "
        +"!$formfield(BatchMagCode)"
        +"</span>"
        +"<a id=\"$topic\"/>[[$topic]]"
        +"%BR%$percntBATCHDESC{\"$formfield(BatchMagCode)\" num=\"$formfield(BatchNumber)\"}$percnt "

  search+"and fields['BatchActivities' and %acts%] "
  header+"|*Batch Acti- vities*"
  format+"| $formfield(BatchActivities) "

  search+"and %checknotes%"
  header+"|*Batch Notes*"
  format+"| <form name=\"Edit\" action=\"%SCRIPTURLPATH{edit}%/%WEB%/\">"
         +"<input type=\"hidden\" name=\"action\" value=\"form\"/>"
         +"<input type=\"hidden\" name=\"topic\" value=\"$topic\"/>"
         +"<input type=\"hidden\" name=\"redirectto\" value=\"%TOPIC%?%QUERYSTRING%#$topic\"/>"
         +"<input type=\"image\" src=\"%ICONURL{pencil}%\" alt=\"Edit Form\" /> </form> $formfield(BatchNotes)"

That may not really work (probably better to keep the query in one place), but it allows the writer of the MACRO call to logically group related parts.

Apart from being able to see each column together (not scrolling up and down to see the whole picture). It also makes adding new columns into the right place; moving columns into new positions; deleting a whole column and keeping the column details in sync very easy.

For the MACRO expression writer it's a trade off. E.g. you will not see the list of column names together in the header. However, it provides the writer the choices and flexibility to construct the macro for maximum legibility and maintenance.

Using param+"more" to control table output, may be the only use case we have (that I can see), but table output is a significant FW use case and quite compelling in it's own right.

-- JulianLevens - 31 Mar 2014

I like this proposal, though I personally prefer

param+=value

instead of

param+value

because the operation is an assignment at the end of the day. That's properly reflected syntactically by +=. On the other hand param+value looks like an arithmetic expression of two different objects of rather different type with an unknown result going nowhere, sort of wink

The normal

param=value + value

makes totally sense. Not so

param+value

-- MichaelDaum - 31 Mar 2014

MichaelDaum, I very much understand your point of view. In my original proposal I had +=, which feels natural to me as a programmer. I considered .= but the . does get rather lost and it's not even slightly intention revealing to users (except a sub-set that happen to program in perl). From my experience with rexx || is use as an abuttal operator, and then I thought of other opions, but only + or += really made any sense.

The plus is at least suggestive of the intention.

As for the equals well on our Wiki I implemented this with the '=' as optional. Originally I used += but soon preferred only +. One of the benefits of this consistency is that you will convert from one form to the other. A simple table (few columns and/or simple cells) is easy enough with header+A+B+C and format+a+b+c, but at some point when you add more columns (certainly if you're dealing with 14 columns as I am in one case) or add formatting complexity then you'll prefer the other form.

Therefore, you'll go from:
  header
    +"|*Col1*"
    +"|*Col2*"
    +"|*Col3*"
    +"|*"
  format
    +"|*Col1*"
    +"|*Col2*"
    +"|*Col3*"
    +"|*"
To:
  header+"|*Col1*"
  format+"|*Col1*"

  header+"|*Col2*"
  format+"|*Col2*"

  header+"|*Col3*"
  format+"|*Col3*"

  header+"|*"
  format+"|*"
And if you have to add that extra =, it will of course get missed.

Remember I have actually been developing and maintaining a few apps here and actively using this feature. So, in practice I found a simple + better overall for both cases, even though I could have used += here and there I found the = getting in the way, if anything.

Of course I'm the only person actively using this, so others could disagree, but that is my experience.

I'm also concious of the need to document this and the which operators we choose won't impact that too much although I have a hunch a simple + is better here as well. One good thing about this concept is that a few well chosen examples should make this really easy to pick up.

It's also feels very natural very quickly smile

Finally, if we need to have a vote and end-up with two distinct operators, then so be it.

-- JulianLevens - 31 Mar 2014

MichaelTempest, I have finally twigged your suggestion about separating the meaning of + and +=. That is by reserving += for appending to an arbitrary parameter allows += without a name to means append to the unnamed parameter.

It still remains an option of course. However, although I think we need that consistency, it is right now not an actual use case. That is to say that the default parameters of the existing macros are not natural candidates for this treatment (I did suggest search of SEARCH, but even I think that is somewhat contrived).

More importantly, I'm looking for the best choices for the most general (if not only) use case: header, format and footer (and other related params). In this case, for me and in actual experience of using this, just + for both cases works best.

That leaves the issue of how to allow appending to the default to provide that ability or leaving that to a future proposal when and if that use-case emerges. As you have noted there is an existing mechanism to achieve that anyway, _DEFAULT+. Furthermore, to really achieve consistency we need to be able to override the default parameter and again _DEFAULT= is already available for that.

I am not saying that we should settle for this status quo and would propose alternatives (as above & more):
%SEARCH{"Banana" type="regex" base+" split"}%
%SEARCH{"Banana" type="regex" base="Cherry"}%
%SEARCH{"Banana" type="regex" _base+" split"}%
%SEARCH{"Banana" type="regex" _base="Cherry"}%
%SEARCH{"Banana" type="regex" @base+" split"}%
%SEARCH{"Banana" type="regex" @base="Cherry"}%
%SEARCH{"Banana" type="regex" _noname+" split"}%
%SEARCH{"Banana" type="regex" _noname="Cherry"}%
%SEARCH{"Banana" type="regex" _no_name+" split"}%
%SEARCH{"Banana" type="regex" _no_name="Cherry"}%
%SEARCH{"Banana" type="regex" no_name+" split"}%
%SEARCH{"Banana" type="regex" no_name="Cherry"}%
%SEARCH{"Banana" type="regex" @+" split"}%
%SEARCH{"Banana" type="regex" @="Cherry"}%
%SEARCH{"Banana" type="regex" *+" split"}%
%SEARCH{"Banana" type="regex" *="Cherry"}%
%SEARCH{"Banana" type="regex" ++" split"}%
%SEARCH{"Banana" type="regex" =="Cherry Pie"}%

I've added in variants on a theme of 'no name' (I think it was you that talked about the unnamed parameter). However, in the end any synonym for _DEFAULT seems to me to only be marginally better at best. It ideally needs a character in the name guaranteed not to clash with any existing parameter name. In a sense some options like @+ and @= could be seen as using @ as a synonym for _DEFAULT. I considered 0+ and 0= as an alternative (it's parameter 0 isn't it?). Another option would be ^+ and ^= (^ as in at beginning of the line?).

I just thought of 1st+ and 1st=, I cannot guarantee these have not been used as a parameter name but I suspect not. The problem is that the _DEFAULT parameter is not necessarily 1st, but maybe it's more than good enough and intention suggestive at least. If we were paranoid then _1st would do. Just to emphasise 1st+"Banana" _DEFAULT+" split" would result in $param->{_DEFAULT} returned as "Banana split" and maybe $param->{1st} as well. I'll think about that some more.

If no name (pun intended) appeals I am drawn to == & ++, which was my favourite, but do we now have a new 1st choice? big grin

I even think I can write documentation around 1st quite easily (and all examples I've seen place it in the 1st position). And even when it's not first, it's still the 1st unnamed parameter.

1st (or _1st) is growing on me, what about you?

%SEARCH{"Banana" type="regex" _1st+" split"}%
%SEARCH{"Banana" type="regex" _1st="Cherry Pie"}%

Do you have some alternative suggestions? Anybody else?

-- JulianLevens - 31 Mar 2014

Julian,

  header=
    "|*Col1*" +
    "|*Col2*" +
    "|*Col3*" +
    "|*"
  format=
    "|*Col1*" +
    "|*Col2*" +
    "|*Col3*" +
    "|*

and

  header+="|*Col1*"
  format+="|*Col1*"

  header+="|*Col2*"
  format+="|*Col2*"

  header+="|*Col3*"
  format+="|*Col3*"

  header+="|*"
  format+="|*"

aren't too shabby either.

-- MichaelDaum - 01 Apr 2014

While I am convinced by '+' (string concatenation) and the redefinition of '=' as an override, I'm still struggling with real-life use cases for '+=' (prefix) and '=+' (postfix) (having one without the other is just churlish). I just can't see a justification for it in real life and no, I don't accept anything above as a justification. Examples of usage, yes, slightly more convenient, yes, but not a justification.

Please help me out here. How does it enable re-use? How does it help with layout? What can I do with it that I can't do without it?

-- CrawfordCurrie - 01 Apr 2014

As far as I understood '+=' means 'append' or 'increment', not 'prefix'.

I can't see a real-world use case for a 'prepend' in this context.

-- MichaelDaum - 01 Apr 2014

I have come to see '+' as string concatenation in all cases. The question is which parameter are you concatenating the string to? The answer to that is the last name seen. I think this will be a good way to document this to users.

  header
    +"|*Col1*"   /* Last name seen is header so concatenate to that */
    +"|*Col2*"   /* Last name seen is header so concatenate to that */
    +"|*Col3*"   /* Last name seen is header so concatenate to that */
    +"|*"
  format
    +"|*Col1*"   /* Last name seen is format so concatenate to that */
    +"|*Col2*"   /* Last name seen is format so concatenate to that */
    +"|*Col3*"   /* Last name seen is format so concatenate to that */
    +"|*"
To:
  header+"|*Col1*"  /* Last name seen is header so concatenate to that */
  format+"|*Col1*"  /* Last name seen is format so concatenate to that */

  header+"|*Col2*"  /* Last name seen is header so concatenate to that */
  format+"|*Col2*"  /* Last name seen is format so concatenate to that */

  header+"|*Col3*"  /* Last name seen is header so concatenate to that */
  format+"|*Col3*"  /* Last name seen is format so concatenate to that */

  header+"|*"
  format+"|*"

I am not wedded to this, but as I said before in practical use I veered away from +=.

Crawford, this whole feature is arguably about convenience and we can do everything without any of this.

We could limit ourselves to name="a" + "b" + "c" and not allow name+"d" and we would still gain much.

If I understand it then the disagreement is about how valuable it would be to be able to group columns together (via name+"more"). You stated 'slightly more convenient' and others would claim somewhat more than slightly.

As for re-use, I could see something like:
%STARTSECTION{name="column1"}%
   header+"|*Column1*"
   format+"|$percntCALC{$COMPLEX($EXPRESSION($formfield(MyFieldName)))}$percnt"
   footer+"|*Column1*"
%ENDSECTION{name="column2"}%

If you were creating many different views of a dataform, then you could re-use the column definitions quite easily.

Even in poor man's reuse context (aka cut and paste). Would you readily cut and paste this monstrosity:
 format="| [[$topic][$formfield(BatchMagCode) $formfield(BatchCageDate) $formfield(BatchNumber) $formfield(BatchLocation)]]%BR%$percntBATCHDESC{\"$formfield(BatchMagCode)\" num=\"$formfield(BatchNumber)\"}$percnt | $formfield(BatchValue) | $formfield(BatchCurrencies) | $formfield(BatchSizes) | $formfield(BatchCount) | $formfield(BatchToQueue) | $formfield(BatchKilled) | $formfield(BatchIgnored) | <div style='background-color:#$percntCALC{$SET(dat,$formfield(BatchCageDate))$SET(pdat,$formfield(BatchProcessDate))$SET(age,$EVAL($percnt$formfield(BatchMagCode)_days$percnt))$IF($EXACT($percntWORKFLOWSTATE{\"$topic\"}$percnt,CHECKED),FAF0D4,$IF($GET(age)>=0,ff3300,$IF($GET(age)>=-2,ff9933,66cc33)))}$percnt'>$formfield(BatchErrors)&nbsp;</div> | $formfield(BatchOperators) | $formfield(BatchActivities) | <form name=\"Edit\" action=\"%SCRIPTURLPATH{edit}%/%WEB%/\"> <input type=\"hidden\" name=\"action\" value=\"form\"/> <input type=\"hidden\" name=\"topic\" value=\"$topic\"/> <input type=\"hidden\" name=\"redirectto\" value=\"%TOPIC%?%QUERYSTRING{encode="url"}%\"/> <input type=\"image\" src=\"%ICONURL{pencil}%\" alt=\"Edit Form\" /> </form> $formfield(BatchNotes) | $percntWORKFLOWHISTORY{\"$topic\"}$percnt | $percntWORKFLOWTRANSITION{\"$topic\" redirectto=\"%TOPIC%?%QUERYSTRING{encode="url"}%\"}$percnt |"

Of course even breaking this into separate lines would help a lot. Breaking it into separate columns would help if you wanted to copy that column elsewhere.

Now I'm getting creative, how about a %COLUMNS% macro that builds a header and format (optionally footer) for a particular formfield or even all formfields using the DataForm definition. You then get:
%SEARCH("form.name ~ '*.MyDataForm' AND whatever" type="query" nonoise="on"
   %COLUMNS{"MyDataForm" include="*" exclude="FieldName3, FieldName7"}%

   header+"|*Special Case"
   format+"|  $percntCALC{$SUM($LEFT())}$percnt"

   %COLUMNS{"MyDataForm" include="FieldName3, FieldName7"}%
}%
That you cannot do with existing syntax.

In this context then a prepend operator might be useful. My first thoughts then are:
%SEARCH{
   header >> "|Column2" >> "|Column3" << "|Column1"
}%
Note I'm now using '>>' to append and '<<' to prepend, so we've lost '+' in this scheme.

I'd be concerned about misused angle brackets breaking HTML, but it does feel quite intention revealing.

-- JulianLevens - 01 Apr 2014

MichaelDaum, CrawfordCurrie and MichaelTempest discussed this proposal on IRC. In a nutshell:
+= (and =+) let you group parts of parameters together in different ways. That grouping helps.

Michael Daum gave an example showing how (in Crawford's words) "it lets you collect (group) parameter modifications into a single IF which is good":
%SEARCH{....header="| *Col1" format="| Data1" %IF{"switch on second col" then="header+=\"| *Col2\" format+=\" Data2\""}% ...}%
Michael Tempest and Crawford wondered if concatenate (param="part1" +"part2") is needed if you have append (param+="part2").

I am not attached to any particular operators.

-- MichaelTempest - 01 Apr 2014

I can clearly see use cases for both

  1. param="string1" + "string2"; motivation: ease crafting parameters with very long values
  2. param="string1" .... param+="string2"; motivation: allow grouping parameter definitions

Both use cases don't share much in common other than incrementally extend an already defined parameter.

-- MichaelDaum - 01 Apr 2014

For the record: Crawford and I had wondered if we need concatenate, not Crawford and Michael Daum.

-- MichaelTempest - 02 Apr 2014

I've attached a new zip file Attrs.zip that implements a version of this with test cases. The implementation is added to the _friendly and _unfriendly types and test cases accordingly.

Extra non-concat test cases for the _unfriendly type have been added as well.

There is an important table in the code that allows easy changes to the operators and even additional operators should anyone come up with anything suitable.

The table is:
my %Ops = (
#    Op  => [ WithName  , Without Name ]
    '='  => [ \&_assign , undef ] ,
    '+=' => [ \&_append , \&_append ] ,
    '-=' => [ \&_prepend, \&_prepend ] ,
);

prepend is not originally part of this proposal and is here to demonstrate the ease of adding further operators, it can just as easily be removed

The functions are very simple and are passed a scalar ref (current value to assign to) and the value passed:
sub _append { ${$_[0]} .= $_[1]; }
sub _prepend { ${$_[0]} = $_[1] . ${$_[0]}; }
sub _assign { ${$_[0]} = $_[1]; }

An alternative implementation to use + and - as the operators would be:
my %Ops = (
#    Op  => [ WithName  , Without Name ]
    '='  => [ \&_assign , undef ] ,
    '+'  => [ \&_append , \&_append ] ,
    '-'  => [ \&_prepend, \&_prepend ] ,
);

Another alternative implementation would be to use + for concat and += for append, - and -= for precat (sic?) and prepend respectively:
my %Ops = (
#    Op  => [ WithName  , Without Name ]
    '='  => [ \&_assign , undef ] ,

    '+'  => [ undef     , \&_append ] ,
    '+=' => [ \&_append , undef ] ,

    '-'  => [ undef     , \&_prepend ] ,
    '-=' => [ \&_prepend, undef ] ,
);

If only named operators are your preference then you could use:
my %Ops = (
#    Op  => [ WithName  , Without Name ]
    '='  => [ \&_assign , undef ] ,
    '+=' => [ \&_append , undef ] ,
    '-=' => [ \&_prepend, undef ] ,
);

Note that there is no necessity for separate operators for concat and append. They are really one and the same. Indeed this is true in general for any operator. The parser keeps track of the most recent name and if a name is not explicitly given then the most recently given name is used (this can include _DEFAULT).

The provided implemented with += and -= was not my 1st choice which was + and -. However, I then worked on the _friendly option with allows name=Fred or name=123. That in turn means that name=123+56+9 is valid and treated as name=123569 which somewhat breaks POLA. Mind you, it's debatable whether name=123+=56+=9 is significantly better.

We need to vote on different options. Please review carefully, there is also a test topic in the Attrs2.zip attached which can be used to see this in action. This is especially useful to actually edit using different operators and to change column positions in the search etc.

We need to vote on a number of distinct issues, let's cover these two first:

  1. Allow name $op "value" and $op "value"; or just name $op "value"?
  2. Please assume that the 1st vote went with both options. Would you then want to see one shared operator or two distinct operators

Name Both or name-only One or Two
Both
One
Both
Don't mind
Both
Two
Both
Two
Both
Two
Both
Two

One we have agreed on this we will know how many operators we need to vote on. We will then need further rounds of votes on the operators to use.

-- JulianLevens - 21 Jun 2014

My vote would go for
  • param="string1" + "string2"
  • param="string1" .... param+="string2"

but no -= operator. Not sure what the above vote options actually mean.

-- MichaelDaum - 27 Jul 2015

my vote:
  • param="val1" + "val2"
  • param+="val3" + "val4"
maybe misunderstand the main point of the above table... cry

-- JozefMojzis - 27 Jul 2015

To explain the choices:
  • Both would mean: param+="Bananas" +=" and custard"
  • Name-only: param+="Banana" param+=" and custard"

In the 2nd case you must specify the name of the param every time. 'Both' allows the last seen 'name' to be used and allow more concise coding.

Once you decide to have 'both' you then have to decide if you want two distinct operators for each case or use one operator for both cases:
  • Two operators (e.g. += & +)
    • param+="Bananas" +"and custard", therefore param+"Bananas" and +="and custard" would now be an error
    • Pro: need to be more explicit so it may reduce some errors
    • Con: we lose an operator for other uses
    • Con: as you edit for clarity you will create errors as above
  • One operator (e.g. +=)
    • param+="Bananas" +="and custard"
    • Pro: easier syntax which may reduce some (albeit different) errors
    • Pro: you can more easily change/re-factor topics without creating new errors (no need to remember to change the operator)

There is in fact a 3rd option which is two operators which can be used interchangeably, but I think this gives us all the Cons without the Pros.

-- JulianLevens - 28 Jul 2015

Michael, Jozef

Based on your responses I have updated the table. Please check with my explanation above to see if you agree with what I have added on your behalf.

-- JulianLevens - 28 Jul 2015

The consensus is 'Both' & 'Two' from the table above (and IRC here), that is
  • foo+="Bananas" +"and custard" bar="rhubarb" +" crumble" foo+=" and even more custard"

Right now this proposal will focus on append and concat only. (prepend and precat left to fight another day in a new proposal)

Crawford are you ready to remove your concern? Thanks

-- JulianLevens - 28 Jul 2015

 
Topic attachments
I Attachment Action Size Date Who Comment
Attrs.zipzip Attrs.zip manage 7 K 21 Jun 2014 - 19:03 JulianLevens Example implementation and test cases
Attrs2.zipzip Attrs2.zip manage 4 K 29 Mar 2014 - 15:11 JulianLevens Attrs.pm with changes to demonstrate feature
Topic revision: r42 - 21 Nov 2015, GeorgeClark - This page was cached on 18 Nov 2017 - 22:30.

The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License