Item14437: Macro expansion runs away, reprocessing some macros many times.

pencil
Priority: Normal
Current State: Proposal Required
Released In: n/a
Target Release:
Applies To: Extension
Component: FoswikiRender, Performance
Branches:
Reported By: GeorgeClark
Waiting For:
Last Change By: GeorgeClark
Found this while trying to debug the "Friendly" attribute parser.

When a "common tags" macro is encountered, expansion "fails", and it's placed back onto the stack and re-processed with much longer argument strings. A simple example is a %TABLE{sort="off"}%, followed by a table. Below is an abbreviated debug trace. It appeared to go through this parsing process 15 times.

QUEUE:'TABLE{sort="off"}'
      '%'
...
CONSIDER %TABLE{sort="off"}
ATTRS Called with sort="off"
=============================EXPAND TABLE FAILED
...
CONSIDER %TABLE{sort="off"}%
     | *Parameter* | *Description* | *Default* |
     | ="string"= | String to encode | "" (empty string) |
     | =type= | Use a predefined encoding (see below). | Default is 'url'. Parameter =type= not be used if =old= or =new= are given. |
     | =old= | Comma-separated list of tokens to replace. Tokens are normally single characters, but can also be sequences of characters. The standard [[format tokens]] may be used in this list. Each token must be unique - you cannot list the same token twice. | May not be used with =type=; required if =new= is used |
     | =new= | comma-separated list of replacement tokens. The elements in this list match 1:1 with the elements in the =old= list. Again, the standard [[format tokens]] may be used. An empty element in the =new= list will result in the corresponding token in the =old= list being deleted from the string. If the =new= list is shorter than the =old= list it will be extended to the same length using the empty element. Tokens do not have to be unique. <div class="foswikiHelp">
...
CONSIDER %TABLE{sort="off"}%
     | *Parameter* | *Description* | *Default* |
     | ="string"= | String to encode | "" (empty string) |
     | =type= | Use a predefined encoding (see below). | Default is 'url'. Parameter =type= not be used if =old= or =new= are given. |
     | =old= | Comma-separated list of tokens to replace. Tokens are normally single characters, but can also be sequences of characters. The standard [[format tokens]] may be used in this list. Each token must be unique - you cannot list the same token twice. | May not be used with =type=; required if =new= is used |
     | =new= | comma-separated list of replacement tokens. The elements in this list match 1:1 with the elements in the =old= list. Again, the standard [[format tokens]] may be used. An empty element in the =new= list will result in the corresponding token in the =old= list being deleted from the string. If the =new= list is shorter than the =old= list it will be extended to the same length using the empty element. Tokens do not have to be unique. <div class="foswikiHelp"><img src="/pub/System/DocumentGraphics/warning.png" alt="ALERT!" title="ALERT!" width="16" height="16" /> 
...
ATTRS Called with sort="off"}%
     | *Parameter* | *Description* | *Default* |
     | ="string"= | String to encode | "" (empty string) |
     | =type= | Use a predefined encoding (see below). | Default is 'url'. Parameter =type= not be used if =old= or =new= are given. |
     | =old= | Comma-separated list of tokens to replace. Tokens are normally single characters, but can also be sequences of characters. The standard [[format tokens]] may be used in this list. Each token must be unique - you cannot list the same token twice. | May not be used with =type=; required if =new= is used |
     | =new= | comma-separated list of replacement tokens. The elements in this list match 1:1 with the elements in the =old= list. Again, the standard [[format tokens]] may be used. An empty element in the =new= list will result in the corresponding token in the =old= list being deleted from the string. If the =new= list is shorter than the =old= list it will be extended to the same length using the empty element. Tokens do not have to be unique. <div class="foswikiHelp"><img src="/pub/System/DocumentGraphics/warning.png" alt="ALERT!" title="ALERT!" width="16" height="16" /> When using =old= and =new=, be aware that the results of applying earlier tokens are not processed again using later tokens. (see examples below)</div> | May not be used with =type=; required if =old= is used |
If =ENCODE= is called with no optional parameters (e.g. =%<nop>ENCODE{"string"
=============================EXPAND TABLE FAILED

-- GeorgeClark - 15 Jul 2017

I wonder if there is a way to improve this. By allowing extensions to "register" tags that will be expanded in a commonTagsHandler as a "hint" to the macro processing might improve things. Then when a macro is encountered, it could be temporarily protected, and restored at the end of the parsing. On the SpreadSheetPlugin page, the CALC macro was processed 200 times with increasing length arguments.

ATTRS CALLED length:23   $PROPERSPACE(WikiGuest)...) 
ATTRS CALLED length:142   $PROPERSPACE(WikiGuest)}%
ATTRS CALLED length:373   $PROPERSPACE(WikiGuest)}%
ATTRS CALLED length:586   $PROPERSPACE(WikiGuest)}%
...
ATTRS CALLED length:21259   $PROPERSPACE(WikiGuest)}%

I'm not sure how to address it, but I notices another issue. Macros escaped with exclamation mark are removed from consideration by encoding the %. However macros escaped with <nop> are not actually escaped and get considered repeatedly.

-- GeorgeClark - 16 Jul 2017

Changing this to Proposal needed. This would be a pretty big change.

-- GeorgeClark - 12 Dec 2017
 

ItemTemplate edit

Summary Macro expansion runs away, reprocessing some macros many times.
ReportedBy GeorgeClark
Codebase 2.1.4
SVN Range
AppliesTo Extension
Component FoswikiRender, Performance
Priority Normal
CurrentState Proposal Required
WaitingFor
Checkins
ReleasedIn n/a
CheckinsOnBranches
trunkCheckins
masterCheckins
ItemBranchCheckins
Release02x01Checkins
Release02x00Checkins
Release01x01Checkins
Topic revision: r3 - 12 Dec 2017, GeorgeClark
The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License    Legal Imprint    Privacy Policy