Item8924: Support export of arbitrary Foswiki preferences to Javascript

http://foswiki.org/Development/UsingJQueryWithFoswiki#Using_jQuery_and_AJAX has a list of js variables - scriptSuffix is not one

also, the example AJAX should really use these vars, rather than a pre-cooked viewurl..

-- SvenDowideit - 19 Apr 2010

Poking TinyMCEPlugin today, something it has that I haven't seen in JQueryPlugin is the ability to expand out common vars browser-side. Eg. %PUBURL%/Foo/Bar

-- PaulHarvey - 20 Apr 2010

... and probably a getUrl()

tbh, both these functionalities need to be in the foswiki namespace - and if possible, in plain jane javascript

-- SvenDowideit - 20 Apr 2010

I seem to recall noting this once before, so there may be a duplicate task somewhere. I'm dialing it up to Urgent, because it's pretty fundamental to writing ajaxy stuff.

The current list is:

Name	Content
foswiki.web	%WEB%
foswiki.topic	%TOPIC%"
foswiki.scriptUrl	%SCRIPTURL%
foswiki.scriptUrlPath	%SCRIPTURLPATH%
foswiki.pubUrl	%PUBURL%
foswiki.pubUrlPath	%PUBURLPATH%
foswiki.systemWebName	%SYSTEMWEB%
foswiki.usersWebName	%USERSWEB%
foswiki.wikiName	%WIKINAME%
foswiki.loginName	%USERNAME%
foswiki.wikiUserName	%WIKIUSERNAME%
foswiki.serverTime	%SERVERTIME%
foswiki.ImagePluginEnabled	%IF{"context ImagePluginEnabled" then="true" else="false"}%
foswiki.MathModePluginEnabled	%IF{"context MathModePluginEnabled" then="true" else="false"}%

These variables are (I assume) populated from META tags.

Personally I dislike this approach quite a lot; it's inflexible and difficult to extend. I'd far rather we deprecated these functions and instead gave some control over what gets exported to JS, for example,

• Set EXPORT_MACROS = WEB,TOPIC,SCRIPTURL,SCRIPTURLPATH,PUBURL,PUBURLPATH,....

and on the client side:

foswiki.getMacro = function(name) { ... }

foswiki.getMacro('SCRIPTURLPATH')+'/view'+foswiki.getMacro('SCRIPTSUFFIX')+'/'+foswiki.getMacro('WEB')+'/'+foswiki.getMacro('TOPIC')

Not sure what to do with the context vars; I assume from the fact these were hacked in by someone that they are used in one or more jquery plugins.

-- CrawfordCurrie - 20 Apr 2010

Is it just me or would it be preferable to just have a single <script> tag processing a single lump of JSON somewhere? There would be less iterating-and-checking-and-eval'ing-over-meta-tags to do, not to mention a slight decrease in page size - if, say, we were to just dump dozens, hundreds of vars out to the browser ( %ADDTOZONE{"JSON" text=", \"foo\": \"bar\""}% perhaps?)

-- PaulHarvey - 20 Apr 2010

That works for me. Mozilla has "issues" iterating over META tags which means a slow algorithm has to be used to recover them, so a JS block would no doubt be faster in the client.

-- CrawfordCurrie - 20 Apr 2010

damn, I just needed ICONTOPIC too. - though I don't want the web - just the topic portion..

-- SvenDowideit - 21 Apr 2010

Which clearly shows that these should not be js properties , but accessor methods, which can, if need be, do an AJAX call to grab a needed value. Or perhaps, a REQUIREVALUE{"ANYTML" name="anytml"} to bring that value/accseeor into the namespace at render time?

-- SvenDowideit - 20 May 2010

Sven, I had a similar thought {REQUIREVALUE}. Probably, I would want the name= to be optional (as ugly as uppercase is in JS).

Problem with ajax accessor methods is - need to document difference between dynamic eval and the eval you get when explicitly REQUIREd (Eg. you might want the CALC{"$RAND()"}% value that was obtained on the initial view, not the new one you get when evaluating it again via rest)

RenderPlugin could do the job at the Foswiki side of things for ajax.

This obviously needs to be a feature proposal for 2.0; a "standard" set of vars should be established, and then foswiki app writers can REQUIREVALUE additional.

For now, we should just add the IconUrl and friends as more <meta/> tags, IMHO

-- PaulHarvey - 20 May 2010

Well, I did it "properly" but to do so I had to extend the FOREACH macro.

-- CrawfordCurrie - 23 May 2010

Nice!

-- PaulHarvey - 23 May 2010

Actually, this needs to change. By removing the <meta> tag functionality, and replacing it with a global =%,nop>PREFS2JS% variable, we no longer have an easy (additive) way to export values ad-hoc from perl to JS.

Either plugin authors are going to be stomping on each other's %PREFS2JS% or we will have users having to make manual tweaks in SitePreferences to make Contrib/Plugin extensions happy.

-- PaulHarvey - 27 May 2010

Ah, buggerit, you are right.

-- CrawfordCurrie - 27 May 2010

I've fixed this by porting getPrefrences() to jquery.foswiki and reworked the code to generate a set of metas again instead of generating inline javascript in the html head.

A couple of plugins broke as they relied on the foswiki object to auto-populate it with properties generated from any meta tag named foswiki.xxx automatically. For now, I've added the most common ones again but documented these as beeing deprecated and reworked all other plugins to use foswiki.getPreference() instead.

I've also reestablished the convention to use the foswiki. prefix in meta tag names again, as that's a critical namespace to prevent keys to overlap with those of meta tags generated by other extensions/libraries/skins.

-- MichaelDaum - 28 May 2010

Actually, Paul is not right. The following code in an initPlugin:

    my $pref = Foswiki::Func::getPreferencesValue('EXPORTEDPREFERENCES')||'';
    my @list = split(/[,\s]+/, $pref);
    unless (grep { /^MYPREF$/ } @list) {
        push(@list, 'MYPREF');
    }
    Foswiki::Func::setPreferencesValue('EXPORTEDPREFERENCES', join(',', @list));

works nicely. (I'm going to rename PREFS2JS to EXPORTEDPREFERENCES).

-- CrawfordCurrie - 28 May 2010

Documented that code in DefaultPreferences and changed the preference name.

Changed the headline of this report; was "js foswiki namespace should have a 'scriptSuffix'" -- CrawfordCurrie - 28 May 2010

---

Wiki applications need to be able export arbitrary values too. We still have a loss of functionality and I don't want all my ajax/JQuery stuff to have to use perl code just for this.

-- PaulHarvey - 29 May 2010

The more I think about it, the more I think we need to restore the <meta> functionality.

We killed a few sites that we know of with the ZonePlugin stuff, and I'd hate to continue to give the impression of instability. That is, after all, the whole reason why we added JQueryPlugin to core.

I can't be the only one out there that's built wiki applications using the <meta> tags to export values.

I am not against removing the functionality but I think it's too late for 1.1 and should go through the proposal process.

-- PaulHarvey - 29 May 2010

(1) Presumably your wiki application uses template modifications to create the <meta tag with the parameter value. So you could always modify the template to:

<script> foswiki.preferences['THEPREF']='%THEPREF%'; </script>

(2) The foswiki.getPreference code still supports <meta for foswiki.getPreference, and foswiki.getMetaTag is still there, so apps that use that methodology should still work

-- CrawfordCurrie - 29 May 2010

An exported preference with " in the value results in javascript that does not compile, which breaks the preferences export. I tried to fix this with ENCODE, but a newline in the preference was strangely encoded (and preferences can contain newlines if their value comes from a plugin).

This is the kind of thing I saw - note that the first newline was converted to <br /> before the whole was encoded:

%ENCODE{mode: "textareas",
editor_selector : "foswikiWysiwygEdit",
save_on_tinymce_forms:true}%

renders as

mode:%20%22textareas%22%2c%3cbr%20/%3eeditor_selector%20:%20%22foswikiWysiwygEdit%22%2c%0asave_on_tinymce_forms:true

The obvious thing is to try a different encoding mode, but the quotes in the preference value also mess up the macro attributes parsing. The attributes parsing results in this preference being exported as an empty string:

• Set NOTNULL=a="b"

I think what is needed is one of the following

• the ability to tell ENCODE to expand its default parameter prior to encoding
• the ability to tell ENCODE to expand the value of a particular preference
• the ability to tell EXPAND to encode its output

I would prefer the first, so that I could do something like this where the preferences are exported: %ENCODE{type="quotes" expand="on" "$percntMYPREFERENCE$percnt"}%

I've added Crawford to the WaitingFor list, as I said to him I would look at this, but I have not found a satisfactory solution.

-- MichaelTempest - 30 May 2010

Exporting prefs via

<script> foswiki.preferences['THEPREF']='%THEPREF%'; </script>

is NOT recommended as it breaks encapsulation of the foswiki js object and might potentially conflict with other fragments of the same pattern. Instead, use

<meta name="foswiki.foobar" content="..." />

to set js preferences and

foswiki.getPreference("foobar")

to get it. The only difference to jQuery plugin before is that you will have to rework any occurence of a direct access to the property foswiki.foo with the above getter.

I am very sorry for this late interface change. Yes, there is loss of functionality with this change, as settings in meta aren't translated into a json sub object of foswiki automatically anymore. Hope is that most didn't use this feature for now.

Further note, that I did not release the latest changes in JQueryPlugin on trunk as there are other plugins that need to be fixed accordingly. It is strongly recommended not to use JQueryPlugin/trunk in a production release. Please, wait until it has been released as you most probably will run into other plugins erroring out.

Again, I am very sorry for this change. On the plus side we now have the ability to

• extend the set of meta preferences; this was hard-coded to a selected set of properties
• fetch preference variables asynchronously using ajax

All this is hidden behind the new getter foswiki.getPreference().

-- MichaelDaum - 31 May 2010

Note that where Michael says "settings in meta aren't translated into a json sub object of foswiki automatically anymore", it is true that META isn't automatically loaded exhaustively into the foswiki.preferences sub-object on startup. Instead it is loaded on demand - when you refer to a preference via foswiki.getPreference. I'm not aware of any loss of functionality - perhaps someone can explain?

I have to agree that explicit modification of the foswiki.preferences object is a bad idea. I was just illustrating the possibility.

I'm not desperately enamoured of any of the proposed encoding "fixes"

-- CrawfordCurrie - 31 May 2010

ENCODE type url would have been the right choice. Alas it is broken. This code fragment from ENCODE.pm

    elsif ( $type =~ /^url$/i ) {
        $text =~ s/\r*\n\r*/<br \/>/;    # Legacy.
        return urlEncode($text);
    }

is the reason why. Can somebody elaborate the "Legacy" marker?

-- MichaelDaum - 31 May 2010

Crawford, the loss of functionality is because you deleted the following code from jquery.foswiki in changeset 7513.

51         /******************************************************** 
52          * populate foswiki obj with meta data 
53          */ 
54         $(function() { 
55           $("head meta[name^='foswiki.']").each(function() { 
56             var val = this.content, keys; 
57             if (val == "false") { 
58               val = false; // convert to Boolean 
59             } else if (val == "true") { 
60               val = true; // convert to Boolean 
61             } else if (val.match(/^\{.*\}$/)) { 
62               val = eval("("+val+")"); // convert to object 
63             } else if (val.match(/^function/)) { 
64               val = eval("("+val+")"); // convert to Function 
65             } 
66             keys = this.name.split(/\./); 
67             keys.shift(); // take out the first one 
68             foswiki.createMember(foswiki, keys, val); 
69           }); 
70         }); 

It converted the content of a meta tag to either Boolean, a json object or a function recursively. That's n/a anymore.

-- MichaelDaum - 31 May 2010

OK - I thought it was just related to populating the foswiki. static variables, as suggested by the documentation. I can see now it does a lot more than that; I presume you intend to reinstate it?

-- CrawfordCurrie - 01 Jun 2010

The EXPORTEDPREFERENCES expansion code in the templates does not handle single-quotes in preference values. For example, an exported preference like this causes havoc if ASSERTs are enabled:

   * Set BREAKAGE=' /> </html>

-- MichaelTempest - 07 Jun 2010

yes, encoding is necessary. Added it.

-- CrawfordCurrie - 08 Jun 2010

I am still frustrated that EXPORTEDPREFERENCES can only realistically be modified by perl plugins, but I suppose it will do for 1.1.

I would have very much rather seen a <script>var foswiki.preferences = {"json" : {"lots" : "of", "josn" : "dude"} }</script> type implementation where a string of JSON was built incrementally with a macro that was a wrapper to VarADDTOZONE somehow, but anyway.

Maybe if we had

   * Set EXPORTEDPREFERENCES = %TMPL:P{"meta:preferences"}%

Then one could build template covers that built on "meta:preferences" with the recursive templates feature:

%TMPL:DEF{"meta:preferences"}%%TMPL:P{"meta:preferences"}%, FOO%TMPL:END%

Removing myself from WaitingFor.

-- PaulHarvey - 09 Jun 2010

Crawford, thanks for digging up the distro:336175d219c7 on the <br />-ENCODE issue (and for fixing it ). Removing myself from WaitingFor.

-- MichaelTempest - 09 Jun 2010

Paul, once you are in template land, you don't have to go via EXPORTEDPREFERENCES. Just add your own meta tags to the head using something like

%TMPL:DEF{"aftertext_or_whatever"}%
...
%ADDTOZONE{"head" tag="MYSKIN::META" text="<meta name='foswiki.MySkin.fooFlag' content='barValue' />"}%
...
%TMPL:END%

and make use of it in javascript land with

...
var fooFlag = foswiki.getPreference('MySkin.fooFlag');
...

-- MichaelDaum - 09 Jun 2010

I think this one can be closed now. Normally, things like that require a feature proposal.

-- MichaelDaum - 14 Jun 2010