Feature Proposal: New macros to simplify & formalize adding of JS/CSS to a page
Motivation
VarADDTOZONE was a great leap forward in managing script/css markup in Foswiki, but the job is not yet done.
VarADDTOZONE requires us to write our own
<link
or
<script
markup, which has a number of problems:
- It's too verbose, more room for error
- Wiki-app, skin and plugin authors must all agree on HTML5 syntax (or not)
- There's no possibility to automatically decorate script/CSS URLs with version identifiers, so every upgrade of a Foswiki installation (or one of its plugins) must involve fights between browser caches and
mod_expires
headers
URI versioning is not part of this proposal, but should cater for the possibility
- There's no possibility to automatically concatenate some JS/CSS together
JS/CSS concatenation is not part of this proposal, but should cater for the possibility
- Applying security policies (such as content whitelists) becomes unnecessarily complex and difficult (involves expanding macros and parsing out
<script>
tag attributes very late in the rendering pipeline)
Content whitelisting is not part of this proposal, but hsould cater for the possibility
Description and Documentation
Introduce
%EMBEDINPAGE{"what" from="where"}%
and
%ADDTOPAGE{"what" from="where"}%
macros.
These are almost equivalent to the
ADDJS
and
ADDCSS
from past discussions.
The role of EMBEDIN/ADDTOPAGE is to take a type (just
js
or
css
initially), along with the location of some content (topic section, attachment or URL), and - as per
VarADDTOZONE - optional
requires
id list, optional
id
to identify added content
These new macros take care of decorating the content with
<script
or
<link
markup appropriate for the page DOCTYPE and content type.
Restrictions, when compared with
VarADDTOZONE:
- This proposal stipulates that macros in topic sections will NOT be expanded when added via EMBEDINPAGE.
- VarADDTOZONE's
text
parameter will NOT be supported in ADDTOPAGE or EMBEDINPAGE.
- There's an assumption (not enforced, or should we?) that the arguments to the %EMBEDIN/ADDTOPAGE macros will not be built from inner macros, or if they are, those inner macros won't do silly things such as
from="/Some/Trusted/URL?foo=%URLPARAM{"evil" default="params"}%"
Examples
# in-line JS. %MACROs are NOT expanded.
%EMBEDINPAGE{"js" topic="Web/Topic" section="js"}%
# Creates <script src="/bin/view/Web/Topic?section=js....></script>
%ADDTOPAGE{"js" topic="Web/Topic" section="js"}%
%ADDTOPAGE{"js" attachment="Web/Topic/something.js"}%
%ADDTOPAGE{"js" url="http://example.com/something.js"}%
Impact
Implementation
--
Contributors: PaulHarvey - 10 Feb 2012
Discussion
Past discussions
http://irclogs.foswiki.org/bin/irclogger_log/foswiki?date=2009-11-26,Thu&sel=308#l304
- ADDJS/ADDCSS mentioned
- translating linefeed encoding
- merging/concatenation of multiple cs/js files
MichaelDaum: ArthurClemens, been looking at jammit. frankly I'd prefer what drupal did and add %ADDJS and %ADDCSS with optional merge + compress
- MTempest
- ArthurClemens: Thank you
- MichaelDaum
- having img's inside html is interesting but scarifies bandwidth for latency as you reload the same img in different pages again and again...not sure which amount of inlined imgs makes sense.
(or img's inlined in css)
maybe we can have ADDCSS and ADDJS as part of this BrowserBoosterPlugin: there's all the logic to gather css and js in there already.
http://irclogs.foswiki.org/bin/irclogger_log/foswiki?date=2009-11-25,Wed&sel=280#l276
- linefeed/charset encoding
MichaelDaum: an %ADDCSS{merge="on" will have to take care to unify linefeed encoding as well as charset
same for %ADDJS
http://irclogs.foswiki.org/bin/irclogger_log/foswiki?date=2009-11-04,Wed&sel=551#l547
- CDot working with %MACRO-generated script using
<script src=""...
attribute pointing to a topic section
CDot: damn, the %ADDTOHEAD technique works for most of the JS, but not the bits generated using foswiki macros
- MichaelDaum
- well then it is up to the %MACRO to make a Foswiki::Func::addToHEAD on demand
- CDot
- yeah. Or I can put the dyna-data into hidden divs
which is cleaner anyway
- *
- MTempest has joined #foswiki
- JanD72
- Hm, so trying to disable mod_perl ... :-/
- CDot
- oh, cute. Put the pages that generate the JS into a topic. Then get the topic with ?content-type=text/javascript
<script type="text/javascript" src="http://foswiki.org/System/JsStuff?contenttype=text/javascript;skin=text"> works
- -
- CDot is feeling evil after that
- pharvey_
- CDot: Tricky
For some reason I've never liked the idea of dynamically generated JS code.
Not for the purpose of setting up per-session state stuff anyway.
- MichaelDaum
- CDot, anything requiring perl slows it down. best is to let apache see some static file and redirect the 404 error page to an AttachContentPlugin page.
- CDot
- MichaelDaum: right now I don't care about the speed; I just want it working
- CDot
- actually, I could even add the JS to a hidden div and eval() it
- MichaelDaum
- try jquery.metadata for that
can read json objects from html attributes like the class attribute but also dom nodes
http://irclogs.foswiki.org/bin/irclogger_log/foswiki?date=2009-11-04,Wed&sel=615#l611
- ADDJS/ADDCSS discussion
- Mentions automatic src concatenations to reduce the number of HTTP requests
- Some mention of Drupal's equivalent API, concatenation ideas and difficulties
pharvey_: MichaelDaum: What issues are there to take existing usage of ADDTOHEAD and extend its implementation so that ADDTOHEAD could transparently combine all JS and CSS files? I think: access control (via viewfile). CSS and JS would become one continuous chunk, whereas before it was possible for a JS ADDTOHEAD to requires=some CSS
and.... parsing out the text= to be added so it can automagically understand that you meant to simply load a JS or CSS file, that's messy I suppose
- MichaelDaum
- pharvey_, basically you don't specify the css source directly using ADDTOHEAD, you add some link or script tag.
this in itself is error prone
best would be to just specify the files to be combined and then generate the necessary markup after concatenating them
- pharvey_
- Yes, I mean <script src="foo.js"></script> tags, we would have to parse that out.
- MichaelDaum
- ... which is not the best approach imho
- pharvey_
- no
- pharvey_
- So new ADDTOHEAD params, or a new tag...
ADDTOHEAD{file="foo.js" requires="bar.js"}
- MichaelDaum
- js files should not be added to the head. they should go to the end of the document to improve loading time.
means: we could create an ADDTOBOTTOM
however
I'd prefer to "register" a css or js file to some internal service that then takes care of all of the rest.
- Babar
- or make ADDTOHEAD clever. js => bottom, CSS => top
but ok, sometimes you want your JS to go on top
anyway, that's not the point here
- pharvey_
- register, so not in TML?
- MichaelDaum
- how about two new macros %ADDCSS and %ADDJS
paralleled with an appropriate Func api
- Babar
- yeah, as for IE, %ADDCSS should use @import to avoid the 20 files limitation
EugenMayer: what does Drupal do?
- EugenMayer
- For what? reading log
- MichaelDaum
- Babar, the idea was to concatenate ann added css into one. same for all registered js code
- Babar
- EugenMayer: for adding CSS and JS files from pages
MichaelDaum: with transparent minifier calls?
oh... I remember this discussion...
- MichaelDaum
- and gzip
- EugenMayer
- Babar: they use drupal_add_js / drupal_add_css
- MichaelDaum
- EugenMayer, bingo
- pharvey_
- nifty
- EugenMayer
- similar to the current Foswiki implementation, but a bit more poweful. (mergeing, caching, unqiue loads only)
- MichaelDaum
- there u r
- Babar
- MichaelDaum: iirc, the problem we had was, among others: how to deal with addjs a, addjs b, addjs c, vs a c b. Will it make the same file?
- EugenMayer
- important feature is, that you can use it this way
drupal_add_css('module','eugen','eugen.css')
- MichaelDaum
- Babar, obviously order matters
- Babar
- and also, concatenating into one single file is silly imho because you might loose caching. in fact, you most certainly will
- EugenMayer
- that would be also something interesting for us
- Babar
- MichaelDaum: then you will kill caching
therefore I'm strongly against concatenation
- EugenMayer
- Babar: well it depends what you merge
- pharvey_
- Babar: The idea was that each permutation of load orderings would be concatenated into a unique HEX.js file
- Babar
- yeah, so you will end up loading 20 times the same js file when you already have it
- pharvey_
- a has of the files required would generate the HEX id to load... if not exist... then generate
- EugenMayer
- Babar they e.g. merge theme css, but don't merge plugin css
- pharvey_
- maybe you're right
- Babar
- if you set your expire headers properly, you'll load your JS and CSS files once every other week or so
- EugenMayer
- but this is all in the settings, you can switch merge on and off
The have a internal cache-api anyway
- pharvey_
- Perhaps closer study of drupal is required
- EugenMayer
- We should often do, yes. They seem to have good ideas
- Babar
- and to have manpower to code them :)
http://irclogs.foswiki.org/bin/irclogger_log/foswiki?date=2010-07-24,Sat&sel=212#l208
-
ADDFILE{"/pub/foo.js"}
idea
pharvey: Babar: started hacking a StaticFilesPlugin, which used templates for URI version decoration
TMPL:DEF{"StaticFilesPlugin::Type::JS"}%<script type="text/javascript" src="%FILEPATH%%FILEVER%"></script>%TMPL:END% - or something like that
used when you do an ADDJS, or as I started to do it, ADDFILE{"/pub/Something.js"} (magic file extension detection)
this way if you want HTML5 output you can redefine the DEF to the minimal HTML5 boilerpalte
--
PaulHarvey - 11 Feb 2012
Wow, these are
very old irc citations.
Judging from their name I don't really get the difference between those two new macros EMBEDINPAGE and ADDTOPAGE.
Those macros don't really differ much from ADDTOZONE other than giving the content to be added to the page a specific type that
- (a) can be leveraged for further optimization, e.g. by merging and compressing all css into one file and
- (b) ease the way those bits are added to the page instead of having to craft
script
or link
tags.
While I fully agree with the basic intent, I'd rather not add YAPONAM (yet another poorly named macro) to the TML language.
It seems much more straight forward to add a
type
attribute to %ADDTOZONE directly.
Further note, that we also have a high level mechanism build on top of %ADDTOZONE, called %JQREQUIRE.
From a user's perspective, %JQREQUIRE is a lot easier to use: you just specify what you want and the rest is done automatically: it adds all css and/or js automatically; it automatically loads any additional requirement that the feature the user asked for is depending on. So it totally hides any complexity.
From this point it seems only consequent to leverage %JQREQUIRE from its use case within the jquery integration to foswiki to a more broader scope, by renaming/superseeding it with a %REQUIRE macro that basically does the same:
- (a) hide any complexity from the user and
- (b) open up the alley to further optimizations of the material added to a page.
In that sense a %REQUIRE macro would fulfill the basic intention of the above proposed %EMBEDINPAGE macros but would be much more
in line with what we already have
and go even beyond in terms of simplification.
--
MichaelDaum - 13 Feb 2012
JQREQUIRE has been unacceptable because it forces something into perl-land that doesn't belong there.
The wiki-pattern should be
%INCLUDE{"System.JQueryCycle"}%
rather than
%JQREQUIRE{"cycle"}%
.
%ADDTOZONE%
seemed to have a specific mission: adding markup to a zone. I'm not sure that overloading it too much is a great idea. Apart from
type
arg, it also needs to disallow
text
arg for
script
zone for security reasons, and also needs content locations other than
topic/section
.
--
PaulHarvey - 13 Feb 2012
I support Michael's idea of a
type
parameter on ADDTOZONE over adding new macros. But I think that content (or content-source) whitelisting is critical, and
should be part of this proposal, otherwise something is going to get missed.
--
CrawfordCurrie - 13 Feb 2012
Ok. I'll re-work the proposal at a later date.
--
PaulHarvey - 13 Feb 2012
I am parking this proposal since nobody seems to be working on carrying it forward. Please feel free to set this back to e.g.
UnderInvestigation if you want to make this proposal happen.
--
JanKrueger - 22 Feb 2016