Feature Proposal: New macros to simplify & formalize adding of JS/CSS to a page

Motivation

VarADDTOZONE was a great leap forward in managing script/css markup in Foswiki, but the job is not yet done. VarADDTOZONE requires us to write our own <link or <script markup, which has a number of problems:
  • It's too verbose, more room for error
  • Wiki-app, skin and plugin authors must all agree on HTML5 syntax (or not)
  • There's no possibility to automatically decorate script/CSS URLs with version identifiers, so every upgrade of a Foswiki installation (or one of its plugins) must involve fights between browser caches and mod_expires headers
    IDEA! URI versioning is not part of this proposal, but should cater for the possibility
  • There's no possibility to automatically concatenate some JS/CSS together
    IDEA! JS/CSS concatenation is not part of this proposal, but should cater for the possibility
  • Applying security policies (such as content whitelists) becomes unnecessarily complex and difficult (involves expanding macros and parsing out <script> tag attributes very late in the rendering pipeline)
    IDEA! Content whitelisting is not part of this proposal, but hsould cater for the possibility

Description and Documentation

Introduce %EMBEDINPAGE{"what" from="where"}% and %ADDTOPAGE{"what" from="where"}% macros.

These are almost equivalent to the ADDJS and ADDCSS from past discussions.

The role of EMBEDIN/ADDTOPAGE is to take a type (just js or css initially), along with the location of some content (topic section, attachment or URL), and - as per VarADDTOZONE - optional requires id list, optional id to identify added content

These new macros take care of decorating the content with <script or <link markup appropriate for the page DOCTYPE and content type.

Restrictions, when compared with VarADDTOZONE:
  • This proposal stipulates that macros in topic sections will NOT be expanded when added via EMBEDINPAGE.
  • VarADDTOZONE's text parameter will NOT be supported in ADDTOPAGE or EMBEDINPAGE.
  • There's an assumption (not enforced, or should we?) that the arguments to the %EMBEDIN/ADDTOPAGE macros will not be built from inner macros, or if they are, those inner macros won't do silly things such as from="/Some/Trusted/URL?foo=%URLPARAM{"evil" default="params"}%"

Examples

# in-line JS. %MACROs are NOT expanded.
%EMBEDINPAGE{"js" topic="Web/Topic" section="js"}%
# Creates <script src="/bin/view/Web/Topic?section=js....></script>
%ADDTOPAGE{"js" topic="Web/Topic" section="js"}%
%ADDTOPAGE{"js" attachment="Web/Topic/something.js"}%
%ADDTOPAGE{"js" url="http://example.com/something.js"}%

Impact

%WHATDOESITAFFECT%
edit

Implementation

-- Contributors: PaulHarvey - 10 Feb 2012

Discussion

Past discussions

http://irclogs.foswiki.org/bin/irclogger_log/foswiki?date=2009-11-26,Thu&sel=308#l304
  • ADDJS/ADDCSS mentioned
  • translating linefeed encoding
  • merging/concatenation of multiple cs/js files
MichaelDaum: ArthurClemens, been looking at jammit. frankly I'd prefer what drupal did and add %ADDJS and %ADDCSS with optional merge + compress
MTempest
ArthurClemens: Thank you
MichaelDaum
having img's inside html is interesting but scarifies bandwidth for latency as you reload the same img in different pages again and again...not sure which amount of inlined imgs makes sense.
(or img's inlined in css)
maybe we can have ADDCSS and ADDJS as part of this BrowserBoosterPlugin: there's all the logic to gather css and js in there already.
http://irclogs.foswiki.org/bin/irclogger_log/foswiki?date=2009-11-25,Wed&sel=280#l276
  • linefeed/charset encoding
MichaelDaum: an %ADDCSS{merge="on" will have to take care to unify linefeed encoding as well as charset
same for %ADDJS
http://irclogs.foswiki.org/bin/irclogger_log/foswiki?date=2009-11-04,Wed&sel=551#l547
  • CDot working with %MACRO-generated script using <script src=""... attribute pointing to a topic section
CDot: damn, the %ADDTOHEAD technique works for most of the JS, but not the bits generated using foswiki macros frown, sad smile
MichaelDaum
well then it is up to the %MACRO to make a Foswiki::Func::addToHEAD on demand
CDot
yeah. Or I can put the dyna-data into hidden divs
which is cleaner anyway
*
MTempest has joined #foswiki
JanD72
Hm, so trying to disable mod_perl ... :-/
CDot
oh, cute. Put the pages that generate the JS into a topic. Then get the topic with ?content-type=text/javascript
<script type="text/javascript" src="http://foswiki.org/System/JsStuff?contenttype=text/javascript;skin=text"> works smile
-
CDot is feeling evil after that
pharvey_
CDot: Tricky wink
For some reason I've never liked the idea of dynamically generated JS code.
Not for the purpose of setting up per-session state stuff anyway.
MichaelDaum
CDot, anything requiring perl slows it down. best is to let apache see some static file and redirect the 404 error page to an AttachContentPlugin page.
CDot
MichaelDaum: right now I don't care about the speed; I just want it working
CDot
actually, I could even add the JS to a hidden div and eval() it
MichaelDaum
try jquery.metadata for that
can read json objects from html attributes like the class attribute but also dom nodes
http://irclogs.foswiki.org/bin/irclogger_log/foswiki?date=2009-11-04,Wed&sel=615#l611
  • ADDJS/ADDCSS discussion
  • Mentions automatic src concatenations to reduce the number of HTTP requests
  • Some mention of Drupal's equivalent API, concatenation ideas and difficulties
pharvey_: MichaelDaum: What issues are there to take existing usage of ADDTOHEAD and extend its implementation so that ADDTOHEAD could transparently combine all JS and CSS files? I think: access control (via viewfile). CSS and JS would become one continuous chunk, whereas before it was possible for a JS ADDTOHEAD to requires=some CSS
and.... parsing out the text= to be added so it can automagically understand that you meant to simply load a JS or CSS file, that's messy I suppose
MichaelDaum
pharvey_, basically you don't specify the css source directly using ADDTOHEAD, you add some link or script tag.
this in itself is error prone
best would be to just specify the files to be combined and then generate the necessary markup after concatenating them
pharvey_
Yes, I mean <script src="foo.js"></script> tags, we would have to parse that out.
MichaelDaum
... which is not the best approach imho
pharvey_
no
pharvey_
So new ADDTOHEAD params, or a new tag... smile
ADDTOHEAD{file="foo.js" requires="bar.js"}
MichaelDaum
js files should not be added to the head. they should go to the end of the document to improve loading time.
means: we could create an ADDTOBOTTOM
however
I'd prefer to "register" a css or js file to some internal service that then takes care of all of the rest.
Babar
or make ADDTOHEAD clever. js => bottom, CSS => top
but ok, sometimes you want your JS to go on top
anyway, that's not the point here
pharvey_
register, so not in TML?
MichaelDaum
how about two new macros %ADDCSS and %ADDJS
paralleled with an appropriate Func api
Babar
yeah, as for IE, %ADDCSS should use @import to avoid the 20 files limitation
EugenMayer: what does Drupal do? smile
EugenMayer
For what? reading log
MichaelDaum
Babar, the idea was to concatenate ann added css into one. same for all registered js code
Babar
EugenMayer: for adding CSS and JS files from pages
MichaelDaum: with transparent minifier calls?
oh... I remember this discussion...
MichaelDaum
and gzip
EugenMayer
Babar: they use drupal_add_js / drupal_add_css
MichaelDaum
EugenMayer, bingo
pharvey_
nifty
EugenMayer
similar to the current Foswiki implementation, but a bit more poweful. (mergeing, caching, unqiue loads only)
MichaelDaum
there u r
Babar
MichaelDaum: iirc, the problem we had was, among others: how to deal with addjs a, addjs b, addjs c, vs a c b. Will it make the same file?
EugenMayer
important feature is, that you can use it this way
drupal_add_css('module','eugen','eugen.css')
MichaelDaum
Babar, obviously order matters
Babar
and also, concatenating into one single file is silly imho because you might loose caching. in fact, you most certainly will
EugenMayer
that would be also something interesting for us
Babar
MichaelDaum: then you will kill caching
therefore I'm strongly against concatenation smile
EugenMayer
Babar: well it depends what you merge
pharvey_
Babar: The idea was that each permutation of load orderings would be concatenated into a unique HEX.js file
Babar
yeah, so you will end up loading 20 times the same js file when you already have it
pharvey_
a has of the files required would generate the HEX id to load... if not exist... then generate
EugenMayer
Babar they e.g. merge theme css, but don't merge plugin css
pharvey_
maybe you're right
Babar
if you set your expire headers properly, you'll load your JS and CSS files once every other week or so
EugenMayer
but this is all in the settings, you can switch merge on and off
The have a internal cache-api anyway
pharvey_
Perhaps closer study of drupal is required
EugenMayer
We should often do, yes. They seem to have good ideas
Babar
and to have manpower to code them :)
http://irclogs.foswiki.org/bin/irclogger_log/foswiki?date=2010-07-24,Sat&sel=212#l208
  • ADDFILE{"/pub/foo.js"} idea
pharvey: Babar: started hacking a StaticFilesPlugin, which used templates for URI version decoration
TMPL:DEF{"StaticFilesPlugin::Type::JS"}%<script type="text/javascript" src="%FILEPATH%%FILEVER%"></script>%TMPL:END% - or something like that
used when you do an ADDJS, or as I started to do it, ADDFILE{"/pub/Something.js"} (magic file extension detection)
this way if you want HTML5 output you can redefine the DEF to the minimal HTML5 boilerpalte

-- PaulHarvey - 11 Feb 2012

Wow, these are very old irc citations.

Judging from their name I don't really get the difference between those two new macros EMBEDINPAGE and ADDTOPAGE.

Those macros don't really differ much from ADDTOZONE other than giving the content to be added to the page a specific type that

  • (a) can be leveraged for further optimization, e.g. by merging and compressing all css into one file and
  • (b) ease the way those bits are added to the page instead of having to craft script or link tags.

While I fully agree with the basic intent, I'd rather not add YAPONAM (yet another poorly named macro) to the TML language.

It seems much more straight forward to add a type attribute to %ADDTOZONE directly.

Further note, that we also have a high level mechanism build on top of %ADDTOZONE, called %JQREQUIRE.

From a user's perspective, %JQREQUIRE is a lot easier to use: you just specify what you want and the rest is done automatically: it adds all css and/or js automatically; it automatically loads any additional requirement that the feature the user asked for is depending on. So it totally hides any complexity.

From this point it seems only consequent to leverage %JQREQUIRE from its use case within the jquery integration to foswiki to a more broader scope, by renaming/superseeding it with a %REQUIRE macro that basically does the same:

  • (a) hide any complexity from the user and
  • (b) open up the alley to further optimizations of the material added to a page.

In that sense a %REQUIRE macro would fulfill the basic intention of the above proposed %EMBEDINPAGE macros but would be much more in line with what we already have and go even beyond in terms of simplification.

-- MichaelDaum - 13 Feb 2012

JQREQUIRE has been unacceptable because it forces something into perl-land that doesn't belong there.

The wiki-pattern should be %INCLUDE{"System.JQueryCycle"}% rather than %JQREQUIRE{"cycle"}%.

%ADDTOZONE% seemed to have a specific mission: adding markup to a zone. I'm not sure that overloading it too much is a great idea. Apart from type arg, it also needs to disallow text arg for script zone for security reasons, and also needs content locations other than topic/section.

-- PaulHarvey - 13 Feb 2012

I support Michael's idea of a type parameter on ADDTOZONE over adding new macros. But I think that content (or content-source) whitelisting is critical, and should be part of this proposal, otherwise something is going to get missed.

-- CrawfordCurrie - 13 Feb 2012

Ok. I'll re-work the proposal at a later date.

-- PaulHarvey - 13 Feb 2012

I am parking this proposal since nobody seems to be working on carrying it forward. Please feel free to set this back to e.g. UnderInvestigation if you want to make this proposal happen.

-- JanKrueger - 22 Feb 2016
 

ChangeProposalForm edit

TopicSummary Simplify adding JS/CSS to a page, at the same time reduce validation problems, improve security & cache features
CurrentState ParkedProposal
CommittedDeveloper
ReasonForDecision None
DateOfCommitment
ConcernRaisedBy MichaelDaum
BugTracking
RelatedTopics
PlannedFor
Topic revision: r7 - 22 Feb 2016, JanKrueger
The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License    Legal Imprint    Privacy Policy