Item2535: Create W3CValidatorPlugin

Implement an easy way to validate the output of Foswiki, and offer a way of adding the W3C "passed validation" logo dynamically to pages based on cached results (looked up via a hash of the output).

• Ad-hoc, single-shot recording of output and validation results should be absolutely trivial to encourage the Foswiki community to write validating pages.
• Build a wiki app that plots or tabulates stats over time, broken down by scripts/topics/templates.
• Conventional link checkers/crawlers (and search engines for that matter) could do the job of poking Foswiki to exercise templates in various ways, but not all corner cases are easily covered automatically, hence the desire for a feasible "dynamic" W3C validation check on all pages for production wikis.

Eventually it should be that site admins could subscribe to the reports web and be notified of validation problems as they are detected.

---

This plugin hooks into completePageHandler to validate the output rendered by Foswiki using WebService::Validator::HTML::W3C.

A hash is computed against the output (after some experimental filtering to remove ?t=12nnnnnnn epoch values and <meta/> tags containing dates, etc.), which is then used to lookup and re-use existing validation reports. This should avoid running the W3C validator for a large number of requests.

Usage

An example template is provided that may be activated via PreferenceSettings as a skin cover. The most convenient method is to add Eg. ?cover=w3cval to the Foswiki URL.

• w3cval: If criteria are met, records the Foswiki output and schedules it for processing. If available, prints a message about the validation result; offers a link to process otherwise.
• w3cvalforce: As above, but do the validation regardless of includeurls, excludeurls.rules.

By default the reports are generated in the Sandbox.W3CValidatorReports subweb, each of which has a copy of the output that was processed attached to it. The underlying wiki application is based in System.W3CValidator.

REST handler

The following functions are available:

• /bin/rest/W3CValidatorPlugin/validate/(hash): Trigger processing of a particular Foswiki output file identified by its hash. Parameters:
  • reprocess: Force reprocessing (updates report attachment)
  • redirectto: A URL or Web.Topic that the browser will be redirected to after successful processing. Default: Report topic.
  • quiet: No result formatting
  • passformat: As per %W3CVALID{}% macro
  • warnformat: As per %W3CVALID{}% macro
  • errorformat: As per %W3CVALID{}% macro
• /bin/rest/W3CValidatorPlugin/validate_pending: Start processing Foswiki output files in the plugin's WorkingDir which have yet to be processed. Parameters:
  • web: Web to process. Default: * (all)
  • limit: Maximum number of output files to process. Default: 25
  • oldestfirst: Process from the oldest files first. Default: on
  • includeurlsfirst: As per %W3CVALID{}% macro..
  • includeurls: As per %W3CVALID{}% macro..
  • excludeurls: As per %W3CVALID{}% macro..
  • reportweb: As per %W3CVALID{}% macro.
  • redirectto: A URL or Web.Topic that the browser will be redirected to after successful processing. Default: %BASEWEB%.%BASETOPIC%
• /bin/rest/W3CValidatorPlugin/rebuildindex: For each Web, an index file is maintained in the plugin's WorkingDir to quickly check whether the Foswiki output has been reported against already. This function rebuilds that file from the reports gathered in the reportweb.
  • web: Web to rebuild index for. Default: * (all)
  • reportweb: As per %W3CVALID{}% macro.
  • redirectto: A URL or Web.Topic that the browser will be redirected to after successful processing. Default: %BASEWEB%.%BASETOPIC%

Configuration

The heart of the w3cval cover is the %W3CVALID{}% macro, which takes the following parameters (each of which overrides an equivalent Preference Setting variable, listed below each parameter):

• validatoruri: Default: http://localhost/cgi-bin/check
  • W3CVALIDATORPLUGIN_VALIDATORURI
  • NB: You may also wish to use the public service provided by the W3C themselves, http://validator.w3.org/check, however http://validator.w3.org/docs/api.html has a warning:

    Note: Please be considerate in using this shared, free resource. Consider Installing your own instance of the validator for smooth and fast operation. Excessive use of the W3C Validation Service will be blocked.

• reportweb: The web which will hold the validation report, if generated. Default: Sandbox.W3CValidatorReports
  • W3CVALIDATORPLUGIN_REPORTWEB
• includeurlsfirst: A URL matched with includeurls won't be excluded if it also matches excludeurls. Default: on
  • W3CVALIDATORPLUGIN_INCLUDEURLSFIRST
• includeurls: Regexp to match against full URL. Default: .*
  • W3CVALIDATORPLUGIN_INCLUDEURLS
  • NB: You may wish to use this parameter with %IF{}% statements to take advantage of Foswiki conditionals instead of writing regular expressions. Example:
    includeurls="%IF{"context='preview' OR 'System'='%BASEWEB%' then='.*'}%"
    (assuming excludeurls does not override it, generates a report if viewing the System web or the user is previewing an edit)
• excludeurls: Regexp if matched against a full URL, no report will be generated (assuming includeurls does not override it).
  • W3CVALIDATORPLUGIN_EXCLUDEURLS
• Format strings: The %W3CVALID{}% macro is replaced with passformat, warnformat, errorformat or schedformat depending on the result (and availability) of the validation..
  • passformat default:

    <p class='w3cvalpass'><a href='http://validator.w3.org/check/referer'><img src='http://www.w3.org/Icons/valid-xhtml10' alt='Valid XHTML 1.0, $passhits hits out of $nhits' height='31' width='88' /></a></p>

    • W3CVALIDATORPLUGIN_PASSFORMAT
  • warnformat default:

    <p class='w3cvalwarn'>[[$reportweb.$reporttopic][W3C Markup Validation Report]], $warnhits hits  out of $nhits.</p>

    • W3CVALIDATORPLUGIN_WARNFORMAT
  • errorformat default:

    <p class='w3cvalerror'>[[$reportweb.$reporttopic][W3C Markup Validation Report]], $errorhits hits out of $nhits!</p>

    • W3CVALIDATORPLUGIN_ERRORFORMAT
  • schedformat: Used if there is no report available for the calculated Foswiki output hash. Default:

    <p class='w3cvalsched' id='$hash'><a href='$valurl' title='Click to generate'>W3C Markup Validation Report</a></p>

    • W3CVALIDATORPLUGIN_SCHEDFORMAT
  • Available format tokens:
    • $reportweb: Web that reports are being generated in.
    • $reporttopic: Report topic for the Foswiki output that has been rendered.
    • $valdate: Date and time the Foswiki output with matching hash was last processed.
    • $valurl: Full URL including hash id to the plugin's validate rest handler
    • $hash: Computed hash of the Foswiki output as used in the report topics
    • $nhits: Number of times the hash has been encountered before
    • $schedhits: ... hits that occurred before processing.
    • $passhits: ... and passed validation
    • $warnhits: ... and only had warnings in the validation report
    • $errorhits: ... contained errors in the validation report.
  • quiet: No result formatting
    • W3CVALIDATORPLUGIN_QUIET

TODO

• CSS: Report inline fragments by hash id, /pub dir files by mod time
• Atom/RSS: Detect Content-Type headers and apply Atom/RSS validation (needs some smarts so as not to report on every single hit)

---

-- PaulHarvey - 28 Dec 2009

This work has stalled pending more urgent matters.

-- PaulHarvey - 16 Jan 2010

Hi Paul, Looks like a very interesting start. Is any of this in a repo somewhere? Or is it dead / we should close the task?

-- GeorgeClark - 07 Jan 2015