Feature Proposal: Strikeone and friends need a cleaner API

Motivation

There needs to be a cleaner API to enable applications to POST to our /bin/scripts.

Examples:

• Support.Question317 - DiabJerius had an upload script working with (tm)wiki which had to be substantially reworked to cooperate with strikeone
• Extensions.SlickSitemapContrib - SvenDowideit's AJAX-ification allows drag & drop reparenting of topics, but only if {Validation}{Method} = none.
• Other applications which PaulHarvey is trying to develop depend on Javascript being able to make modifications to topics
• ChecklistPlugin and ChecklistTablePlugin should be re-written to work with HTTP POST + strikeone instead of HTTP GET to modify topic metadata; but in order to do tihs, they would need some way to obtain validation_keys

Foswiki's Javascript in general needs some refactoring (for example, strikeone.js provides functions in global context, why can't it be attached to the foswiki object?).

Two problems:

• For developers doing AJAX stuff, Foswiki's Javascript objects (or lack thereof) need to be more reusable (including perhaps for {Validation}{Method} possibilities other than strikeone)
• There needs to be a better way of getting a validation key to the client other than the current method of embedding the validation key into a FORM.

It should be possible, Eg. add GET method for /bin/save, add a new /bin/rest handler, etc. to easily obtain the secrets necessary to construct a good validation key against a desired topic.

Description and Documentation

Examples

Impact

WhatDoesItAffect:

%WHATDOESITAFFECT%

edit

Implementation

-- Contributors: PaulHarvey - 10 Nov 2009

Discussion

I'm hardly an expert on this one but if I recall correctly the combination of GET and save was removed because it allows cross site scripting.

-- CarloSchulz - 10 Nov 2009

I'm not an expert either, but my idea was that a GET on the save script might return media type: text/plain and contain nothing but the secret required to build a POST with the proper validation_key field required for strikeone.

I believe we removed GET from save and friends so that a GET could not perform any sort of write action. I don't think my idea conflicts with that goal.

It's how we currently have to do things anyway, although the GET is on the view script instead.

But I suppose the other possibility is a /bin/rest handler, or maybe yet-another-viewscript-parameter+template-or-something...

-- PaulHarvey - 10 Nov 2009

I have been developing a plugin (was actually in the process of packaging it up when this topic caught my attention) that puts into practice Paul's last suggestion above. It uses the completePageHandler with JSON, the redirectCgiHandler, some COVER templates (one for each of the bin/scripts) and, of course, javascript (jQuery) to reliably (i.e. w/o guesswork) implement the foswiki workflow via ajax, e.g. if a save goes wrong the Oops message gets shown to the user in an #oopsMessage div.

I did this oblivious to the CSRF concerns and strikeone etc. and now I've just read up about it, looked through the source code, validate.tmpl etc. and I don't see anything that can really prevent a hacker getting what he wants. I'm not an expert and I don't want to upset anyone but what is to stop a malicious piece of code sending the request to a foswiki server via XMLHttpRequest, waiting for the inevitable Validation response, skimming for the validation_key and submitting that?

I must be missing something because my understanding of it makes a CSRF attack inevitable...

-- DavidPatterson - 10 Nov 2009

Strikeone implements "double submit", which relies on the fact that 3rd-party JS from other websites can't access the value of the FOSWIKISTRIKEONE cookie.

BTW: Excellent work packaging up a plugin, looking forward to seeing it

-- PaulHarvey - 10 Nov 2009

Yes, I think I get it now, thanks. I'd missed the fact that CSRF is considered the opposite of XSS rather than an example of it and the cookie containing the secret cannot be read outside the domain of the server as it says in the pod.

-- DavidPatterson - 10 Nov 2009

It's probably going to take until next week to get this packaged properly (busy :/) so here is the zip file of the HijaxPlugin that I've got so far (you won't be able to simply unpack it and run it as is, no topic page for one thing, but you can look at the .pm file and the javascript in the HierarchyViewTemplate file). Like I say, it uses the redirectCgiHandler so it'll only work if there are no other plugins earlier in the alphabet that also use the redirectCgiHandler

It uses a simple ajax COVER that wipes the entire page but puts templates like content in tags that the completePageHandler recognises and returns to the client in a JSON object. Using COVER also means that VIEW_TEMPLATES are inherently respected. At the moment there are view.ajax.tmpl, oops.ajax.tmpl and login.ajax.tmpl (they all include ajaxbase.tmpl ). Each one has their own context template which also gets returned to the client so it can know if a save action was successful or not: e.g. if (json.context is 'view') then {success} else if (oops) then {problem, write the oops message to the #oopsMessage div}

I haven't implemented any code yet to handle a need to login to assess access rights for example, but the logic is simple

ajax.save {data: 'cover = ajax',
success: json
if (login) show json.content in modal
  hijax the login submit
    ajax.login {data: 'cover = ajax',
    success: response
      if (response.context = login)
        insert response.errorstep and fadeIn()
      else if oops {
         close modal
         $('#oopsMessage').prepend(response.content)
      else window.location.reload()

It would be a simple extension of this to create a validate.ajax.tmpl and then look for a validate context in the javascript to handle the validation request automatically.

This can also be used to do ajax uploads and return "file renamed" messages back to the client to flash up on the top of the page etc.

BTW: the plugin also includes the beginnings of an ajax enabled Hierarchy Application, HierarchyViewTemplate, where you can see the javascript code in action creating, reparenting and deleting topics with oops management. Currently it only offers Organigram building that uses the Slickmap CSS and jQuery (+ UI and Galleriffic plugins) - still a bit messy as I'm still extracting it from my system at work. Note that Sven's jQuery reparenting code uses textStatus which the jQuery API doco has the following to say about, 'Apparently, only "success" is returned when you make an Ajax call in this way. Other errors silently fail.' I haven't tested this.

-- DavidPatterson - 10 Nov 2009

The reason I didn't incorporate strikeone in the foswiki JS object was simply that I didn't think of it at the time. I can't think of any reaosn not to do that, though I also can't think of any reason (except tidiness) to do it either.

I agree that the API is not well documented, but it is quite useable - for example, I was able to integrate JHotDrawPlugin, and that is probably the most complex application example possible. I am reluctlant to rush into redesigning the API because in most cases, strikeone should "just work". In the other cases, we really need to know how the application wants to handle submission in order to work out what has to be done - for example, is the problem that the same key will be used repeatedly on submissions?

BTW I did consider a REST API to "get a validation key" but rejected it after I couldn't find an application for it.

-- CrawfordCurrie - 13 Nov 2009

In these cases, the AJAX app needs to submit many times, yes. Before each submit it seems we need to get a new validation key.

I didn't mean to rubbish the JS we have in foswiki, just that with all the debugging and work in edit templates with a gajillion plugins enabled (and even on test installs with a few pulgins enabled) our namespace in the DOM isn't consistent or tidy

Anyway, forgetting API changes.

We just need a nice way to get a new validation key. COVER template is how I was going to do it and it seems DavidPatterson has already done it.

-- PaulHarvey - 14 Nov 2009

Added Extensions.ChecklistPlugin and Extensions.ChecklistTablePlugin as example applications that need to submit more than once.

-- PaulHarvey - 18 Nov 2009

I considered adding a "get me a fresh key" API when I was working on JHotDraw, but eventually realised I just didn't need it.

The scenario under which you "need" a new validation key is when you have made a request to a validated script, and need to make a second request. A validated script is any script that changes the server state, such as save or upload. There are two cases where you may want to make the same request twice: (1) you have a form on the page and want to invoke the action from JS more than once or (2) you want the 'Back' button to allow resubmission without the annoying confirmation prompt. Without going into the code of the plugins you mention, which of these cases is the one you are concerned about?

BTW I'm currently of the opinion that a plugin should not use the standard save and upload scripts for this sort of duty. Instead it should use a REST handler with highly restricted functionality that supports validation without expiring the validation key (for an example of this, see Foswiki::Plugins::JHotDrawPlugin::_restUpload for an example of a validating REST handler). Writing this up for the DevelopersBible in HowToIntegrateWithRequestValidation.

-- CrawfordCurrie - 18 Nov 2009

Thanks for that topic Crawford, it's exactly what I needed.

I still feel that it shouldn't be up to an individual AJAX application to implement its own rest handlers (how many bugs and security flaws will accumulate in a Javascript hacker's Perl code vs a "supported" Foswiki API? :-), but perhaps we can generalise DavidPatterson, your JHotDraw stuff into a standard plugin for these situations.

(The plugins I've listed are examples of (1) where a Javascript app needs to submit many changes)

-- PaulHarvey - 19 Nov 2009

You have a good point there. There are two angles to that:

1. Javascript module to talk to REST handlers (perhaps a jQuery plugin?)
2. Encapsulation of the returnRESTResult functions that have blossomed around the place. These are the functions that do not return through the "standard" HTML return path, but instead return a "something else" result - for example, some other non-HTML MIME type. Typical examples of this are Foswiki::Plugins::WysiwygPlugin::returnRESTResult and Foswiki::Plugins::JHotDrawPlugin::returnRESTResult. A generic implementation might look something like this:

package Foswiki::Func;

REST_return($response, \%params, $body)

Helper function for REST handlers. Often you don't want to return a HTML result from a REST handler, but want to handle the return result yourself. Typically this is when you are returning something other than HTML, such as text/json or some other MIME type, or when you want control over the status of the result or some other HTTP header in the response. This function is provided to help you do this.

To use the function:

package Foswiki::Plugins::BathPlugin;

sub initPlugin {
   ...
   Foswiki::Func::registerRESTHandler('example', \&_REST_putPlugIn);
   ...
}

sub _REST_putPlugIn {
   my ( $session, $plugin, $verb, $response ) = @_;
   ...
   return Foswiki::Func::RESTreturn($response, { status => 418, type => 'text/plain' }, 'I'm a shower');
}

-- CrawfordCurrie - 19 Nov 2009

Commenting is disabled while not logged in