TIP XSendFileContrib is not installed on Foswiki.org.

XSendFileContrib

Introduction

This package implements a more efficient way to send static files using a web application like Foswiki. A standard configuration of a Foswiki uses the viewfile service to send static files over to the browser thus enforcing access rights on them. In doing so it reads the static file into memory completely before handing it over to the HTTP web server which takes over responsibility to contact the browser. This of course is far from optimal as it introduces a lot of overhead by copying the static file several times in memory. The impact on the overall system performance is even higher for large files being downloaded from a Foswiki server. For now the only alternative solution to this problem is not to protect static files by Foswiki's access control at all, and let the HTTP web server do its job on static files all on its own. Web servers are in fact quite good in sending static files, even large ones. So the best you can do is to leave this job to them.

Still you might require access control, even on large static files.

There's a quite unknown yet very important feature in Apache2, Nginx and Lighttpd, called "xsendfile", to interact with an upstream web application before sending a static file. This feature can be used to solve the above problem: it lets you control access to static files while still using the HTTP web server to do the heavy lifting.

This basically works like this:

  • client requests an url
  • the web server calls an upstream web application for this url
  • the web app performs its specific actions and returns a specific HTTP header in its response, e.g. X-Lighttp-send-file, but not the data being requested, when access is granted, or an HTTP error otherwise
  • the web server parses the HTTP headers returned by the web app looking for the X-Send-File header
  • if it finds the X-Send-File header then the will the web serve transfer the static file this header points to over to the client
  • otherwise it will return the response generated by the web app as usual

Configuring Foswiki

XSendFileContrib comes with a separate service script .../bin/xsendfile replacing the standard .../bin/viewfile. It differs from viewfile only in two respects

  • any HTTP error is delivered immediately, that is without requiring Foswiki to render an error page while processing an exception.
  • instead of copying over the static file to the downstream HTTP web server, it generates an empty response with the approrpiate X-Send-File header set

There are two parameters that need to be adjusted according to your web server's type and configuration:

  • $Foswiki::cfg{XSendFileContrib}{Header} ... the name of the HTTP header to trigger the X-Send-File feature in your web server (see below)
  • $Foswiki::cfg{XSendFileContrib}{Location} ... the uri prefix that the web server serves the actual static file from

Configuring Lighttpd

(see http://redmine.lighttpd.net/projects/1/wiki/X-LIGHTTPD-send-file for lighttpd)

The X-Send-File header for Lighttpd is called X-LIGHTTPD-send-file and points to the file path on disk that the web server should send to the client.

If you are already using Foswiki behind a lighttpd web server, all you need to do is to add "allow-x-send-file" => "enable" to your fastcgi stanza

url.rewrite-once += ( "^/pub/((?!System|Applications|images|cache).*)/(.*?)$" => "/bin/xsendfile/$1/$2" )

$HTTP["url"] =~ "^/bin/" {
  alias.url += ( "/bin" => "/path/to/bin/foswiki.fcgi" )

  fastcgi.server = ( ".fcgi" => ((
...
      "allow-x-send-file" => "enable"
    ),
  ))
}
$HTTP["url"] =~ "^/bin/xsendfile)" {
  expire.url = ( "" => "access 12 hours")
}

Configuring Nginx

(see http://wiki.nginx.org/XSendfile, http://wiki.nginx.org/X-accel)

In Nginx this feature is called X-Accel-Redirect. Yet the idea is the same. Requirements on the upstream web applications are almost identical besides using a different X-Send-File header called X-Accel-Redirect.

In addition Nginx requires a special "location" stanza for internal use only. Protected files will be served from there one the web app checked the original uri the.

location ~ ^/pub/(System|Applications|images|cache)/ {
  root /path/to/foswiki;
  expires 12h;
  gzip_static on;
}

location /pub {
  rewrite ^/pub/(.*)$ /bin/xsendfile/$1;
}

location /protected_files {
   internal;
   alias /path/to/foswiki/pub/;
}

Configuring Apache2

Apache itself requires an additional simple module that processes an X-Sendfile header. In a classical Foswiki Apache2 installation you have to modify your web server configuration as follows:

XSendFile on
XSendFilePath /path/to/pub
RewriteRule ^/+pub/+(.*)$  /bin/xsendfile/$1 [L,PT]

See https://tn123.org/mod_xsendfile/ for more information on how to compile mod_xsendfile and how to configure apache.

Installation

You do not need to install anything in the browser to use this extension. The following instructions are for the administrator who installs the extension on the server.

Open configure, and open the "Extensions" section. Use "Find More Extensions" to get a list of available extensions. Select "Install".

If you have any problems, or if the extension isn't available in configure, then you can still install manually from the command-line. See http://foswiki.org/Support/ManuallyInstallingExtensions for more help.

See above for more information on how to configure Foswiki and your web server.

Info

Copyright ©: 2013-2015, Michael Daum http://michaeldaumconsulting.com
Dependencies:
NameVersionDescription
Foswiki::Contrib::FastCGIEngineContrib>=0.9.5Optional.
File::MMagic>0Required.
Change History:  
17 Jul 2015 added support for Foswiki-2.0
29 Aug 2014 improved decoding and untainting url components
28 May 2014 fixed file check on wrong location
15 Apr 2014 untaint attachment filenames
18 Mar 2014 return a proper 404 for invalid filename parameters
06 Nov 2013 fixed filenames with spaces not being found
01 Nov 2013 added support for if-modified-since http headers
22 May 2013 using mime-magic as a fallback in case file extensions don't unveil the mime-type
28 Mar 2013 implemented {AccessRules} to allow any kind of access control list for attachments
1.00: Initial version
Topic revision: r12 - 17 Jul 2015, MichaelDaum - This page was cached on 30 Aug 2016 - 13:59.

The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License