Helper library to stringify binary document formats
This extension serves as a helper to stringify documents in a way that they can be indexed
by full-text search engines such as Foswiki:Extensions/KinoSearchContrib
It supports all major office document formats such as:
is organized in plugins to serialization a document format by delegating it to according backends.
For some formats there are alternative backends to chose from. For example a DOC file can be serialized
by any of
. Use the one that serves best your needs and
is available on your platform. For instance
is a very good choice to serve as a document converter.
However using it is rather performance demanding. The more simpler ones suffice most of the time but may
have an inferior quality of text being extracted.
Backends for Word Documents
To index Word Documents (
) you will need to install one of the following:
Backend for PDF
files you need to install
Backend for PPT
files you may select one of the following:
Backends for DOCX, PPTX, XLSX
To index these file types, you will need to install the following tools from Sourceforge:
Then set the command path to these tools in
Backend for OpenDocument and Staroffice documents
To index these file types you need to install odt2txt
Installing the Contrib
You do not need to install anything in the browser to use this extension. The following instructions are for the administrator who installs the extension on the server.
Open configure, and open the "Extensions" section. Use "Find More Extensions" to get a list of available extensions. Select "Install".
If you have any problems, or if the extension isn't available in
, then you can still install manually from the command-line. See http://foswiki.org/Support/ManuallyInstallingExtensions
for more help.
There are a number of settings that need to be set in
before you can use the Contrib.
Test of the Installation
- Test if the installation was successful:
- Check that
wvHtml is in place: Type
wvHtml on the prompt and check that the command exists.
- Check that
pdftotext is in place: Type
pdftotext on the prompt and check that the command exists.
- Check that
ppthtml is in place: Type
ppthtml on the prompt and check that the command exists.
stringify some files (see below)
Test of Stringification with
Some users report problems with the stringification: The stringifier scipts
fails, takes too long on attachments. Some times this may result from
installation errors, especially of the installation of the backends for the
give you the opportunity to test the stringification in advance.
In the result you see, which stringifier is used and the result of the
Umlaute: Grober, Uberschall, Anderung
In this extension, a plug-in mechanism is implemented, so that additional
stringifiers can be added without changing the existing code. All stringifier
plugins are stored in the directory
You can add new stringifier plugins by just adding new files here. The minimum
things to be implemented are:
- The plugin must inherit from
- The plugin must register itself by
- The plugin must implement the method
$text = stringForFile ($filename)
All the stringifiers have unit tests associated with them, and we would
encourage you to provide unit tests for any you wish to contribute. See
for more information on unit testing.
|Spreadsheet::ParseExcel||>0||Required for |
|Spreadsheet::XLSX||>0||Required for |
|pdftotext||>0||Required for indexing |
.pdf. Part of poppler-utils
|soffice||>0||One of antiword, abiword, soffice or wvWare is required for |
|antiword||>0||One of antiword, abiword, soffice or wvWare is required for =.doc files|
|abiword||>0||One of antiword, abiword, soffice or wvWare is required for |
|wvWare||>0||One of antiword, abiword, soffice or wvWare is required for |
|html2text||>0||Required for indexing html files|
|odt2txt||>0||Required for indexing OpenDocument and StarOffice documents|
| 18 Oct 2015:
|| (4.20) removed dependency on File::MMagic; now using extension-based mime detection
| 01 Oct 2015:
|| (4.10) don't default to pass-through for non-supported document types; fixed unit tests
| 29 Sep 2015:
|| (4.00) added unicode support with Foswiki > 2.0
| 22 Jul 2015:
|| (3.10) added support for stringification of ppt using catdoc as ppthtml isn't available on some distros
| 29 Aug 2014:
|| (3.00) added support for stringification using open/libreoffice
| 07 May 2012:
|| (2.20) added configuration parameter to specify the encoding of the output of each external helper in use
| 17 Oct 2011:
|| (2.10) using wvText instead of wvHtml now; encoding stringified files to the site's charset now; fixed unit tests to use utf8 exclusively
| 05 Sep 2011:
|| (2.00) added OpenDocument serializer; removed dependency left-over on Text::Iconv; added dependency on odt2txt; fixed defaults for wv serializer
| 01 Dec 2010:
|| (1.20) moved core from StringifierContrib to Stringifier not to disturb
| 12 Nov 2010:
|| (1.14) Foswiki:Main.PadraigLennon - Foswikitask:Item9311
| 23 Oct 2010:
|| (1.12) made system fault-tolerant in case of missing dependencies for a given file type; doc cleanup -- Foswiki:Main.WillNorris
| 12 Feb 2010:
|| robust parsing of password protected XLS files
| 02 Oct 2009:
|| extracted from Foswiki:Extensions/KinoSearchContrib (MD)