TIP MultiSearchPlugin is not installed on Foswiki.org.

MultiSearchPlugin

Perform multiple searches with one single efficient macro

This plugin is a supplement to the normally powerful built-in SEARCH feature in Foswiki.

SEARCH can perform very advanced searches with a large number of Foswiki topics and return the results in a very flexible way. You can create nested searches where a search depends on the result of a previous search.

But there are searches that the normal SEARCH only can do by repeating the same search over and over again. Each time you execute a new SEARCH all the same topics are being searched again and again.

MultiSearchPlugin enables you to perform only a few searches and display the result formatted by common index values, or by presenting the search results devided in intervals (numbers or dates)

The performance result can be dramatic. An example of searching 1300 records by 2 search criteria by 80 intervals may take 30 seconds using a FORMAT which again contains two SEARCH macros. Same can be done in 4 seconds using one single MULTISEARCH macro.

The MultiSearchPlugin always assumes searches to be query type searches and the search syntax is the same as in the normal SEARCH macro.

MultiSearchPlugin has 2 presentation modes

  • index - where all the search results are grouped by the existing values of the chosen index fields
  • interval - where all the search results are groups by defined intervals between a start and an end value. The feature is mainly meant for dates but can also be used for pure number fields using text fields

An index field is a named field in a form. You can define the index field as either text, multi, or date.

  • text - the results are groups by the raw text content of the field
  • multi - this is used when a field can have multiple values and you want the same search result to be counted for each value in the field
  • date - the date is treated as a date field

Index mode

Let us say that we have a bug reporting system. Each bug is assigned to an engineer. We would like to create a list of all the engineers that are current having at least one unresolved bug assigned to them and we want the list of bugs listed as a comma separated list of topic name links.

Here is how we can do it

%MULTISEARCH{
    search1="State='Assessed' AND Release='Project X'"
    web="Track"
    indexfield1="Assignee"
    indextype="text"
    indexmode="index"
    listseparator1=", "
    listformat1="[[$web.$topic][$topic]]"
    format="   * $indexfield - $list1$n"
}%

Another example is we have a customer complaint system. We would like see how many complaints each business area has and we would like to see how many are still not resolved. We want this in a table with one row for each business and one column for all products and for not yet resolved issues.

The records have a Business and a State field. The business field can have multiple values as the same product may cover several businesses. In a 3rd column we want the number of open severity High complaints. In a 4th column we want a list of all the open issues.

And finally a footer with the totals.

%MULTISEARCH{
    search1="State != 'Rejected'"
    search2="State = 'Open'"
    search3="State = 'Open' AND Severity = 'High'"
    web="Complaints"
    indexfield1="Business"
    indexfield2="Business"
    indexfield3="Business"
    indextype="multi"
    indexmode="index"
    listseparator2=", "
    listformat2="[[$web.$topic][$topic]]"
    header="| *Business* | *Total* | *Open* | *Open A-level* | *List of Opens* |$n"  
    format="| $indexfield | $nhits1 | $nhits2 | $nhits3 | $list2 |$n"
    footer="| | $ntopics1 | $ntopics2 | $ntopics3 | |$n"
}%

Interval mode

In this example we have web where we register issues. We would like to produce a chart of the arrival and closure rate accumulated per week. We want it only for project X. And we want the metric from the project start on January 1st till and including this week we are in.

The ChartPlugin needs a table with 3 columns. The X axis text, the data for the first series and the seconds series. For our own need it would be nice if we could click directly to the individual bugs so we add two more columns with links to the topics.

We do not want to count rejected and duplicates in the metric.

We have a field DateOriginated that gets set when we create the record. And we have a field DatePerformed which is set the day the issue is resolved.

%TABLE{name="srrate" columnwidths="150,100,100,200,200"}%
%MULTISEARCH{
    search1="(State!='Duplicate' OR State!='Rejected') AND Release='Project X'"
    search2="(State='Closed' OR State='Performed')  AND Release='Project X'"
    web="Track"
    indexfield1="DateOriginated"
    indexfield2="DatePerformed"
    indextype="date"
    indexformat="$day $mon $year"
    indexmode="interval"
    indexstart="01 Jan 2014"
    indexend="next Sunday"
    indexstep="1 week"
    listseparator1=", "
    listseparator2=", "
    listformat1="[[$web.$topic]]"
    listformat2="[[$web.$topic]]"
    format="| $indexfield | $ntopics1 | $ntopics2 | $list1 | $list2 |$n"
    header="| *Date* | *Opened* | *Resolved* | *Topics* | *Topics* |$n"
}%

Syntax Rules

MULTISEARCH -- perform multiple formatted searches in one efficient macro

The %MULTISEARCH% variable is handled by the MultiSearchPlugin

Parameters

Parameter Description Default
"..." Unnamed parameter is not used in this macro  
web Web in which the search is performed current web
topic Limit search to topics e.g.
topic="WebPreferences"
topic="*Bug"
topic="MyTopic,YourTopic"
A topic, a topic with asterisk wildcards, or a list of topics separated by comma.
Note this is a list of topic names and must not include web names. Adding a topic restriction to a search can greatly improve the search performance.
All topics in a web
excludetopic Exclude topics from search e.g.
excludetopic="Web*"
excludetopic="WebHome, WebChanges"
A topic, a topic with asterisk wildcards, or a list of topics separated by comma.
Note this is a list of topic names and must not include web names.
''
format Format of the resulting search for each index value of interval. Typically defines a bullet point or a table row ''
header The header is output just before the formatted search results. Typically used for table headers. See tokens used in headers below  
footer The footer is output just after the formatted search results. Typically used for table footers and for summary texts. See tokens used in headers below  
indexmode Defines the mode of the search. Valid values are index and interval index
indextype Defined how to treat the index fields. All index fields are treated as the same type.
text is plain text
multi means the field has multiple values
date means the field is interpreted as a date. The plugin will understand dates in the format described in System/TimeSpecifications and any form field defined as a date field
text
indexformat Only for date type fields. Defines how the intervals are displayed when showing the start date. Defined by using Time Format Tokens. See below. If this is not defined dates are shown as epoch and text just presented in the original format undefined
indexstart For interval searches this defines the start value. For dates this is the date of the first interval. See note 2 about dates below. undefined
indexend For interval searches this defines the end value. The actual end of the last interval will depends on the indexstart and indexstep because the plugin assumes you will always want the last interval to be a complete interval. See note 2 about dates below. undefined
indexstep The size of each interval. For dates it can be number of seconds or it can be any positive relative time. See note 3 below 1 or 1 week
search# The # is a number starting from 1. The search syntax is 100% identical with query type searches using standard SEARCH. You cannot jump the number sequency. The number must start with 1 and increment by 1 for each new search option. ''
listformat# The format that is used for each found topic. These formatted lists are then joined together using the listseparator and output with the $list# (# - the search number) token inside the format parameter ''
listseparator# The number # must also be used in a search# option. This is the format used for the $list# token used in the format string. Typically this is set to ', ' (comma space) to create a list separated by commas. ''
indexfield# The name of the form field used for search number #. ''
delay="#" Number of times the MULTISEARCH will delay its own expansion by changing the first and last %-sign to $percnt and all double quotes (") to $quot. This is used when you use MULTISAVE inside the format parameter of a SEARCH or FORMAT macro and you do not want to expand MULTISEARCH until the outer SEARCH/FORMAT is completed. Set the delay to 1 if it is a single level SEARCH/FORMAT. If you have nested SEARCHes you can set delay to the level of nesting. 0

  • Note1: The parameters formats, headers, and footers do not by default put a new line at the end. You must always specify this with a $n if you need a new line (you will need this in most cases)
  • Note2: Dates given to indexstart and indexend can be entered in two different formats
    • It will first try the normal Foswiki date/time formats as described in TimeSpecifications.
    • If the above fails it will try and calculate the date based on relative time that the CPAN library Time::ParseDate can handle. E.g. "-3 months" , "last Monday", "+2 days", "next Wednesday", "2 weeks"
  • Note3: indexstep relative times is always relative to now. You can use any time format that the CPAN library Time::ParseDate can handle. E.g. "1 month", "7 days", "1 week", "2 weeks"

Parameter Tokens

header="..." parameter, footer="..." parameter

Use the header and footer parameter to specify the header and footer of a search result. It should correspond to the format of the format parameter. Both parameters are optional.

Example header:
header="| *Business* | *Total ($ntotal* | *Open* | *Open A-level* | *List of Opens* |$n"

Example footer:
 footer="| | $ntopics1 | $ntopics2 | $ntopics3 | |$n"

Format tokens that can be used in the header and footer strings:

Name: Expands To:
$ntopics# Number of topics found in current web. The total is calculated AFTER all the searches are complete
$calc(...) Perform simple calculations of numbers inside the ( ). Only numbers and +, -, *, and / is allowed. $calc is evaluated after $ntopics# so you can write $calc($ntopics1 - $ntopics2) to get the difference between the two results

  • In addition you can use all the standard format tokens with the header and footer parameters. See below.

listformat#="..." parameter

Example:
listformat1="$formfield(Originator)" listformat2="[[$web.$topic][$topic]]"

Format tokens that can be used in the listformat# string:

Name: Expands To:
$web Name of the web
$topic Name of the found topic
$formfield(name) Content of any formfield of the given topic. No formatting is done with the formfield content. It may contain characters that needs to be dealt with. (This may be enhanced in later versions of the plugin)

listseparator#="..." parameter

Example:
listseparator2=", "

You can use all the standard format tokens with the listseparator# parameter. See below.

format="..." parameter

Example:
format="| $indexfield | $ntopics1 | $ntopics2 | $list1 | $list2 |$n"

Format tokens that can be used in the format string.

Name: Expands To:
$indexfield When indexmode is index $indexfield show the common value of the indexfields
When indexmode is interval $indexfield shows the lower value of the interval
$list# The # is the number of the corresponding search number. $list# displays all the values given by the listformat# parameter which are joined together by the string defined by the listseparator# parameter
$nhits# The number of topics found by this search number # for this index value or interval.
$ntopics# $ntopics# is the total of topics found in search# in the accumulated indexes or intervals shown until now. NOTE! For intervals the $ntopics value starts at the number of topics found having an indexfield value below/before the indexstart parameter value
$calc(...) Perform simple calculations of numbers inside the ( ). Only numbers and +, -, *, and / is allowed. $calc is evaluated after all other tokens are done so you can write $calc($ntopics1 - $ntopics2) to get the difference between the two results

  • In addition you can use all the standard format tokens with the format parameter. See below.

indexformat="..." parameter

indexformat is only used when the indextype is date and it accepts these special time format tokens

Token: Unit: Example
$seconds seconds 59
$minutes minutes 59
$hours hours 23
$day day of month 31
$wday day of the Week (Sun, Mon, Tue, Wed, Thu, Fri, Sat) Thu
$dow day of the week (Sun = 0) 2
$week number of week in year (ISO 8601) 34
$month short name of month Dec
$mo 2 digit month 12
$year 4 digit year 1999
$ye 2 digit year 99
$tz either "GMT" (if set to gmtime), or "Local" (if set to servertime) GMT
$iso ISO format timestamp 2024-03-28T23:59:26Z
$rcs RCS format timestamp 2024/03/28 23:59:26
$http E-mail & http format timestamp Thu, 28 Mar 2024 23:59:26 GMT
$epoch Number of seconds since 00:00 on 1st January, 1970 1711670366

Tokens can be shortened to 3 characters

Standard Tokens for header, footer, listseparator#, and format parameters

Name: Expands To:
$n or $n() New line. Use $n() if followed by alphanumeric character, e.g. write Foo$n()Bar instead of Foo$nBar

TIP Most macros accept parameter strings which are split over multiple lines. This is usually more readable than using $n tokens. If you are familiar with sectional includes, you might also consider nested sectional includes to hold the newline content outside of the parameter string entirely.

TIP Note that newline is not a line break. The browser will wrap the lines together. If you require a line break, displaying the results on two lines, use %BR%. Or use two consecutive newlines to create a TML "Paragraph".

$nop or $nop() Is a "no operation". This token gets removed; useful for nested search
$quot Double quote (") (\" also works)
$percent Percent sign (%) ($percnt also works)
$dollar Dollar sign ($)
$lt Less than sign (<)
$gt Greater than sign (>)
$amp Ampersand (&)
$comma Comma (,)

Installation

You do not need to install anything in the browser to use this extension. The following instructions are for the administrator who installs the extension on the server.

Open configure, and open the "Extensions" section. Use "Find More Extensions" to get a list of available extensions. Select "Install".

If you have any problems, or if the extension isn't available in configure, then you can still install manually from the command-line. See http://foswiki.org/Support/ManuallyInstallingExtensions for more help.

The plugin requires the CPAN library Time::ParseDate. Redhat/Centos users can run 'yum install perl-Time-ParseDate-2013' with the 'epel' repo enabled. Debian based distributions can install libtime-modules-perl

Plugin Info

Change History:  
1.3 (08 Dec 2015) Fixed a crash of the plugin when a search did not return any results
1.2 (19 Aug 2015) Added support for topic and excludetopic parameters to improve performance. Added support for the $calc() token to enable simple calculations within the plugin. Added more user input check of the field names
1.1 (18 Aug 2015) Better handling when using month as relative time (same date in next month instead of fixed number of seconds).
More checks on input values to avoid infinite loops
Support of the delay parameter when using MULTISEARCH inside a nested SEARCH or FORMAT
1.0 (17 Aug 2015) Initial release
Dependencies:
NameVersionDescription
Time::ParseDate>0Required

I Attachment Action Size Date Who Comment
MultiSearchPlugin.md5md5 MultiSearchPlugin.md5 manage 174 bytes 07 Dec 2015 - 23:28 KennethLavrsen  
MultiSearchPlugin.sha1sha1 MultiSearchPlugin.sha1 manage 198 bytes 07 Dec 2015 - 23:29 KennethLavrsen  
MultiSearchPlugin.tgztgz MultiSearchPlugin.tgz manage 14 K 07 Dec 2015 - 23:28 KennethLavrsen  
MultiSearchPlugin.zipzip MultiSearchPlugin.zip manage 17 K 07 Dec 2015 - 23:28 KennethLavrsen  
MultiSearchPlugin_installerEXT MultiSearchPlugin_installer manage 4 K 07 Dec 2015 - 23:28 KennethLavrsen  
Topic revision: r4 - 07 Dec 2015, KennethLavrsen
The copyright of the content on this website is held by the contributing authors, except where stated elsewhere. See Copyright Statement. Creative Commons License    Legal Imprint    Privacy Policy