Feature Proposal: Fix SEARCH limit and paging mess

Note: Most of this proposal could be done as a bugfixing Task. But because this touches the API (historic and future) I would like to see it covered well.

Motivation

We have a number of outstanding bug reports on the issue that the limit param does not work well with paging:

• Tasks.Item2232
• Tasks.Item10349
• Tasks.Item10471

In thinking and code those 2 concepts have been mixed up.

Old behaviour:

• limit sets the results per web. This cannot be changed because existing wiki apps rely on this.

Problems with limit combined with pager

• Limit is somehow changed to the page size
• Results per web are grouped but also paged, meaning that the results per web are arbitrarily cut off.
• Result count per web is shown, but is actually the number of results shown ON THAT PAGE - very confusing.
• We still don't have a way to limit the total results.

Current implementation problems:

• in Search.pm, limit is not obeyed: its value is set to pagesize.
• pagesize is used even if pager is off. So you get one page of results without seeing the rest. This looks like a results limit, but the name and implementation are confusing.
• in WebSearch:
  • limit is default set to DEFAULTPAGESIZE, which is 50 in DefaultPreferences
  • pager is hardcoded to on, and cannot be set by url parameter.
  • pagesize is set to DEFAULTPAGESIZE, cannot be set by url parameter

Description and Documentation

Proposed fixes:

• Keep limit per web only.
• Do not combine web limit and paging: if limit is set to a number (not 0 or all), paging will not be used, and pager and pagesize will be ignored.
• Possibly do not combine limit with other kinds of grouping (like with the new groupby)
• Create a new param maxresults to limit the total number of results. This does not group results per web.
• In WebSearch enable all SEARCH options by url param
• In WebSearch set the defaults to no limit, no maxresults, and pager on

Examples

Impact

WhatDoesItAffect:

%WHATDOESITAFFECT%

edit

Implementation

-- Contributors: ArthurClemens - 03 Mar 2012

Discussion

What I was heading towards was that limit becomes deprecated, as it was a hack to deal with large result sets before we had paging.

so I would change the above list to :

• Keep limit per web only.
• Do not combine web limit and paging: if paging is set to a number (not 0 or all), limit will be ignored.
• Do not combine limit with other kinds of grouping (like with the new groupby)
• Use pagesize (and pager="") to limit the total number of results. This does not group results per web.
• In WebSearch enable all SEARCH options by url param - covered by other feat-req
• In WebSearch set the defaults to no limit, and pager on - covered by other feat-req

There is another accepted and partially implemented feature proposal for adding paging to all shipped topics - I can't recal why i stopped part way though.

-- SvenDowideit - 04 Mar 2012

We should make more explicit documentation of pagesize and showpage to show how they can be used as skip and limit.

-- PaulHarvey - 05 Mar 2012

This does still not solve the problems when limit and pagesize are used together. This will happen by default in current wiki apps where limit is used.

SEARCH options pager and pagesize have a default value that work against limit. Together they will produce these problematic results. To view this, just use the current WebSearch.

Even if limit is deprecated we could give it precedence over paging. This would render results the old way, without paging.

Using pagesize as limit is quite obscure. If I wanted to show the 5 most recent topics, I would need to use pagesize="5" pager="off". Who would think of looking that those option descriptions? It is less intuitive than just maxresults="5" (and of course the pager should not be shown if the number of results shown is smaller than pagesize).

-- ArthurClemens - 05 Mar 2012

I believe ArthurClemens solution makes a lot of sense. I think that we already deprecated limit, so it should not be appearing in any of our templates. If it does, it should be removed. (Or we need to ship a LegacyWebSearch topic which preserves the old limit-per-web behavior). If someone uses the deprecated limit they get the obsolete / deprecated per-web behavior.

-- GeorgeClark - 06 Mar 2012

I think adding yet more parameters is a bad idea.

limit is deprecated, and pager is optional - so any apps that had limit in them must get edited to add a pager to them anyway.

but that said - there's a little more to it.

I made setting a total limit without a pager obscure for a reason - its rare that -_user_ wants to make the other results inaccessible - so adding a new single parameter that means the same as pager=off pagesize=5 is really odd.

I don't think need a legacy WebSearch that does limit. There should already be a sufficient mix or modern elements (I think) to give a similar but superior result - for eg

FORMAT{ "WEBLIST" format="SEARCH{ web="$item" pagesize="10" }"

}

would result in a WebSearch that is grouped by web and has a pager per web that can lead (I think) to the next page of only that web

OR

SEARCH{ web="public" pagesize=30 groupby=web }

would give you the first 30 matches, partitioning on the web name - so if each web has 45 hits, the first page would be just the first web, the second would have 2.

yes, both are not identical to what we have now - I think the first is superior, and the second 'interesting'

so I guess my biggest objection to maxsize is that it is obscure, and probably should be obscure.

-- SvenDowideit - 09 Mar 2012

Developer is no longer contributing. Changed status to 'ParkedProposal'

-- Main.CrawfordCurrie - 11 Jan 2016 - 18:00