Current State: Closed
Released In: 2.0.0
Target Release: major
While this is initially implemented to improve MongoDBPlugin
speed, the interface changes in the foswiki core will allow us to make dramatic improvements for all store & cache systems.
The core feature will move the ACL check from outside the ResultSet
iteration to a FilterIterator
wrapping the ResultSet
as it comes back from the Search and query Algo. This FilterItr
will be pluggable, to allow for other implementations, and for advanced query backends (like Mongo and Solr) can be a no-op - allowing full speed results.
The same technique will be applied to paging.
- 17 May 2011
We've successfully made MongoDBPlugin
massively faster at getting the first N hits of a query, but this doesn't help practical dashboards that need to show a paged view, and total number of hits for a given query.
Foswiki has no way of getting a "real" hit count unless it processes all 100,000 matching topics for ACLs. Because maybe the user only has permission to view 99,999 of them. Or just 6. Either way, we're back to square 1 with performance until we avoid forcing Foswiki to eval every topic for inclusion according to ACLs.
- 09 Mar 2011
Raised to normal, because eventually I really want to Foswiki to only be processing the topics it absolutely needs to render a given page
- 08 Apr 2011
this is going to be extremely difficult and require some very deep and risky changes to the core.
the result will be that the query in Item10432
should be very fast - at this point, line 745 of Foswiki::Search prevents it, because like most of the foswiki codebase, its coded to assume you need to parse and load a meta object to find out if its viewable.
- 28 Apr 2011
If there's any way I can help, let me know.
- 28 Apr 2011
because of the way they are interwoven, I've started by extracting the skipping for the pager - and trunk's pager should be O(1) as soon as i dcommit - as the core returns an aggregate iterator of ListIterators
, and arrays are easy to dynamically skip 10,000 elements instantly.
- 01 Jun 2011
this is feature complete - bugs should be raised in a new task.
unhappily, I commited MongoDBPlugin
work under this task too, but
It needs a better headline, as it really should be a HUGE speedup - page 1000 should take about the same amount of time to render as page 1.
- 01 Mar 2012