[pmwiki-users] smarter PageStore::ls() parameters?

[Patrick R. Michaud]

On Mon, Oct 16, 2006 at 11:10:03AM -0500, Ben Stallings wrote:
> I made some encouraging progress yesterday in speeding up the
> pagelisting of database information in the still-very-experimental
> dataquery recipe.  By caching data in a global variable and using
> ADODB's functions more wisely, I was able to reduce the number of
> database queries from one per record plus one per table to two queries
> total!  So that was encouraging.  The new version is at Cookbook/DataQuery.
> 
> But of course caching the entire contents of a database in a variable
> raises the specter of memory overflow.  And it's still pretty slow.
> 
> So I'm wondering if Pm could take a minute to share his thoughts about
> how a smarter pagelist function is likely to work in the future.  I'm
> not saying it should be a top priority for core development, but if I
> could prepare my recipe to work that way in advance of the core, it
> would save me a lot of work.  ;-)
> 
> In particular, the PageStore::ls() method currently generates a list of
> all pages in the wiki, regardless of whether group= and name=
> restrictions were placed on the pagelist, and so DataStore::ls() does
> the same for all records in the database.  

Actually, PageStore::ls() doesn't have to generate a list of all
pages in the wiki -- the $pats argument to ls() is an array of
name patterns the caller is looking for.  The ls() function is
allowed to return more than what the caller is looking for (e.g.,
the list of all pages in the store), but it can also be smarter
and reduce the number of pages being searched.

The current plan is to add an $options parameter to PageStore::ls(),
or (for compatibility reasons) to have a separate PageStore::select() 
method that takes this argument.  The $options parameter would
contain the things specified in the pagelist command, and the
PageStore may be able to use that to optimize the resulting query.
For example, if an author writes

    (:pagelist group=XYZ:)

then PageStore::ls() would be called with $options['group'] == 'XYZ'.

However, note that I don't know that this makes things much simpler
for PageStore::ls() over what already exists now.  In particular,
the author could write:

    (:pagelist group=X*,S*,-Sa* name=XYZ,-HomePage:)

and now PageStore::ls() has to be quite a bit smarter about
any optimizations it performs based on the $options array.  It may 
be better to simply filter based on the regex patterns already computed 
by pagelist (from the group=, name=, list=, and other options).

Hope this helps,

Pm