[pmwiki-users] PageLists of database contents

Patrick R. Michaud pmichaud at pobox.com
Wed Oct 4 16:24:39 CDT 2006


On Wed, Oct 04, 2006 at 03:37:20PM -0500, Ben Stallings wrote:
> Patrick R. Michaud wrote:
> >1.  Create a custom PageStore object that converts database records
> >into (virtual) pages.  
> 
> Success!  I've got a new DataStore object (based on PageStore) that 
> supports the read, ls, and exists methods ....

Wow... that was quick!  Congratulations, excellent work!

> So that much is good.  However, I could use some help with a few 
> things... sorry to be pesky.
> 
> First and most obviously, this is extremely slow and inefficient on even 
> a modest-size database.  The ls() method generates a list of every 
> single record in every table of the database, and then PageList calls 
> the read() method to individually query each record and gloms its fields 
> together into a big block of text before attempting to parse the text 
> back up again into page text variables.  

Well, this is why there's a "1." in front of my "Create a custom
PageStore object ..." step above.  Step "2." is to get MakePageList
to be able to determine the set of pages without having
to call the ls() method.   :-)

Note that you've now literally reached the bleeding edge of PmWiki 
core development-- the hooks to allow custom inputs and filters to 
MakePageList didn't even exist in PmWiki as of a week ago.  But that's
really a good thing, as it's always better to have real implementation
issues at hand when doing design and development.

> ... I want to be able to use PageList's 
> powerful searching syntax and templates, but without the overhead of 
> converting every single record into text and back again.  Suggestions?

The key to making this more efficient will be to reduce the
overall number of "pages" (records) that MakePageList has to perform 
its searches on.   I'm thinking the best approach to this will
be for us to provide either (1) options to ListPages() and ls()
or (2) alternate methods entirely so that we can tell a 
PageStore/DataStore object "we want the names of 'pages' that
might match these criteria" as opposed to "give me a list of
all pages".  Then we can optimize things quite a bit.

I'm also needing to re-think a few of PmWiki's internals here,
especially the $PCache array.  PmWiki uses $PCache in a large
number of places to try to avoid doing much in the way of
"redundant" I/O, but it's been done haphazardly without any real
overarching design.  I wasn't planning to revamp $PCache for
2.2... but with the pagelist and page variable changes it's
becoming far more relevant, and it might be worth making the
change now (as part of a "major release") than to postpone
the pain until later when more modules may be depending on
the existing haphazard design.

> Second, if you try searching the Source group at 
> http://www.workscited.net/ben/Site/Search , it doesn't appear to search 
> page text variables... which was the whole point of putting the records 
> in that format.  For example, search for $:firstname=Will.  No dice, 
> although you can see by clicking the editing link above that record 12's 
> firstname is Will.  I'm using 2.0.0-beta-9.  Ideas?

Not at the moment -- seems like it ought to work.  What if
you just use a page text variable directly in a page, as opposed
to going through (:pagelist:) to get it -- does it correctly
grab values of {$:firstname} for records in the Source group?

I.e., does {Source.12$:firstname} work?

Pm




More information about the pmwiki-users mailing list