[pmwiki-users] Trouble with .pageindex when too much _new_ data to index (+ sqlite)

[Patrick R. Michaud]

On Wed, Jan 28, 2015 at 10:10:06PM +0100, ABClf wrote:
> Main issue encountered is how .pageindex is handling its indexation task.
> It sounds like it definitely stops working when the amount of _new_ data is
> too big. [...]

I don't think this is quite how it works.  If you're examining the code to
see how things work, it would help a lot if you could refer to functions by
name.

Here are a couple of possibly helpful items, though.  The amount of time
PmWiki spends building .pageindex is controlled by the $PageIndexTime variable,
which defaults to 10 seconds.  This variable controls how long PmWiki spends
indexing pages after serving a page in response to a search; increasing the
value of this variable should allow more pages to be indexed during each
request.

As far as I remember, PmWiki doesn't check to see if the amount of new 
data is acceptable to index.  PmWiki simply has a list of pages that it
knows aren't in the index, indexes as many as it can in $PageIndexTime 
seconds, and leaves the rest for a later request to index.  

If there's a single page that is too large for PmWiki to index because of 
PHP memory limits, I'm not sure there's a whole lot we can do about that 
short of increasing the memory limits.

I think it would not be difficult to create a custom action that does
nothing but pageindexing for a very large page or set of pages... but I
really need to know more precise details about what you're trying to do
before I can help further, such as specific page names and exactly what
they contain.

Pm