• View
• Edit
• History
• Attach
• Print
• Backlinks

CompressedPageStore

Question answered by this recipe

PmWiki saves its pages in flat files -- Is there any way to store those files in compressed (gzip) format?

Answer

The pagestoregz.phpΔ script defines an alternate storage mechanism for storing pages as compressed files in the wiki.d/ directory. To use this script, simply place it in your cookbook directory and add the following lines to local/config.php:

    include_once('cookbook/pagestoregz.php');

There's no need to convert or modify your existing pages; pagestoregz.php can read existing (uncompressed) pages without any difficulty. As pages are edited and saved, they will be then be saved as compressed files in wiki.d/. Please make sure the above cookbook script is loaded before other scripts or localization files, otherwise changed localizations will not display.

If you want to read compressed files and save them uncompressed, just add the following line to local/config.php

    $EnablePageStoreGZ = 0;

If you want to compress or uncompress all of your files at once add the following line to local/config.php:

    ConvertGZ();

Depending in the state of $EnablePageStoreGZ all your files are compressed or uncompressed.

By default compressed files are reread and checked for size difference. If the difference is too high it is assumed that the saved file was corrupted or something else went wrong. You'll then get a message that saving was not possible. As this consumes some time you can switch off this behaviour by setting the following value. But be careful! Once a file is corrupted you are not able to restore it (except from a backup).

    $EnablePageStoreVerify = 0;    // Disables verification of saved compressed file

Notes

History

• 2005-01-26 Added Variable $CompressPagesOnSave which controls whether the pages are compressed when saved or not (default=1 : compress on save). Nice if you want to switch back to uncompressed later. Added also a function ConvertGZ which compresses/uncompresses all existing pages.
• 2005-01-27 Found a problem and fixed it: When copying compressed files from a linux server to a windows server the files were not readable due to different end-of-line characters (at least I think this is the reason). Now LF is assumed as end-of-line character when reading compressed files.
• 2005-04-13 Updated script to contain same optimizations in storage as introduced with PmWiki 2.0 beta 29. (Still no problems with this script Klonk)
• 2005-09-02 Updated script to changed storig format as introduced with PmWiki 2.0.0 Klonk
• 2005-09-03 Updated script to changed storig format as introduced with PmWiki 2.0.2 Klonk
• 2005-09-05 Added switchable checking when saving compressed files. Klonk
• 2005-09-09 Updated script to match pmwiki-2.0.5 save algorithm and correct handling of $Newline, changed name of flag variables to be $EnablePageStoreGZ and $EnablePageStoreVerify (to match PmWiki's configuration variable conventions).

See Also

Contributors

• Pm, 2005-01-14 (original script)
• Klonk, 2005-01-26 (added $CompressPagesOnSave variable and ConvertGZ function)
• Pm, 2005-09-09 -- updates for 2.0.5

Comments

• Great idea! One question... how does PmWiki do a search on compressed files? Uncompress them one at a time? What does this do to response time on searches? Keith Campbell
  • To be honest, I'm not really sure, but as soon a file is opened (read) it is decompressed, i.e. a PHP function is used that handles compressed files directly.Klonk
  • The files are uncompressed as they are read, so searches may be slightly slower but shouldn't be noticeably slower. --Pm
• Just for information: I had this script running for 6 weeks now and encountered no problems :) Klonk
  • This is a great recipe but it doesn't work with Cookbook.CreditsBlock.
    • I don't see a reason why it shouldn't work. What I once noticed that I had to take care to call his script as early as possible as otherwise some page could not be read after editing, thus compressing them. BTW as I included Cookbook.CreditsBlock I didn't get any display. No matter whether I included pagestoregz or not.--Klonk
      • You are correct! I placed include_once('cookbook/pagestoregz.php'); as the second line in my config.php (after turning on zlib compression) and now everything works wonderfully, including Cookbook.CreditsBlock. Thank you for your assistance.
• Good work but I found problem with wikifarm. If pages under $FarmD/wiki.d/ are compressed and one field wiki uses a page, for example Main.ABC which does not exist in the field wiki but does in $FarmD/wiki.d/, the field one will use $FarmD/wiki.d/Main.ABC but display nothing. --weijiang September 25, 2005, at 09:47 AM
  • Make sure to load pagestoregz.php early enough in your config.php! This can make a great difference! - Klonk
• I am sorry, but is it possible to use similar technique to uploaded files? I mean, it would be great if you can upload some files and the would saved compressed. I think this problem can be solved as web-server extension but i don't want to have apache on my pocket flash drive. Michael Savchuk
• This does not see to work with the PerGroupSubDirectories recipe. Jonathan Cutrer