[pmwiki-users] Slashdotting?

J. Meijer commentgg at hotmail.com
Mon May 14 20:56:44 CDT 2007


On Mon, May 14, 2007 at 02:37:14PM -0700, pmwiki at 911networks.com wrote:
> On Mon, 14 May 2007 15:58:37 -0500
> "Patrick R. Michaud"  wrote:
>
> > These cannot be cached because they generate 
> > tags in the  section of the resulting HTML, as opposed
> > to most other markups which appear in a continuous section
> > of the .  So, in order to generate those entries
> > in the output HTML we have to actively process the markup.
> >
> > Yes, it would be nice if there was a way for us to keep
> > track of any output that needs to be cached for the
> >  section, but there's also the issue that the
> > order of entries in  matters as well.  So, it's not
> > a simple problem.
>
> What about caching the complete page, and if the date/time is more
> recent ...
> or
> What about a directive that say:
>    (:alwayscache:)
> Then the user is responsible...

On 5/14/07,  Patrick R. Michaud  wrote:
> We'd have to cache the HTTP headers as well.  But that might 
> be doable.



I think I've mentioned this before: it is possible to make some form of "overload protection" by taking the dumbest approach to caching: void the HTML cache entry (for browsing pages) if:

1. An author saves a new version of the page (obviously), and
2. It's been there for say 30 seconds. However, continue to serve the cache entry while the new version is being made. 

Maybe some indirect updates should also invalidate the cache. But since all cache pages will expire quickly, little harm will be done if that's not done. Most likely only developers will notice something out of the ordinary. 

Seems like this would be quite effective in decapitating the slashdot/digg-effect (for static pages) and it would cost just a few lines of code.  


Philosophy: we get mostly once chance to being seen: most visitors won't come back. So I think overload protection is an essential -core- feature. What use is a site when it can't cope with sudden popularity? All that effort waisted and the reason being is that the site chose PmWiki!? I believe no commercial software would make this mistake. 

Note: the digg-effect is likely just taking off. In a few months I've seen the most popular diggs rising from the 1000 to the 3000 mark. Much heavier loads are to be expected in the near future. Maybe digg itself will then start to enable access to a cached copy. 

/jm




ps With a 3Mbps pipe and a page-size of say 10..300kb, roughly 1..30 pages per second can be served. In 8 peak-hours that would allow 3k..100k page-views. With cache-entries lasting only 5 seconds, the demand on pmwiki could be reduced by 5..150x for your one special page. 

_________________________________________________________________
Create the ultimate e-mail address book. Import your contacts to Windows Live Hotmail.
www.windowslive-hotmail.com/learnmore/managemail2.html?locale=en-us&ocid=TXT_TAGLM_HMWL_reten_impcont_0507


More information about the pmwiki-users mailing list