[mnet-devel] Ickle me, pickle me, tickle me too...

[Wes Felter]

On Thursday, August 7, 2003, at 04:49  PM, Jim McCoy wrote:
>
> 3) I can also just write out copies of the changed index items (each 
> insertion or deletion touches several index items) at each 
> insert/delete and push those objects into the pickle file, with the 
> object reconstitution consisting of creating an empty index and then 
> replaying all of the pickled index items. The downside to this is that 
> there may end up being a lot of these items and lots of "overwrites" 
> as things move along.

Ah, to checkpoint or to log, that is the question. I recall that some 
systems use a hybrid approach: they take a full checkpoint and then log 
every change after that. When the log reaches a certain size, take a 
full checkpoint and repeat. If you remember the number of entries 
you've written to the log then you can compare it against the size of 
your index; if the log has more entries than there are items in the 
index then it's probably a good time to take another checkpoint.

I question Zooko's suggestion of overwriting the checkpoint file 
in-place; I wonder what could happen during a crash. Maybe it's better 
to write the checkpoint to a new file, fsync it, and then rename it 
over the old one.

Wes Felter - wesley at felter.org - http://felter.org/wesley/



-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including
Data Reports, E-commerce, Portals, and Forums are available now.
Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01
_______________________________________________
mnet-devel mailing list
mnet-devel at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mnet-devel