What's this all about? Well, having worked with csync2 combined with 10M+ files, with 100+ updates/second I can guess I have seen the limits of csync2, and they are well below 10 concurrent updates/second in a 2 way cluster. So the need arised to fix it.
Fixes have resulted in patches:
- sqlite3 support (is in upstream)
- limit support to limit the amount of files handled per batch (needed for database locking)
- Alternate temp directory option for csync
- csync2id daemon that uses inotify to keep stuff hinted to csync2.
- splitting of the database in a per table seperate database. It overcomes a lot, but it also introduces deadlocks ;-).
(You can find the remainder of the patches here:
http://projects.kwaak.net/twiki/pub/CSync2/WebHome/csync2-r401-experimental-patches.patch )
But none of them really fixes the big issue: concurrent database access on an sqlite database sucks bad, really really bad...
So I am about to say that the only solution is to make the database access part a single process, in other words: a client server architecture.
But which way to go? Just using an sqlite wrapper like uSQLiteserver would that solve stuff?
Well probably. It actually doesn't really matter: we are now at a point that we
must describe the states and interaction with other hosts in order to survive the current locking madness.
SomeDiagrams
Topic revision: r13 - 12 Mar 2010 - 07:35:31 -
TWikiGuest