for the record this is not an exciting or cool video… its just me… cutting up fruit… silently

The segfaulting was caused by stupidly using $this->function() from inside of $this->function() for recursion, overloading the stack. Note to self: don’t be lazy about recursion

If you’re interested in the project I have it setup over here: http://code.svn.wordpress.org/lockd/ so please feel free to try it out.

What is it?  It’s a network based locking mechanism.  You can connect to it over TCPIP and get either exclusive or shared locks for as many arbitrary strings as you like.  Shared locks have the added benefit of being counted.  You can query to see if a string is locked (and how many times for shared locks.) You can also release locks.

All of this is useful, but… The nice thing about lockd, though, is that if your client disconnects (say the process which had locked a string quit unexpectedly) lockd will automatically orphan those locks for you.  So you have no need to try and cleanup and guess the lock state of a failed process.   And since this runs as a network daemon you can have as many of them as you like.  You can use them for local (locks on the same box) or distributed (locks on the same resource from many boxes.)  It also comes with built in stats which you could hook up to munin, or nagios, via netcat without too much trouble.

The protocol is also extremely simple

• g (string) — get an exclusive lock on the string
• sg (string) — get a shared lock on the string
• r (string) — release an exclusive lock
• sr (string) — release a shared lock
• i (string) — inspect an exclusive lock
• si (string) — inspect a shared lock
• q — query stats
• q full — query stats and dump the exclusive and shared lock arrays

Thats it.  “(int) (string)” is returned for these commands. where (int) is for programmatic use, and (string) is for human consumption.

We use this with the Jobs system (also available as an OSS project) for WordPress.com, Intensedebate, Gravatar, and more, and is currently handling nearly a million operations per day… per server…

This is Itch-Scratch-Ware at its finest.

Step 1: add this to the top of your entry point php file

if ( $_SERVER['REMOTE_ADDR'] == '127.0.0.1' ) {
       error_log( ' :: ' . getmypid() );
       sleep( 10 );
}

Step 2: use curl on the localhost to make the request

Step 3: (this assumes your error log is /tmp/php-error-output) run the following command in a second (root) terminal window

strace -p $(tail -n 1000 /tmp/php-error-output | grep ' :: ' | tail -n 1 | sed -r s/'^.+ :: '//g) -s 10240 2>&1

Good luck…

I asked Joseph to go over it and submit the good stuff upstream to the wordpress.org project (he’s had a lot more experience working with the actual xmlrpc clients than I do, so his experience there in making sure that they wouldn’t be disturbed by my changes was very valuable.)

The takeaway from this is pretty simple, but at the same time quite profound. When working with arbitrary data inputs that you expect to be large you really have to watch the copy-by-value operations that you do.  Referencing is one of those things that you may not need to know in detail for every day coding, but its good to be aware of them and have an idea of what they are/do.  In cases where you find yourself inexplicably running out of RAM a couple of references can save the day.

Related/restated take-aways are:

• The idea of balacing two variables in ram by adding X bytes to one and then subtracting those xbytes from the other. (xml parsing)
• Also that certain types of operations create a temporary variable in ram with which to work with your data, so limiting the amount of data they need to function can be helpful when the variable is of considerable size (preg_replace)

All in all this should be a considerable win for WordPress (dot com and dot org) xmlrpc users.  If you’re running a self-hosted wordpress install and want to test it out and comment on the patch i’m sure everyone would be greatful.

DK