php debugging the really really hard way

If you’re ever in a situation where something is only happening intermittently, and only on a live server, and only while it’s under load… Lets say its not generating any error_log or stderr output, and you cant run it manually to reproduce… (we’ve all been in this situation) How do you get any debugging output at all?

Step 1: add this to the top of your entry point php file

if ( $_SERVER['REMOTE_ADDR'] == '127.0.0.1' ) {
       error_log( ' :: ' . getmypid() );
       sleep( 10 );
}

Step 2: use curl on the localhost to make the request

Step 3: (this assumes your error log is /tmp/php-error-output) run the following command in a second (root) terminal window

strace -p $(tail -n 1000 /tmp/php-error-output | grep ' :: ' | tail -n 1 | sed -r s/'^.+ :: '//g) -s 10240 2>&1

Good luck…

My inner geek is really showing tonight

I think its the Penny Arcade posts… I really do… But lately I miss the old days when I used to play Magic the Gathering, or Dungeons and Dragons. Maybe some geek(s) out there in cyberspace and in the Pittsburg, Ca area will find this floating message in a bottle and reach out. Maybe not, and if they don’t maybe its for the better?

Some WordPress XMLRPC Love

So I had to help fix an internal issue on WordPress.com where some largsish bits of data are transferred around via XMLRPC. We like XMLRPC, its simple, extensible, and works. But one thing that the IXR_Server is not is ‘light’. Because of the way that it works all of the data is transferred base64_encoded which means that a 10mb attachment just grows and grows and grows in ram on the server side… Which is exactly the problem I was faced with. I spent about a full day pouring through the code and its current and peak memory usage at various places in the flow of execution and put together a patch to help address some of the most glaring memory hogs.

I asked Joseph to go over it and submit the good stuff upstream to the wordpress.org project (he’s had a lot more experience working with the actual xmlrpc clients than I do, so his experience there in making sure that they wouldn’t be disturbed by my changes was very valuable.)

The takeaway from this is pretty simple, but at the same time quite profound. When working with arbitrary data inputs that you expect to be large you really have to watch the copy-by-value operations that you do.  Referencing is one of those things that you may not need to know in detail for every day coding, but its good to be aware of them and have an idea of what they are/do.  In cases where you find yourself inexplicably running out of RAM a couple of references can save the day.

Related/restated take-aways are:

  • The idea of balacing two variables in ram by adding X bytes to one and then subtracting those xbytes from the other. (xml parsing)
  • Also that certain types of operations create a temporary variable in ram with which to work with your data, so limiting the amount of data they need to function can be helpful when the variable is of considerable size (preg_replace)

All in all this should be a considerable win for WordPress (dot com and dot org) xmlrpc users.  If you’re running a self-hosted wordpress install and want to test it out and comment on the patch i’m sure everyone would be greatful.

DK

Just because you build it, doesnt mean they will come

Here’s a small bit of advice for all you would-be “cloud storage providers.” Just because you have a buttload of disks doesn’t mean people will be falling over themselves to use your software. If I have to spend *any* of my time worrying about your load, storage, or other internal algorithms (or unnecessary limitations for that matter) then YOU . HAVE . FAILED.

If I have to take the time to shard my data into 4096 different containers because you couldn’t be bothered to think “hey what if a service with a lot of users that create a lot of stuff decides to use us as a store?” Then you’re obviously not in it to win it (so to speak.)

Give us ABSTRACTED storage. Non abstracted storage we can do on our own thank you.

Just what you need to know to write a CouchDB reduce function

Lets say you have the CouchDB classes (located here) all compiled together and included into your test.php script. Lets also say that you have created a database with the built-in web ui called “testing”. Finally let us say that your test.php has the following code in it, which would add a record to the db every time it is run. (i know that the data in the document serves no useful purpose… but really I just want to figure out this map/reduce thing so that I can make awesome views… so this suffices sufficiently.)

require_once dirname( dirname( __FILE__ ) ) . '/includes/couchdb.php';
$couchdb = new CouchDB('testing', 'localhost', 5984);
$key = microtime();
$result = $couchdb->send(
    '/'.md5($key),
    'put',
    json_encode(
        array(
            "_id" => md5($key),
            "time" => $key,
            'md5' => md5($key),
            'sha1' => sha1($key),
            'crc' => crc32($key)
        )
    )
);
print_r($result->getBody(true));

After running the code a bunch of times you would end up with a bunch of documents which look more or less like this:

picture-1(click for full size)

Now lets say you want to write a view that told you what the first characters of the _id were and how many documents share that first letter. This is analogous to the following in MySQL

SELECT LEFT(md5, 1) AS `lchar`, count(md5) FROM `md5table` GROUP BY `lchar`

Your map function is easy, because you dont have any selection criteria, so we process all rows

function(doc){ emit(doc._id,doc); }

The reduce function is where the actual programming comes in… And it seems there aren’t many well explained examples of exactly how to do this (I just brute forced it by trial and error)

function(key, values, rereduce) { 
    var output = {};
    if ( rereduce ) { 
        // key is null, and values are values returned by previous calls
	//
	// see http://wiki.apache.org/couchdb/Introduction_to_CouchDB_views
	//
	// essentially we are taking the previously reduced view, and the 
	// reduced view for new records, and we are reducing those two things
	// together.  Summarizing two summaries, essentially
        for ( var i in values ) {
	    // here we have multiple prebuilt output objects and we're simply combining them
  	    // just like below we have an array with a numeric id and an output object
	    // 
	    // retrieve a summary
            var vals = values[i];
            for ( var key in vals ) {
		// debugging
                // log(key);
		// 
		// store in or increment our new output object 
                if ( output[key] == undefined )
                    output[key] = vals[key];
                else
                    output[key] = output[key] + vals[key];
            }
        }
    } else {
        // key is an array, which we dont care about, and values are the 
	// values returned by the map
	//
	// see http://wiki.apache.org/couchdb/Introduction_to_CouchDB_views
	//
	// we are taking each document and processing that, reducing it down
	// to a summary object (output) for each of the rows passed
        for ( var i in values ) {
	    // we have an array, values, with numeric ids and a document objects
	    //
	    // retrieve a document
            var doc = values[i];
	    // get what we want from it, the first char of the md5
            var key = doc._id.substr(0, 1);
	    // debugging
            // log( key + " :: " + doc._id );
	    //
	    // store or increment the output object
            if ( output[key] !== undefined )
                output[key] = output[key] + 1;
            else
                output[key] = 1;
        }
    }
    // done
    return output;
}

and in code, using a temporary view, ( if you used this view all the time you would want to make it permanent… but this is about how to lay out a reduce function, nothing more ) so request code that looks like this

$view = array(
    'map' => 'function(doc){ emit(doc._id,doc); }',
    'reduce' => '
        function(key, values, rereduce) { 
            var output = {};
            if ( rereduce ) { 
                // key is null, and values are values returned by previous calls
                for ( var i in values ) {
                    var vals = values[i];
                    for ( var key in vals ) {
                        // log(key);
                        if ( output[key] == undefined )
                            output[key] = vals[key];
                        else
                            output[key] = output[key] + vals[key];
                    }
                }
            } else {
                // key is an array, which we dont care about, and values are the values returneb by the map
                for ( var i in values ) {
                    var doc = values[i];
                    var key = doc._id.substr(0, 1);
                    // log( key + " :: " + doc._id );
                    if ( output[key] !== undefined )
                        output[key] = output[key] + 1;
                    else
                        output[key] = 1;
                }
            }
            return output;
        }
    '
    );
$result = $couchdb->send('/_temp_view', 'POST', json_encode($view) );
print_r($result->getBody(true));

would give you output that looks like this:

stdClass Object
(
    [rows] => Array
        (
            [0] => stdClass Object
                (
                    [key] => 
                    [value] => stdClass Object
                        (
                            [0] => 15
                            [1] => 17
                            [2] => 16
                            [3] => 13
                            [4] => 27
                            [5] => 18
                            [6] => 26
                            [7] => 15
                            [8] => 18
                            [9] => 21
                            [a] => 12
                            [b] => 23
                            [c] => 20
                            [d] => 27
                            [e] => 28
                            [f] => 26
                        )

                )

        )

)

I hope this helps somebody out.

random php… a multi-channel chat rooom class using memcached for persistence

why? i dunno… just because… just a toy…

no sql, no flat file, no write permissions required anywhere, no fuss

class mc_chat {

        var $chan = null;
        var $mc = null;
        var $ret = 5;

        function __construct($memcached, $channel, $retention=5) {
                $this->mc = $memcached;
                $this->chan = $channel;
                $this->ret = $retention;
        }

        function messages( $from=0 ) {
                $max = (int)$this->mc->get("$this->chan:max:posted");
                $min = (int)$this->mc->get("$this->chan:min:posted");
                $messages = array();
                for ( $i=$min; $i< =$max; $i++ ) {
                        if ( $i < $from )
                                continue;
                        $m = $this->get($i);
                        if ( $m['user'] && $m['message'] )
                                $messages[$i] = $m;
                }
                return $messages;
        }

        function get($id) {
                return array(
                        'user' =>(string)$this->mc->get("$this->chan:msg:$id:user"),
                        'message' => (string)$this->mc->get("$this->chan:msg:$id"),
                );
        }

        function add($user, $message) {
                $id = (int)$this->mc->increment("$this->chan:max:posted");
                if ( !$id ) {
                        $id=1;
                        $this->mc->set("$this->chan:max:posted", 1);
                }
                $this->mc->set("$this->chan:msg:$id:user", (string)$user);
                $this->mc->set("$this->chan:msg:$id", (string)$message);
                if ( $id >= $this->ret ) {
                        if ( !$this->mc->increment("$this->chan:min:posted") )
                                $this->mc->set("$this->chan:min:posted", 1);
                }

        }

}

$mc = new Memcache;
$mc->connect('localhost', 11211);
$keep_messages = 10;
$chatter_id = 1;
$chat = new mc_chat($mc, 'chat-room-id', $keep_messages);
$chat->add($chatter_id, date("r").": $chatter_id : foo");
$chat->messages(37); // messages only above id=37
$chat->messages(); // all the latest messages

Debian Lenny, Avahi, AFP… Linux Fileserver for OSX Clients

If you’re like me you have an OSX computer or 3 at home, and a debian file server. If you’re like me you hate samba/nfs on principle and want your debian server to show up in finder.  If you’re like me you arent using debian 3 which is what most of the walkthroughs seem to expect…  This is how I did it… With Debian Lenny.

What we’re using, and why:

  • Avahi handles zeroconf (making it show up in finder) (most howtos involve howl which is no longer in apt)
  • netatalk has afpd
  • afpd is the fileserver

From: http://blog.damontimm.com/how-to-install-netatalk-afp-on-ubuntu-with-encrypted-authentication/

  • apt-get update
  • mkdir -p ~/src/netatalk
  • cd ~/src/netatalk
  • apt-get install cracklib2-dev libssl-dev
  • apt-get source netatalk
  • apt-get build-dep netatalk
  • cd netatalk-2.0.3

From: http://www.sharedknowhow.com/2008/05/installing-netatalk-under-centos-5-with-leopard-support/

  • vim bin/cnid/cnid_index.c ## replace “ret = db->stat(db, &sp, 0);” with “ret = db->stat(db, NULL, &sp, 0);” line 277
  • vim etc/cnid_dbd/dbif.c ## replace “ret = db->stat(db, &sp, 0);” with “ret = db->stat(db, NULL, &sp, 0);” line 517

Mine

  • ./configure –prefix=/usr/local/netatalk
  • make
  • make install
  • vim /etc/rc.local ## add “/usr/local/netatalk/sbin/afpd”
  • /usr/local/netatalk/sbin/afpd

From: http://www.disgruntled-dutch.com/2007/general/how-to-get-your-linux-based-afp-server-to-show-up-correctly-in-leopards-new-finder

  • apt-get install avahi-daemon
  • vim /etc/nsswitch.conf ## make the hosts line read “hosts: files dns mdns4”
  • cd /etc/avahi/services
  • wget http://www.disgruntled-dutch.com/media/afpd.service
  • /etc/init.d/avahi-daemon restart

in case that file drops off the face of the net, this is its contents (except “< ?” is “<?” and “< !” is “<!”) :

< ?xml version="1.0" standalone='no'?>
< !DOCTYPE service-group SYSTEM "avahi-service.dtd">

%h

_afpovertcp._tcp
548


At this point your server should show up under the network in your finder… and you should be able to connect with any system username/pw combo