So I had to help fix an internal issue on WordPress.com where some largsish bits of data are transferred around via XMLRPC. We like XMLRPC, its simple, extensible, and works. But one thing that the IXR_Server is not is ‘light’. Because of the way that it works all of the data is transferred base64_encoded which means that a 10mb attachment just grows and grows and grows in ram on the server side… Which is exactly the problem I was faced with. I spent about a full day pouring through the code and its current and peak memory usage at various places in the flow of execution and put together a patch to help address some of the most glaring memory hogs.

I asked Joseph to go over it and submit the good stuff upstream to the wordpress.org project (he’s had a lot more experience working with the actual xmlrpc clients than I do, so his experience there in making sure that they wouldn’t be disturbed by my changes was very valuable.)

The takeaway from this is pretty simple, but at the same time quite profound. When working with arbitrary data inputs that you expect to be large you really have to watch the copy-by-value operations that you do.  Referencing is one of those things that you may not need to know in detail for every day coding, but its good to be aware of them and have an idea of what they are/do.  In cases where you find yourself inexplicably running out of RAM a couple of references can save the day.

Related/restated take-aways are:

• The idea of balacing two variables in ram by adding X bytes to one and then subtracting those xbytes from the other. (xml parsing)
• Also that certain types of operations create a temporary variable in ram with which to work with your data, so limiting the amount of data they need to function can be helpful when the variable is of considerable size (preg_replace)

All in all this should be a considerable win for WordPress (dot com and dot org) xmlrpc users.  If you’re running a self-hosted wordpress install and want to test it out and comment on the patch i’m sure everyone would be greatful.

DK