A class for normalizing mixed value containers

There are times (like when dealing with simplexml) when you just wish you had an array that you can iterate over for whatever your reasons are (especially when dealing with variable structured multidimensional unpredictable input.) This is also a good example of recursive functions, simplification of difficult specialized problems, and how one might use a class to accomplish large tasks in an encapsulated fashion.
[coolcode]

/**
* A class for [N]ormalizing a [C]ontainer to an array
*
* This will class will take any type of variable container and return a normalized array
* For example this will take the output of simplexml_load_string and render it all into
* a multidimensional array.
*
*
*/

/**
*
*/

class nc2array {

private $original_container;
private $normalized_container;

/*
* Takes a variable, and builds a normalized comtainer out of it
*/
function __construct($container) {
$this->original_container=$container;
if ( is_object($container) ) {
$this->normalized_container=$this->recursive_parse_object($container);
} elseif ( is_array($container) ) {
$this->normalized_container=$this->recursive_parse_array($container);
} else {
$this->normalized_container[]=$container;
}
return($this->normalized_container);
}

/*
* takes an array and parses it recursively, passing objects off to recursive_parse_object
*/
function recursive_parse_array($array) {
foreach ( $array as $idx => $val ) {
if ( is_array($val) ) {
$rval[$idx]=$this->recursive_parse_array($val);
} elseif( is_object($val) ) {
$rval[$idx]=$this->recursive_parse_object($val);
} else {
$rval[$idx]=$val;
}
}
return($rval);
}

/*
* Takes an object and parses it recursively, passing arrays back off to recursive_parse_array
*/
function recursive_parse_object($obj) {
$vals=get_object_vars($obj);
while (list($idx, $val) = each($vals)) {
if ( is_object($val) ) {
$rval[$idx]=$this->recursive_parse_object($val);
} elseif ( is_array($val) ) {
$rval[$idx]=$this->recursive_parse_array($val);
} else {
$rval[$idx]=$val;
}
}
return($rval);
}
}
?>

[/coolcode]

I’ve become a MySQL snob…

I find that, after keeping alive a database whose size I cant comment on specifically, on a budget that I cant comment on specifically (which would shock you if only you knew), I’ve become aloof to a great many people and their MySQL war stories… 300Mb, 1Gb, 100Gb, 1Tb, HAH! HAH I SAY! Here’s a quarter call someone who’ll be impressed.

Next time you have to move your datacenter across the country (literally coast to coast,) and you cant let your application go down, and you have 14 days to plan and execute, alone, oh and you cant cease taking information in on the old coast until you start can taking it in on the new coast, with over 1Tb of data to move, and replication to deal with… Oh, yea, and you dont get to take a break from running the old datacenter to impliment the new one, you have to do both simultaneously… yea… then come knock on my door.

Till then go take your 2Gb “one server is enough but i have a second server for backup purposes” MySQL (or any brand really) database, and be thankful you arent playing with the big boys. I’ve had the potential to lose more data to a power outage than you’ve got total.

Dude, You’ve been hacked!

While investigating a breakin on a freinds server I came across this pretty damn cool little utility: The RootKit Hunter

It’s a spiffly little piece of work, and it helped considerably. This server, for reasons I can’t say, couldnt be simply wiped and reloaded in a short time span… and HAD to, for a particular service that it performed, stay up for the duration of the week. This little tool helped considerably in cleaning the server out relatively well.

Still, when possible, wipe and reload after a hack attempt is the BEST choice. Cleaning is an imprecise art at best.

Information overload and search vs relevance

Increasingly people are having less of a problem finding the information, and are now having more of a problem managing that information. Often times I find myself frustrated at people who cant find what they want via a search engine, and then a matter of seconds later (with the same search engine) I’ve pulled up a list of highly relevant information for them. The problem isnt that they dont know how to search… the problem is that they dont know how to apply relevance.

As an example… A dog lover who hits the internet for the first time might pop over to google and search for “training bitches” You can imagine what they might find. Any tech savvy person would have searched for “Dog training bitches” It’s amazing what a little bit of relevance will do when used properly! The problem is that most people arent tech savvy, and most people dont think in terms of relevance!

As an engineer, and as an ops guy I’m constantly aware of the relevance of what I’m doing as well as how it affects the neural network of aparati around whatever mode or framework that I am operating inside (be it reality, an application, or google.) My grandmother is not.

Of course when people think search and relevance they think spyware, and understandably so. But would it really be so difficult to build an application which was really able to exploit relevance in such a way as to be easy to even the worst novice? I could imagine several ways, all of which would get me booed off my soap box… and the ones that wouldnt wouldnt please the techophiles…

Maybe thats the problem. The divide between the eleetist users and the novice users is shrinking, causing the real eleetist folk to dig in their heels and resist anything which would help close that divide even faster (especially when privacy is mentioned). But until grandma can use the web just as well as I can… the job isnt done…

Web 2.0? hardly. Think 0.75

Paying it forward… To yourself

I’m a pretty lucky guy. Today, when I told my boss “I need to spend a sizeable chunk of time doing this right (that is re-doing it since we’ve solidified on concepts and semantics since the proof of concept version of this code)” he understood that it was important. He gave me the rest of the day to refactor that code.

It would play to my ego to think that since I’m such a wonderful engineer my word was taken as it should be — as the absolute unabashed unbiased truth. Unfortunately I’m not the golden child of engineers; and fortunately I don’t view myself in that way (nor do I wish to do so.) The reality is that Scott took my advice, weighed it, and gave me the chance to pay it forward.

So, four and a half hours of work today allows for a time savings of 75% any time we want to write a new feature for a particular (and important) area. The truth of the matter is that we may not put in more than one feature there before release, but the company that I work for has turned its mind to planning and true agility.

It’s sad to think that a large percentage of companies would have (at this point: pushing towards an initial release, and tantilizingly close) simply written it off as not immediately important. While it’s true that getting to the goal is hugely important, it’s also true that getting there with a torch that no longer supports a flame really defeats the purpose.

So even as tension mounts with personal and business life colliding (as is oft the case with startups), not to mention business relationships stressed by the sheer exhaustion of effort, Scott still seriously considers my advice and allows a fair amount of freedom in my momentum.

Maybe it’s because hes a macro, as opposed to micro, manager. Maybe it’s because we’ve worked together for so long. Maybe it’s because I’ve never let my employer (him or otherwise) down. Heh… Maybe it was just a good day on the other end of the IM session. Whatever it was… Thanks

DK

Building and using large scale arrays

Here’s a real world example of what I was talking about earlier (using all parts of an array as relevant data) This is a small piece of some blogging related code I wrote earlier this week for Ookles (this is a part of a code package that will be made generally available when it’s ready)
[coolcode]

function get_html_metadata($url) {
/*
Example usage/test code

$test_urls[]=”http://jwz.livejournal.com/”;
$test_urls[]=”http://fuzzyblog.wordpress.com/”;
foreach ( $test_urls as $url ) {
$meta_tags=o_get_meta_tags($url);
foreach ( $meta_tags as $tag_type => $val ) {
echo “Tag Type: {$tag_type}
“;
foreach ( $val as $tag_type_number => $val ) {
echo “\t#{$tag_type_number}
“;
foreach ( $val as $tag_meta_key => $val ) {
echo “\t\t{$tag_meta_key}:\t”;
foreach ( $val as $tag__meta_value ) {
echo “{$tag__meta_value} “;
}
echo ”
“;
}
}
}
}

*/
$data=trim(file_get_contents($url));
$data=str_replace(chr(10), ”, $data);
$data=str_replace(‘/>’, ‘>’, $data);
$data=str_replace(‘>’, ‘>’.chr(10), $data);
$data=explode(chr(10), $data);
$meta_array=array();
// Loop through the different tags matching only tags with an = sign in them
foreach ( $data as $line ) {
if (ereg(‘<([^ ]+)[ ]+(.*=.*)>‘, $line, $r) ) {
$meta_type=$r[1];
$meta_counter[$meta_type]++;
$current_meta_counter=$meta_counter[$meta_type];
$mini_data=str_replace(‘=”‘, ‘=’, $r[2]);
$mini_data=str_replace(“='”, ‘=’,$mini_data);
$mini_data=str_replace(‘”‘, chr(10), $mini_data);
$mini_data=str_replace(“‘”, chr(10), $mini_data);
$mini_data=explode(chr(10), trim($mini_data));
// for debugging: $meta_array[$meta_type][$current_meta_counter][‘original’][]=$r[0];
foreach ( $mini_data as $val ) {
$val=explode(‘=’, trim($val));
$meta_key=$val[0];
$meta_val=$val[1];
// build our array
$meta_array[$meta_type][$current_meta_counter][$meta_key][]=$meta_val;
}
}
}
if ( count($meta_array) > 0 ) {
return($meta_array);
} else {
return(FALSE);
}
}
[/coolcode]

This might be of use to someone

I’ve been writing PHP for a long time now. And one of the things that I see quite often is that people just dont get multidimensional arrays, and how to effectively store data in them for looping. Allow me to lead by example.

  • $store[‘fruit’][‘apple’]=3;
  • $store[‘fruit’][‘orange’]=1;
  • $store[‘berries’][‘blueberries’]=100;
  • $store[‘berries’][‘strawberries’]=100;
  • $store[‘fruit’][‘mango’]=5;
  • foreach ( $store as $type => $val ) {
    • foreach ( $val as $name => $qty ) {
      • echo “We have {$qty} {$name} ({$type} are good for you)rn”;
    • }
  • }

of course where the array comes from and what you do with it are completely optional. But it’s important to know that the array indexes should be as informational as the array values. Otherwise you arent using arrays properly.