Let me make a few of my own “The internet during 2007” predictions.

• 2007 will be the year of the format wars. Differing schemas of XML will battle it out for the spot as the predominant method of sharing data in 2007. Because one size will never fit all we’ll probably end up with 2 or 3 schema layouts, varying in complexity and power. Almost nobody will end up using 2 of the 3 😀
• 2007 will usher in the client side scripting wars. Will we still be using a J for AJAX after 2007? Probably, But I bet there will be some headway in finding a more modern, less quirky-by-vendor web scripting language. Something will do for client side scripting what PHP and Ruby have done for server side scripting.
• We’ll see real action in the database-as-a-service mindshare. I’d expect Amazon and Google to weigh in on this action. Microsoft will likely sit out, though it would be a very stupid idea. If Microsoft provided a RESTable and SOAPable database service at a decent cost, they’d soon find themselves up to their ears in just the kind of data that an internet presence should covet! Specifically, though, we’ll see work in 2 areas easy databasing (think simple one to one and one to many relationships, like tags, terms, definitions, etc) and relational databases (think many-many, foreign key, transactions)
• 2007 will see more wasted bits and bytes than gas — with all the uncompressed data interchange formats, and spam, flying around we’ll be wasting vast amounts of resources on the parts of consumable data that isnt really consumable. We probably wont see an answer to this in 2007, or if we do it wont be realized for some time to come.
• Someone will attempt to retrofit e-mail. They wont succeed even though everybody is pulling for them to succeed.
• New phones will be developed and released which have better web 2.0 support. Because after 2006 ends it’s not web2.0 anymore… it’s the web! (at least I’m hoping this is true)
• We’ll see a large number of “old dog” programmers moving from the “hot and hip” web space to the mobile space. There’s such a generation gap between modern browsers mobile browsers that the progression will be pretty natural for those who dont feel like learning “new tricks”
• People will continue to play with new ideas of making the internet social, That wont likely fall off, but what I suspect we’ll see are ways of making the internet more manifested. ether in facilitating physical meetings or actions, or in making a web presence manifest as a physical presence.
• More real world data will be mapped into databases available for processing next year than ever before, from surveys to spatial analysis to trendy places to hang out. We’ll be moving closer and closer to making virtual space analogous to physical space. You wont have to walk to the corner store’s web site (where would the romance in THAT be?) but you can bet we’ll be coming closer and closer to your next door neighbors kids lemonade stand having a web presence.
• Last but not least privacy will slip farther and farther towards unattainable. With so many vectors, so many reasons, so many locations in which one telling piece of information is being stored online, being invisible will be nearly impossible, and staying that way doubly so. But as we slip into a mode where identify is completely fabricatable… what, then does the theft of that identity mean?

I have X number of data objects with up to about min 45,000, max 50,000 possible values associated with each object. I know… Don’t ask (couldn’t tell you anyways). Now to be able to do this in MySQL is… well… possible… but absurd. I’m thinking of trying out the approach I’ve mused about here. It could possibly be a really great way to manage finding commonalities across tens of thousands of objects with a total of hundreds of millions of values. Or it could be a massive time sink.

It would also put some of the things that Amazon has said about their S3 service to the test 🙂 I doubt anyone’s really stored a hundred million objects in an S3 bucket and been concerned enough with seek time to be critical about it 🙂

Or am I missing some magic bullet here. Is there a (free) DBMS I’m not thinking of which handles 50,000 columns in a table with a fast comparitive lookup? (select pk from table where v42000 = (select v42000 from table where pk = referencePk))… I’d love for someone to pop in and say “HEY, STUPID, DB XXX CAN DO THAT!” 🙂 assuming DB XXX isnt expensive 🙂

Hmm… Off to ponder…

Because its not readily apparent when you’re concentrating on learning rails… I’m posting this here. While trying to get rails running on Fedora Core 6 (FC6) I was running into not being able to compile the ruby fcgi gem.

What gives, right? mod_fcgid (what FC6 comes with) is supposed to be binary compatible with mod_fastcgi. And there *IS* no mod_fastcgi or fastcgi in yum! Well it turns out that

mod_fcgid and mod_fastcgi both connect to fastcgi. That bears repeating. mod_fcgid is roughly equivalent to mod_fastcgi but niether are equivalent to fastcgi. (which in retrospect seems obvious as so many things do when you’re searching for answers)

so… on FC6, install mod_fcgid. then download and install fastcgi (not mod_fastcgi) from http://www.fastcgi.com/dist/ and then

gem install fcgi --source \
  http://rubyforge.planetargon.com/gems.rubyforge.org/ \
  -- \
  --with-fcgi-include=/usr/local/include \
  --with-fcgi-lib=/usr/local/lib

will work and everything will be peachy-keen

since a lot of people will be searching about this in a specific scope for Fedora Core 6, and since nobody seems to directly state this issue in plain terms, here it is. Google, eat me up! 😉

Should you or shouldn’t you use Amazons EC2 service? If you believe everything you read without bothering to send it through the hype filter the answer is invariably YES! But in reality (where most of the rest of us live) the question actually depends on a lot of different factors.

The, theoretically, most straight forward of those factors is raw cash. Lets say you were considering purchasing a low end server from, say, Servpath. You’re looking at $1999 per year for a server with a better CPU, but less RAM and HDD space. Versus (365 * 24 * 0.10 = ) $876 with an EC2 instance. Easy, right?

AHH but the devil is in the details. you get 0 (zero, none, nada, zilch, less than any) bandwidth with your Ec2 instance. Thats extra. Lets assume you use all 1500Gb per month you get with your low end server, the price at servpath is still $1999 for the year. The price at EC2 goes up by (1500 * 12 * 0.20 = ) $3600. Even if you pay monthly instead of yearly at servpath its only ($199 * 12 = ) $2388.

Of course if you use significantly less bandwidth… say… 300GB per month…

Servpath: $1999,
Amazon: $876 + (300 * 12 * 0.20 = ) $720 = $1596

But the real complexity comes when you worry about scaling, or handling peak versus off times. And further complexity when you start talking high availability, and load balancing. And yet further if you host static and dynamic content in different places, etc.

The bottom line is that all of these blog posts I see about the choice between traditional versus new-age hosting is not as simple as it is made out to be. Before you jump on the buzzword bandwagon you should really make sure that you’re investing in what makes *sense* for your business and not in what made sense for someone else.

Right about now you’re scratching your head and wondering whether I’m telling you TO use EC2 or NOT TO use EC2. And you’re reading this whole thing wrong (if thats what you’re wondering.) What I am telling you is that EC2 is an extremely flexible and versatile tool which has a huge possible number of advantageous scenarios into which it fills a void previously delegated to the realm of “there’s no solution other than to spend more money or not. period.” You use the sed utility when it makes sense, right? You wouldn’t, for example, attempt to use it to accept http uploads? Of course not that doesn’t even make sense! Well not all situations make sense for EC2 either.

EC2 is a lot like the OSS movement. The up side is that It gives people the power of flexibility and choice. But much like the “Is linux desktop ready?” debate thats been raging on for years you have to deal with the downside which is that it gives people the power of flexibility and choice. Double edged blade indeed. But well worth the risks… If you can learn to wield it properly!

Thoroughly confused yet? That isn’t even the half of it! 🙂

So my little brother had a job working for a storage company. He had only held the job for about 2 weeks, and last Wed he was fired (no reason given). They failed to have his check ready that day; furthermore they failed to have gotten it to him today. They heartily promise to have it ready for him tomorrow morning. My wife and I are going to go down their with him to pick it up. Seems the California labor code was broken in several respects.

Apparently, according to sections 201 and 227.3 of the labor code, they were legally obligated to have his last check on hand and give it to him before he walked out the door. Which they failed to do.

Furthermore they’ve not given him his required last paycheck for a full 7 days (Nov 15 – 21.) So we looked into the labor code and found this little gem: “An employer who willfully fails to pay any wages due a terminated employee (discharged or quit) in the prescribed time frame may be assessed a waiting time penalty. The waiting time penalty is an amount equal to the employees daily rate of pay for each day the wages remain unpaid, up to a maximum of 30 calendar days.”

“Thanks” to the great state of California for making this information so clear and concisely available here:

http://www.dir.ca.gov/dlse/FAQ_Paydays.htm

So, 2 weeks owed wages at about 30 hours per week at about $8.50 per hour?

$433 after taxes

Failing to pay the above amount for 7 days?

$476

The “new girl” breaking the law,
Costing your company 2 times what you should have had to pay,
And realizing that this particular ex employee and friends are, unfortunately, not to be the idiots you’ve been hoping they would be?

Priceless

Thank you http://www.starryhope.com/tech/2006/mac-os-x-home-and-end-keys/!

I was getting the following error when running “/usr/bin/sudo -u vpopmail -H /usr/bin/spamassassin -D bayes –lint”, per the instructions:

[6932] info: rules: meta test DIGEST_MULTIPLE 
    has undefined dependency 'RAZOR2_CHECK'
[6932] info: rules: meta test DIGEST_MULTIPLE 
    has undefined dependency 'DCC_CHECK'

So, after some digging around I wandered over to /usr/share/spamassassin/20_net_tests.cf and set “meta DIGEST_MULTIPLE” to “PYZOR_CHECK > 1” which seems to have fixed the problem (and knowing my luck and the amount of effort I put into this, which is somewhere near 0 it’ll come back someday to bite me in the butt (thus the blog post)) I was then able to run the following

/usr/bin/sudo -u vpopmail -H \
    /usr/bin/spamassassin -D bayes --lint
/usr/bin/sudo -u vpopmail -H \
    /usr/bin/sa-learn -u vpopmail --force-expire

And to add the following to roots crontab

0 0 * * * /usr/bin/sudo -u vpopmail -H \
    /usr/bin/sa-learn -u vpopmail --force-expire 1>/dev/null

I then reversed what I did here: http://blog.apokalyptik.com/?p=145 (well the spam part, I still have clam=no)

Here’s to hoping.

And regardless of whether it works or not, thanks to Shubes for the comment!