Author: apokalyptik

apokalyptikhttp://blog.apokalyptik.com/

Interpreted language modules are not the sysadmins responsibility

Ok, you perl people are aweful about this… Us PHP people are a close second…

Hold on, rewind, let me begin again.

There are two kinds of modules which attach to scripting languages.  Compiled, and interpreted.

A compiled language module is something written in another language (commonly C, C++, or ASSEMBLY) which “plugs in” to the scripting language to give you, as the scripter, new functionality.  The MySQL database connectors are a good example of this.  MySQL provides an interface to the version of the MySQL server currently installed via a C based API.  PHP and PERL both have modules which “wrap” this API up and provide us with neat clean little functions to work with in our npreferred environment.  You would do something like this for a couple of reasons.  Most commonly the reason is performance: It’s simply faster to do some things via certain languages (which might not be the one(s) that you prefer to work in.)  Second is availability.  If something is particularly complicated *or* not publicly available, you would build a wrapper around a vendors API just for the convenience of not having to write your own.

An interpreted module (or library) is written in the language that you’re using.  It’s simply there to provide you with more complicated functions that have already been written (many of the CPAN and all of the PEAR modules fall into this catagory.)  This is *just* like you writing a function in your language and then re-using it elsewhere. There’s no real magic here except that you didnt have to do something yourself.

Compiled modules carry with them the burden that all compiled programs share… including machine architecture and software environment dependencies.  What this means is that a module compiled on machine A will only run on machine B if they are set up in the same way with the same hardware.  If they *are* all the same, then this rant applies to compiled modules as well.. if the environment is *not* the same (because of a bad sysadmin, poor funding, or legacy hard/software) then compiled modules *should* fall into the domain of the sysadmin.

Because interpreted modules are like the libraries you write yourself and reuse over and over (and if you dont then you need to think long and hard about your development model, because its not as efficient as it should be) they can be installed in two ways: globally, and locally.

Globally installing a software module is, basically, just putting it into a place where any piece of software running that language can simply include the file by calling the proper command, and code with the provided functions.

Locally installing the module is putting the module itself into your application and calling it via an include in the same way you call any other library which you’ve already written.

Now let me say that if you A) have a development team, B) have multiple servers, and C) have a complicated environment requiring lots of modules for your web applications to work properly you have no excuse for not having some sort of version control system which gives you a central place from which to check out and update the code which runs on the servers.

Ideally updating the application across X number of web servers would be something as simple as “svn up”.  Period. No, really, thats the end of it.  If you limit your module use to software modules, and install your software modules into the application (as part of your repository) you have complete control over your web environment!  No more “damn sysadmin… if *I* were in control of these servers dot dot dot (…)”  Why arent you in control of your web servers? If you write your applications properly the only thing you should need is a server with the base tools you can be in crontrol.  That way you can leave the admin to do the things he’s supposed to be doing: security work, building you more servers, and generally improving the stability of the platform on which you run your apps! But the benifit is not just for the sysadmin!  If you find that you need the latest X+ 0.00000002549 version of module XYZ you dont have to wait for anybody to install it for you. just pop it in, and update your repo, and distribute. Ahh, scalability, and control!

When the development team works with the ops team something happens that truly deserves a buzzword like “synergy”… when they work at odds against eachother… well… Who wins in that case?  Good question.  Go ponder it.

Cheers

DK

Google Version Control

http://code.google.com/

Not at all a suprise offering from Stein 😀 I’m sure it will be top notch. (remember it’s still beta right now, of course it isnt feature complete!)

What this could give google is a distinct in-road to the emerging generation of web and application developers. What better way to know who to hire for a programming position than to have their entire development history available at a moments notice. Will this be the beginning of google knocking on the door of candidates that THEY want, rather than candidates seeking google out?

hmm.

6 tips for surviving in online communities

Tip #1: Be self assured. If people questioning your point of view frightens or upsets you
, then you’re probably not ready to participate in an open membership (and especially anonymous) online community. Just keep in mind that you have your points, and whether someone agrees with you or not is simply not important.

Tip #2: Online is not Offline. What someone says online is likely not what someone would have said offline. If you find yourself thinking something along the lines of “I dare you to come down here and say that to my face.” You obviously need to remember that online bullies are a lot like the bullies that you faced in third grade. They talk a big game, but in the grand scheme of things they just arent that important.

Tip #3: Patience. Not everybody is sitting at their PC’s hitting refresh waiting for you to have said something to which they can reply. Give it some time. This obviously changes somewhat depending on the medium… IRC might be a one to fifteen minute delay, where a message forum could be a day, and a mailing list might take several days.

Tip #4: Dont beat a dead horse. This is the tip that I’m, personally, most guilty of infringing upon. Whether people agree with you or not… once you have your opinion out it’s really not worth repeating it over and over and over and over and over… especially in the same threads of conversation. Now if you’re actively engaging in a debate this is less important than when answering a question. But even when debating you should always attempt to spin the same facts in different lights if you *have* to reiterate them.

Tip #5: Put on your fire suite. This tip is an extension or melding of tips 1 and 2. You really have to remember that this stuff is not life or death… It simply doesnt matter. Really. It doesnt. People are eventually going to disagree with you, and thats OK. People are eventually going to berate you, chastize you, mock you, put you down, and generally act like asshats. Let them. At the end of the day they just dont matter!

Tip #6: You dont have to. You are not required to do anything you dont want to. If you find yourself answering the same qquestion again and again… and its making you angry. stop. It’s really simple. Someone else will fill in (or not) but no matter how much you feel that it is… it *isnt* specifically your responsibility. And if you just cant help yourself then write up a FAQ, and simply point to it, then leave it at that. It’s a lot better than blowing a gasket after answering the same “newbie” question 1,500 times in a row.

The Failure of Modern Package Management

Package management, in all its necessary infamy/glory, is a joke. The problem with designing a package management system is the various conflicting needs that different groups of people have.

Group 1: “I write documents, use e-mail, and surf the web.”

This group wants stability! They have no desire to be on the bleeding edge, because the bleeding edge is plagued with blood, and blood frightens this group. “We just want it to work like it always has” is their motto, and the only time they want to install something is when they cant view a particular web page. On this group the acceptability of spending hours and hours managing and updating their packages is a frustrating ordeal! Lets face it, unless its going to cause my computer to bleed silicon out the floppy drive I dont really care what updates it needs… A lot of the time we techies try and address these people in a “protect them from themselves” methodology. We writeup our “foolproof” “do this once a week task list. But this fails the first time that a person forgets to do it (leading to forgetting the next time, and so on) or, worse, when something in the process changes and they can no longer follow the directions.

Usually, besides a basic firewall and a virus scanner what they need protection from is their son/daughter/neice/nephe… If I had a dime for every time I heard “my {insert relative or friend here} came over and installed a bunch of stuff and now it doesnt work. {person} is really good with computers, and I’m not.” The rub is that their very own statement is self contradictory… the person who is good with computers… broke yours? yea… uh huh… stop letting little jimmy use the damn thing.

So this group is plagued by a constant stream of changes to their routines which leaves them frustrated, vulnerable, and afraid (though not many of them are sure enough of themselves to admit it… they’re afraid) that the next time they update their packages they’ll have to spend months figuring out how to work around the new changes to do the things that they’ve been doing for years… in the same way they’ve been doing them…

Group 2: “have you played {insert new video game title here}”

This group’s motto is something along the lines of “hold on, I need to reboot again.” These folk are constantly in search of bigger, better, newer, and faster. They’re the ones paying $700 for a video card preorder months in advance. This is the group which is constantly plagued with (because of their 0-day hardware / software nature) has grown accustomed to saying things like “when my video card manufacturer releases the next version of the driver it’ll fix that incompatibility between the $60 game I just bought and the $700 video card I just bought… They say it should only be 2 weeks… I’m stoked, cant wait. Until then I’ll be in {some other game} getting my frag fix there”

Ironically this is the group which is most at ease with modern package management… It’s been their way of life so long… They just accept it. The adrenalyn junkies put up with enough crap in one month of using their computer to make Group 1 want to throw theirs off a twelve story building. But the fact that they can no longer see the foreest for the trees doesnt mean they arent being stepped all over. I’m not sure when it became OK to use your most loyal (and profitable) customers as guinnea pigs (instead of, you know, real-grown-up-big-kid-quality-control) but thats whats happened. It’s suddenly a good thing to ship a product before its ready… because someone else will tell you whats wrong with it… nice.

Group 3: “One hundred and ten more workstations to go”

This is the group who, probably, most loathes the daily, weekly, monthly required updates to products… Oh, they understand that theres a new security hole and it’s got to be fixed… thats a given in corperate {insert country here}… what gets their goat is that they they cant just upgrade whats broken… nope… lots of new features, functionality and dependencies means every time they fix a flaw they leave another feature for Lucy and Bob in accounting to start playing with (which will naturally leave a gigantic hole in their filesystem… or worse.) You’d be amazed how one small change to the word processing program… or spreadsheet program… can wreak havock across an entire business… but the little changes which cause the occasional all-nighter with the tape backup unit are nothing compared to those dreaded words “end of life”… those words cause months of lost sleep due to relentless nightmares.. The fact that corperate {country} actively weighs the pros and cons of sticking with a set of software which is no longer supported by anyone tells you, without any room for misinterpretation, that something is desperately wrong with package (and os) management in this day and age…

Group 4: “The environment I need to run this is pretty custom”

This is the group who’s motto is more or less “I have to get things done, and sometimes I have to go to extremes to do it” They have some pretty odd requirements to operate effectively. Maybe they’re maintaining legacy systems, or perhaps they’re developing their own and have found that the only wat to do X is with a special case Y. Whatever it is “upgrade everything to upgrade anything” is simply NOT an option. And realistically it shouldnt have to be! If you, for some reason simply have to use an older version of, say, glibc… and the absolute latest version of some other package… you’re stuck in maintenence hell… Your only option is to put a lot of manual effort into maintaining your packages… And if you have to do this in a server farm… Well… May a higher power help you!

So what is the answer?

Like so many things in life this is the point where we have to think about 2 things. First the answers might already be out there, and second how benificial is reinventing a particular wheel?

What’s already out there…..

Binary distribution: windows .exe files, RPM, DEB, TGZ… these work but all have dependencies! and anyone who’s managed a redhat system knows what dependency hell can be like (tell me why, again, I need a printer daemon on my web server?). There are also disadvantages to the producer… if your software comes with a lot of optional functionality, or runs on multiple platforms you have to support and maintain and distribute multiple packages to satisfy your user base. Also these kinds of packages tend to require an OS reboot for anything halfway important…. And before you Linux people give me any grief I know at least a few of you out there have done something stupid like a batch software update and didnt realize something critical was replaced which required at least some manual recovery after the first reboot (what… a month from the update?!) which is MORE dangerous that an immediate reboot… at least with an immediate reboot you have a good idea of which packages caused the problems…. but if you’ve updated once per week for 8 weeks and then rebooted because you moved your pc across the room… which package was it, exactly, that is preventing it from coming back up? Now… would your grandma be able to handle this as well as you have? no..

Compile from source: Ahh yes, Gentoo, BSD, Slackware users shine here (And before you start hating, or those of you who know me start nagging I *AM* a Gentoo/BSD person, so this is telling myself off a bit here too) You, the master of your destiny, can build your OS to fit any necessary use! Of course slackware is also akin to having to carve your own little houses out of plastic before you can play manopoly… BSD suffers from (compared to windows, and Linux) a lack of choice. And Gentoo suffers from the need to be bleeding edge (but… just because this ebuild says it needs the latest GCC doesnt mean it wont compile without it… grrr.) If you’re good with a couple scripting languages and have some patience and ingenuity you can thrive in this environment… but… mom wont…

Binary patching… *cough*

The answer, I think, lies in a paradigm shift brought on by server side web applications. Long gone are the days of bulky CGI compiled from something like C or C++ source code. Web developers have realized the potential of interpreted languages and they have set the web free. Now a days, thanks to interpreted languages like PHP, Perl, and Python the internet you know and love is a fast paced, dynamic, carnival of new sites and sounds! The reason that things have progressed more quickly with these languages as opposed to the compiled ones?

They must be faster right? Nope! Compiled code usually runs quite a bit faster than a script does…

They’re more powerful? Not really… in the right hands C/C++ or the likes can absolutely dazzle you and show you that nothing is beyond the true master programmers reach.

So… More portable? Thats half of it! When you write a PHP, Perl, or Python application theres a pretty darn good chance that the code you wrote will work well no matter where you run it… you’ll get the same results all the time. Add GTK to the mix and you have a scripting language capable of being a real client side application platform…

And… Easier to develop? You guessed it! The beauty of debugging and adding features to an interpreted language is the ability to develop one piece at a time withtout all that nasty muching about in the compiler, linking, etc, etc… just run the code… tweak, run, tweak… The old model was tweak, compile, wait, wait, (sometimes wait a LOT longer) run, tweak (repeat) Naturally removing the com[piling and waiting has sped up development quite a lot

So it’s my humble opinion that the empty promise that is package management will necessarily slip into obscurity, and the reign of interpreted code for general consumption will begin to be more and more commonplace. Todays fast machines, and engine side compiler caches, make thefeasibility of this in regards to the end users machine possible (where it was not realistic on your 286 to compile code each time it was run, your P4 3Ghz with 2Gb RAM is readily capable.) It will probably soon be very commonplace to have a php4, php5, php6 perl5, python22, and a python23 interpreter on your system and your applications from day to day will be run and maintained in much the way of the modern web application…

Quote (From my Brain) of the Day

“Think late night, cartographic software enginners gathered for black mass followed by a lovely breakfast and an optional voodoo doll creation and bake sale event”

I know I havent posted in a while. But lets face it. I’m no A-List blogger, and I really doubt you’re reading me anyways… ok… maybe *you* but nobody else, really (hey not everyones as masochistic as you)

Cheers

–Apok

It’s been a long freakin while now.

  • I was getting ALL good at posting to my blog for a while and then *poof* nothing… Well I have some updates for you.
  • Apache (my cat not my web server) has recovered enough to go outside and prance around and be her normal bubbly self again. Which is nice..
  • Buddy is growing like a freaking weed… he’s over 65 lbs now! And he’s entering hisadolescence (which, of course, means he isnt listening for crap)
  • Ookles is coming along nicely. We’ve tacked on the foundations for all the really kickass features to come. Theres very very little left to do under the hood before we can really start with the racing stripes… Hopefuly more on that soon.
  • I’ve been spending a lot of time lately in JavaScript (not my favorite place) but I have to say that the YUI library is making it much more pleasant than it used to be… Look for a good smattering of web++ goodness in ookles when it comes out and then for it to grow like a virus
  • I’m really liking Mac and OSX, but after extensiveuse mail.app sucks really bad for me. Probably due to the use of Imap with truly huge message stores, but so far thunderbird has it beat hands down (so far == 6 hours now) I’ve also tried out camino for the first time. no impression yet tho..
  • I’ve also been paying attention to a dog specific forum lately. It’s been a long time since I participated in a Forum… Deja Vu all over again! It’s funny, though, that after X years (where X is a bunch) the technology really hasnt gone anywhere… it’s still precisely what it was way back when with prettier graphics (sometimes)

You can lead a horse to water but you cant make him think

(note: to the reader who’s commented to me about the cat… this has nothing to do with you ;D promise!)

I keep running into the same basic wall when dealing with people: I am not a good teacher. I’m not. Period. There are many possible reasons, both internal and external, why this is. The bottom line, though, is that I cannot teach people my thought process. It never. EVER works. I’ve come to the conclusion that I don’t see the world from the same angle as all the rest of the people I know. My brain seems to process the input altogether differently.

This is not, at all, to say that the way my brain processes input from the world is better. In fact there are a great many times when I’m a blithering idiot when it comes to simple things (simple for other people.) For example I don’t remember things well unless there is some sort of trigger which my brain associates with the memory. A great example of this is names and times. I remember faces, but not names. I remember events, but not times (up to and including todays date.)

What I do well, though, is (apparently) whatpeople might call “abstract thinking.” I see a problem (note: I dont usually think of things as “problems”, really, it’s just something that needs to be worked out. there’s none of the negative connotation that I associate with the word “problem” when something can be worked out/through,) I take stock of my resources, and I deal with it if it can be dealt with, or I dont.

It doesnt bother me to try something N times before getting it right. I just do it. It doesnt bother me to do a highly tedious task when it has to be done. It has to be done. And it doesnt bother me to deal with that I’m given and figure it out.

And therin, i think, lies the difference. Most people expect to be handed something that solves all their problems. And it never happens that way. Most people have unstated expectations which are rarely met. I’m happy to get half the answer to a problem and work the other half out. To me thats an intriguing process. It’s fun and rewarding. To most people, I think, its work.

Thats the crux of the problem. Thinking is work. And people dont like work, especially that kind of work. So I cringe when people ask me to teach them something… because they usually dont want to be taught what I know, they want a dictionary of situation and response. I dont have one of those. I’ve never needed one. I can search, I can work, I can evaluate, but most importantly: I can think.

And unfortunately… I can’t teach that…