CLI

a dumbed down version of wpdb for sqlite

I’ve been working, gradually, on a project using an sqlite3 database (for its convenience) and found myself missing the clean elegance of wpdb… so I implemented it. It was actually really easy to do, and I figured I would throw it up here for anyone else wishing to use it. The functionality that I build this around is obtainable here: http://php-sqlite3.sourceforge.net/pmwiki/pmwiki.php (don’t freak… its in apt…)

With this I can focus on the sql, which is different enough, and not fumble over function names and such… $db = new sqlite_wpdb($dbfile, 3); var_dump($db->get_results(“SELECT * FROM `mytable` LIMIT 5”));

the code is below… and hopefully not too mangled…

< ?php

class sqlite_wpdb {

        var $version = null;
        var $db = null;
        var $result = null;
        var $error = null;

        function sqwpdb($file, $version=3) { 
                return $this->__construct($file, $version); 
        }

        function __construct($file, $version=3) {
                $function = "sqlite{$version}_open";
                if ( !function_exists($function) )
                        return false;
                if ( !file_exists($file) )
                        return false;
                if ( !$this->db = @$function($file) )
                        return false;
                $this->version = $version;
                $this->fquery = "sqlite{$this->version}_query";
                $this->ferror = "sqlite{$this->version}_error";
                $this->farray = "sqlite{$this->version}_fetch_array";
                return $this;
        }

        function escape($string) {
                return str_replace("'", "''", $string);
        }

        function query($query) {
                if ( $this->result = call_user_func($this->fquery, $this->db, $query) )
                        return $this->result;
                $this->error = call_user_func($this->ferror, $this->db);
                return false;
        }

        function array_to_object($array) {
                if ( ! is_array($array) )
                        return $array;

                $object = new stdClass();
                foreach ( $array as $idx => $val ) {
                        $object->$idx = $val;
                }
                return $object;
        }

        function get_results($query) {
                if ( !$this->query($query) )
                        return false;
                $rval = array();
                while ( $row = $this->array_to_object(call_user_func($this->farray, $this->result)) ) {
                        $rval[] = $row;
                }
                return $rval;
        }

        function get_row($query) {
                if ( ! $results = $this->get_results($query) )
                        return false;
                return array_shift($results);
        }

        function get_var($query) {
                return $this->get_val($query);
        }

        function get_val($query) {
                if ( !$row = $this->get_row($query) )
                        return false;
                $row = get_object_vars($row);
                if ( !count($row) )
                        return false;
                return array_shift($row);
        }

        function get_col($query) {
                if ( !$results = $this->get_results($query) )
                        return false;
                $column = array();
                foreach ( $results as $row ) {
                        $row = get_object_vars($row);
                        if ( !count($row) )
                                continue;
                        $column[] = array_shift($row);
                }
                return $column;
        }

}

?>

Postfix, DKIMproxy, Spamc

If you’re running any moderately busy mail server you’re probably using spamassassins spamc/spamd to check for spam because its tons more efficient than piping the mail through the spamassassin cli. Assuming that you do, and that you plan on adding DKIM proxy to the mix to verify  and sign emails you need to put things in the right order, to save you some headache here’s what I did:

  1. smtp|smtps => -o smtpd_proxy_filter=127.0.0.1:10035 # outgoing dkim verify port
  2. 127.0.0.1:10036 => -o content_filter=spamassassin
  3. spamassassin =>  pipe user=nobody argv=/usr/bin/spamc -f -e /usr/sbin/sendmail -oi -f ${sender} ${recipient} # this delivers to the “pickup” service
  4. pickup => -o content_filter=dksign:127.0.0.1:10037 # outgoing dkim signing port
  5. 127.0.0.1:10038 => -o content_filter= # the buck stops here

If you arent careful with these (which I wasnt) you’ll end up causing an infinite loop between your filters (which I did).  Thus concludes our public service announcement.

Writing your own shell in php

I’ve always wanted to write my own simple shell in php.  Call me a glutin for punishment, but it seems like something that a lot of people could use to be able to do… If your web app had a command line interface for various things… like looking up stats, or users, or suspending naughty accounts, or whatever…. wouldnt that be cool and useful?  Talk about geek porn.  Anyways this this morning I got around to tinkering with the idea, and here is what i came up with… It’s rough, and empty, but its REALLY easy to extend and plug into any php application.

apokalyptik:~/phpshell$ ./shell.php

/home/apokalyptik/phpshell > hello

hi there

/home/apokalyptik/phpshell > hello world

hi there world

/home/apokalyptik/phpshell > cd ..

/home/apokalyptik/ > cd phpshell

/home/apokalyptik/phpshell > ls

shell.php

/home/apokalyptik//phpshell > exit

apokalyptik:~/phpshell$ ./shell.php

See the source here: shell.phps

Command line arguments in bash scripts

This is something that has always annoyed me about bash scripts… The fact that it’s difficult to run 

/path/to/script.sh --foo=bar -v -n 10 blah -one='last arg'

So I decided to write up a bash function that let me easily (once the function was complete) access this type of information. And because I like sharing, here it is:

#!/bin/bash
function getopt() {
  var=""
  wantarg=0
  for (( i=1; i< =$#; i+=1 )); do
    lastvar=$var
    var=${!i}
    if [ "$var" = "" ]; then 
        continue 
    fi
    echo \ $var | grep -q -- '='
    if [ $? -eq 0 ]; then
      ## -*param=value
      var=$(echo \ $var | sed -r s/'^[ ]*-*'/''/)
      myvar=${var%=*}
      myval=${var#*=}
      eval "${myvar}"="'$myval'"
    else
      echo \ $var | grep -E -q -- '^[ ]*-'
      if [ $? -eq 0 ]; then
        # -*param$
        var=$(echo \ $var | sed -r s/'^[ ]*-*'/''/)
        eval "${var}"=1
        wantarg=1
      else
        echo \ $var | grep -E -- '^[ ]*-'
        if [ $? -eq 0 ]; then
          # the current one has a dash, so cannot be
          # the argument to the last parameter
          wantarg=0
        fi
        if [ $wantarg -eq 1 ]; then
          # parameter argument
          val=$var
          var=$lastvar
          eval "${var}"="'${val}'"
          wantarg=0
        else
          # parameter
          if [ "${!var}" = "" ]; then
            eval "${var}"=1
          fi
          wantarg=0
        fi
      fi
    fi
  done
}

OIFS=$IFS; IFS=$(echo -e "
"); getopt $@; IFS=$OIFS

now at this point (assuming the above command line parameter and script) I should have access to the following variables: $foo ("bar") $v (1) $n (10) $blah (1) $one ("last arg"), like so:

OIFS=$IFS; IFS=$(echo -e "
"); getopt $@; IFS=$OIFS

echo -e "
foo:\t$foo
v:\t$v
n:\t$n
blah:\t$blah
one:\t$one
"

You might be curious about this line:

OIFS=$IFS; IFS=$(echo -e "
"); getopt $@; IFS=$OIFS

IFS is the variable that tells bash how strings are separated (and mastering its use will go a long way towards enhancing your bash scripting skills.) Anyhow, by default IFS=” ” which normally is OK, but in our case we dont want “last arg” to be two seperate strings, but one. I cannot put the IFS assignment inside the function because by that point bash has already split the variable, it needs to be done at a level of the script in which $@ has not been touched yet. So I store the current IFS variable in $OIFS (Old IFS) and set IFS to a newline character. After running the function we reassign IFS to what it was beforehand. This is because I dont know what you might be doing with your IFS. There are lots of reasons you might have already assigned it to something else, and I wouldnt want to break your flow. So we do the polite thing.

And in case the above gets munged for some reason you can see the plain text version here: bash-getopt/getopt.sh

Anyways, hope this helps someone out. If not it’s still here for me when *I* need it 😉

Bash Coding Convention ../

We use dirname() a lot in php to make relative paths work from multiple locations like so. The advantages are many:

require dirname( dirname( __FILE__ ) ) . '/required-file.php';
$data = file_get_contents( dirname(__FILE__).'/data/info.dat');

But in bash we often dont do the same thing, we settle for the old standby “../”. Which is a shame because unless your directory structure is set up exactly right, and you have proper permissions, and you run the command from the right spot, it doesnt work as planned. I think part of the reason is that its not obvious how to reliably get a full path to the script from inside itself. Another reason is that ../ is shorter to type and easier to remember. Finally there’s always one time scripts for which this methodology is overkill. But if you’re planning to write a script which other people will (or might) be using, I think it’s good practice to do it right. Googling for things you’d think to search for on this subject does not yeild very informative results, or incomplete (incorrect) methods… so… here’s how to do the above php in bash:

source $(dirname $(dirname $(readlink -f $0)))/required-file.sh 
data=$(cat $(dirname $(readlink -f $0))/data/info.dat)

Hope this helps someone 🙂

As a side note, the OSX readlink binary functions differently. You’ll want to use a package manager to install gnu coreutils, and iether use greadlink, or link greadlink to a higher precedence location on your $PATH (I have /opt/local/bin:/opt/local/sbin: at the beginning of my $PATH)

Simple TCP Daemon Example

Using some stuff I’ve covered in the past on my blog here’s a simple way to put up a daytime server (well to put any service onto a tcp port. I haven’t looked into its bi-directional capabilities yet, this was just sort of a proof-of-concept…

$ apt-get install ipsvd
$ wget http://blog.apokalyptik.com/files/daemonize/daemonize.c
$ cc daemonize.c -o daemonize
$ ./daemonize /var/run/daytime.pid /var/log/daytime.log 'tcpsvd 0 13 date'

start/stop and/or monit script are an extremely short jump from there… And kind of trivial/menial… so I leave that as an exercise to you… if you care 🙂

As Close to A Real Daemon As Bash Scripts Get

I’ve written a little something which is gaining some traction internally, and I always intended to share it with the world. So… Here. daemon-functions.sh

What it does is allow you to write a bash function called “payload” like so:

function payload() {
while [ true ]; do
checkforterm
date
sleep 1
done
}
source path/to/daemon-functions.sh

Once you’ve done that it all just happens.  daemon-functions gives you logging of stderr, stdout, a pid file, start, stop, pause, resume, and more functions.  when you start your daemon it detaches completely from your terminal and runs in the background.  Works very simply with monit straight out of the box.  you can have as many daemons as you wish in the same directory and they wont clobber each other (as the pid, control, and log files all are dynamically keyed off of the original script name.)  Furthermore inside your execution loop inside of the payload function place a checkforterm call at any place which it makes sense for your script to be paused, or stopped. it can detect stale pid files and run anyway if the process isnt really running.  As an added bonus you dont actually have to loop inside payload, you can put any thing in there, have a script thats not a daemon, but will take an hour, day, week, month to finish? stick it in, run it, and forget it.

building sed for osx

I work in linux a lot… not bsd. So the OSX (bsd style) implementation of sed really throws me for a loop when I go text-file-spelunking, whats worse is that my scripts using sed aren’t portable between the two OSs.

A quick googling this morning landed me here: http://wiki.octave.org/wiki.pl?OctaveForMac which gives perfectly good instructions on installing sed. except it didnt work. I grabbed the latest version of sed (4.1.5) and got the error 

sed: 1: "install_sh=/Users/apoka ...": command i expects \ followed by text
sed: 1: "install_sh=/Users/apoka ...": command i expects \ followed by text

Ironic, huh? Well taking a guess that at some point sed hadcome to depend on its own functionality to configure itself I jumped back a version… Figuring i replace BSD sed with an out of date GNU sed, and then use the old GNU sed to build the new GNU sed. Which worked great. I Installed first sed-3.0.2, and then 4.1.5 in this manner:

./configure --prefix=/usr/ --with-included-regex --with-included-gettext && make && sudo make install

I’m happy with my -r again…

# date | sed -r s/'[0-9]'/'?'/g
Thu Apr ?? ??:??:?? PDT ????