Random Thoughts

Autumn Leaves Leaf #2: Feeder

This handy little bot keeps track of RSS feeds, and announces in the channel when one is updated. (note: be sure to edit the path to the datafiles) Each poller runs inside its own ruby thread, and can be run on its own independent schedule

require 'thread'
require 'rss/1.0'
require 'rss/2.0'
require 'open-uri'
require 'fileutils'
require 'digest/md5'

class Feeder < AutumnLeaf

def watch_feed(url, title, sleepfor=300)
  message "Watching (#{title}) [#{url}] every #{sleepfor} seconds"
  feedid = Digest::MD5.hexdigest(title)
  Thread.new {
    while true
      begin
        content = ""
        open(url) { |s|
          content = s.read
        }
        rss = RSS::Parser.parse(content, false)
        rss.items.each { |entry|
          digest = Digest::MD5.hexdigest(entry.title)
          if !File.exist?("/tmp/.rss.#{feedid}.#{digest}")
            FileUtils.touch("/tmp/.rss.#{feedid}.#{digest}")
            message "#{entry.title} (#{title}) #{entry.link}"
          end
          sleep(2)
        }
      rescue
        sleep(2)
      end
      sleep(sleepfor)
    end
  }
  sleep(1)
end

def did_start_up
  watch_feed("http://planet.wordpress.org/rss20.xml", "planet", 600)
  watch_feed("http://wordpress.com/feed/", "wpcom", 300)
end

end

Autumn Leaves Leaf #1: Announcer

This bot is perfect for anything where you need to easily build IRC channel notifications into an existing process. It’s simple, clean, and agnostic. Quite simply you connect to a TCP port, give it one line, the port closes, the line given shows up in the channel. eg: echo ‘hello’ | nc -q 1 bothost 22122

require 'socket'
require 'thread'

class Announcer < AutumnLeaf

        def handle_incoming(sock)
                Thread.new {
                line = sock.gets
                message line
                sock.close
                }
        end

        def did_start_up
                Thread.new {
                        listener = TCPServer.new('',22122)
                        while (new_socket = listener.accept)
                                handle_incoming(new_socket)
                        end
                }
        end

end

Bash: “I Can’t Eat Another Byte”

root@server:/dir/ # ls | wc -l

1060731

root@server:/dir/ # for i in *; do rm -v $i done; done

me@home:~/ #

HUH?

Turns out that bash just couldn’t eat another byte, and next time I logged in I saw this: “bash[5469]: segfault at 0000007fbf7ffff8 rip 00000000004749bf rsp 0000007fbf7fffe0 error 6“… Impressive 🙂

Time is as time was as time will be

Every time I start dealing with time, programatically, I’m reminded of this quote

Our units of temporal measurement, from seconds on up to months,
are so complicated, asymmetrical and disjunctive so as to make
coherent mental reckoning in time all but impossible. Indeed, had
some tyrannical god contrived to enslave our minds to time, to
make it all but impossible for us to escape subjection to sodden
routines and unpleasant surprises, he could hardly have done
better than handing down our present system. It is like a set of
trapezoidal building blocks, with no vertical or horizontal
surfaces, like a language in which the simplest thought demands
ornate constructions, useless particles and lengthy
circumlocutions. Unlike the more successful patterns of language
and science, which enable us to face experience boldly or at least
level-headedly, our system of temporal calculation silently and
persistently encourages our terror of time.

… It is as though architects had to measure length in feet,
width in meters and height in ells; as though basic instruction
manuals demanded a knowledge of five different languages. It is
no wonder then that we often look into our own immediate past or
future, last Tuesday or a week from Sunday, with feelings of
helpless confusion. …

— Robert Grudin, `Time and the Art of Living’.

# info coreutils date

Scripting without killing system load

Let us pretend for a moment that you have a critical system which can *just* handle the strain that it’s under (I’m sure all of you have workloads well under your system capabilities, or capabilities well over your workload requirements, of course; still for the sake of argument…) And you have a job to do which will induce more load. The job has to be done. The system has to remain responsive. Your classic response to this problem is adding a delay, for example:

     #!/bin/bash
     cd /foo
     find ./ -type d -daystart -ctime +1 -maxdepth 1 | head -n 500 | xargs -- rm -rv
     while [ $? -eq 0 ]; do
          sleep 60
          find ./ -type d -daystart -ctime +1 -maxdepth 1 | head -n 500 | xargs -- rm -rv
     done

Of course this is a fairly simplistic example. Still it illustrates my point. The problem with this solution is that the machine you’re working on is likely to have a variable workload where its main use comes in surges. By defining a sleep time you have to iether sleep so long that the job takes forever to finish, or skirt with high loads and slow response times. Ideally you would be able to let her rip while the load is low and throttle her back while the load is high, right? Well we can! Like so:

     #!/bin/bash
     function waitonload() {
          loadAvg=$(cat /proc/loadavg | cut -f1 -d'.')
          while [ $loadAvg -gt $1 ]; do
               sleep 1
               echo -n .
               loadAvg=$(cat /proc/loadavg | cut -f1 -d'.')
               if [ $loadAvg -le $1 ]; then echo; fi
          done
     }

     waitonload 1
     find ./ -type d -daystart -ctime +1 -maxdepth 1 | head -n 500 | xargs -- rm -rv
     while [ $? -eq 0 ]; do
          waitonload 1
          find ./ -type d -daystart -ctime +1 -maxdepth 1 | head -n 500 | xargs -- rm -rv
     done

This modification will only run the desired commands when the system load is less than 2, it will wait for that condition to continue the loop. This can be very handy for very large jobs needing to be run on loaded systems. Especially jobs which can be subdivided into small tasks!

And we’re off

So now back off to the vet to pick up the dog who’s gonna spend most of the next few weeks being MISERABLE because he’s gonna have to be crated a a LOT, wont be able to play with the other dogs at all for a while, and will have to wear a satellite-dish-collar. On the plus side he’ll have a drain installed in his ear and he’ll be leaking gross fluids form his head for a while. I guess theres ALWAYS a silver lining, insn’t there?!

quick update

Well… everything that could go wrong this weekend (outside work) has gone wrong.

My replay tv is dying

My wifes laptop is dying

I have one dog getting out of surgery right now because of a hematoma

I have one cat with ringworm

One cat got the crap beat up out of her in a fight

I had my tire punctured so badly it needs replacing (this was on the way to the vet yesterday night when we noticed the swelling in buddy’s ear (the hematoma))

The dogs chewed a hose

One dog ate my wifes flipflops while we were away and puked pieces of it up all over the house

We have to go back to Fremont sometime this week to return the old cable receiver and modem or get sent to collections

Oh, and the IRS sent us a letter about a form missing from my tax return (I used TaxCut to avoid exactly this kind of crap!)

On the plus side… We’re alive 🙂

When simple plans attack!

Well at the zoo we lost a pair of glasses, so off to lensecrafters… no eye exam in 3 years… so eye exam… 2 hors for glasses… and WHAM a day thats supposed to end at 4:00pm ends at 9:00pm… sigh… 🙂 we now return you to your regularly scheduled blogging.

Adventures in babysitting

So apparently I, and my wife, are signed up to babysit… well… a baby (or very young kid)… so I guess I’ll be on my best behavior and do my best not to step on her by accident :)  Seriously though… We’ll be going to the zoo, so thats cool.  I haven’t been to a zoo in ages.  I may come home early, and not stay the full 6 or however many hours.  Welp.  Time to go pack up the ol’ iBook