Planet Openbox

Desligando (ou reiniciando) o computador via SSH (Secure Shell)

2012-02-03T18:48:00+00:00

Vejo que muitas pessoas acabam entrando no meu site querendo desligar ou reiniciar o computador via SSH e acabam indo parar em outro post. Para acabar com as dúvidas, caso queira reiniciar (ou desligar) o computador em uma sessão SSH basta fazer:

Para reiniciar:

# reboot

# init 6

E para desligar:

# halt

# init 0

É bom dar uma olhada nos comandos shutdown e na tabela de códigos do init.

Watch out, or you'll end up in Pleasant Valley

2012-02-03T01:30:14+00:00

I love this sign, along Interstate 5 near Coalinga. Pleasant Valley State Prison.

I guess if you have to be locked away, Pleasant Valley doesn't sound like the worst place to be.

Do Pleasant Valley ex-cons have a hard time getting respect when people find out where they did their hard time?

I love picturing the local parents trying to strike fear into their kids' hearts. "If you don't straighten up, Junior, you're going to end up in ... PLEASANT VALLEY!"

Or the local judges -- "You're not getting off this time. No, I'm giving you ten years in ... PLEASANT VALLEY!"

Customising ActiveRecord's attribute formatting on inspect

2012-02-02T13:30:35+00:00

We all know that you can change how a Ruby object is represented textually through the inspect method. ActiveRecord::Base provides a useful out-of-the-box implementation of inspect that allows you to see the content of all of a record’s attributes:

>> Gig.last
=> #<Gig id: 123, name: "ICELAB presents Ruby: A Rock Opera in 1.9.3 Thrilli...", starts_at: "2012-03-03 09:30:00", capacity: 750, available_ticket_numbers: nil, sold_ticket_numbers: nil, created_at: "2012-01-19 06:47:02", updated_at: "2012-01-19 06:47:02">

What you might not know is that ActiveRecord provides a hook for you to customise how each of these attributes is formatted in this inspect output. Let’s see how it is used in ActiveRecord::Base#inspect:

# Returns the contents of the record as a nicely formatted string.
def inspect
  attributes_as_nice_string = self.class.column_names.collect { |name|
    if has_attribute?(name) || new_record?
      "#{name}: #{attribute_for_inspect(name)}"
    end
  }.compact.join(", ")
  "#<#{self.class} #{attributes_as_nice_string}>"
end

You’ll see that it iterates over each of the attributes and passes them through attribute_for_inspect(name). This is also a public method that truncates a string if it is longer than 50 characters, shows dates in their DB-native UTC format, the otherwise just calls inspect on whatever the attribute value is.

Why would you want to customise this? We have an ActiveRecord with a few columns where we are serializing large arrays to the database

class Gig < ActiveRecord::Base
  serialize :available_ticket_numbers
  serialize :sold_ticket_numbers
end

This can lead to some very messy console output:

>> Gig.last
=> #<Gig id: 123, name: "ICELAB presents Ruby: A Rock Opera in 1.9.3 Thrilli...", starts_at: "2012-03-03 09:30:00", capacity: 750, available_ticket_numbers: nil, sold_ticket_numbers: ["1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13", "14", "15", "16", "17", "18", "19", "20", "21", "22", "23", "24", "25", "26", "27", "28", "29", "30", "31", "32", "33", "34", "35", "36", "37", "38", "39", "40", "41", "42", "43", "44", "45", "46", "47", "48", "49", "50", "51", "52", "53", "54", "55", "56", "57", "58", "59", "60", "61", "62", "63", "64", "65", "66", "67", "68", "69", "70", "71", "72", "73", "74", "75", "76", "77", "78", "79", "80", "81", "82", "83", "84", "85", "86", "87", "88", "89", "90", "91", "92", "93", "94", "95", "96", "97", "98", "99", "100", "101", "102", "103", "104", "105", "106", "107", "108", "109", "110", "111", "112", "113", "114", "115", "116", "117", "118", "119", "120", "121", "122", "123", "124", "125", "126", "127", "128", "129", "130", "131", "132", "133", "134", "135", "136", "137", "138", "139", "140", "141", "142", "143", "144", "145", "146", "147", "148", "149", "150", "151", "152", "153", "154", "155", "156", "157", "158", "159", "160", "161", "162", "163", "164", "165", "166", "167", "168", "169", "170", "171", "172", "173", "174", "175", "176", "177", "178", "179", "180", "181", "182", "183", "184", "185", "186", "187", "188", "189", "190", "191", "192", "193", "194", "195", "196", "197", "198", "199", "200", "201", "202", "203", "204", "205", "206", "207", "208", "209", "210", "211", "212", "213", "214", "215", "216", "217", "218", "219", "220", "221", "222", "223", "224", "225", "226", "227", "228", "229", "230", "231", "232", "233", "234", "235", "236", "237", "238", "239", "240", "241", "242", "243", "244", "245", "246", "247", "248", "249", "250", "251", "252", "253", "254", "255", "256", "257", "258", "259", "260", "261", "262", "263", "264", "265", "266", "267", "268", "269", "270", "271", "272", "273", "274", "275", "276", "277", "278", "279", "280", "281", "282", "283", "284", "285", "286", "287", "288", "289", "290", "291", "292", "293", "294", "295", "296", "297", "298", "299", "300", "301", "302", "303", "304", "305", "306", "307", "308", "309", "310", "311", "312", "313", "314", "315", "316", "317", "318", "319", "320", "321", "322", "323", "324", "325", "326", "327", "328", "329", "330", "331", "332", "333", "334", "335", "336", "337", "338", "339", "340", "341", "342", "343", "344", "345", "346", "347", "348", "349", "350", "351", "352", "353", "354", "355", "356", "357", "358", "359", "360", "361", "362", "363", "364", "365", "366", "367", "368", "369", "370", "371", "372", "373", "374", "375", "376", "377", "378", "379", "380", "381", "382", "383", "384", "385", "386", "387", "388", "389", "390", "391", "392", "393", "394", "395", "396", "397", "398", "399", "400", "401", "402", "403", "404", "405", "406", "407", "408", "409", "410", "411", "412", "413", "414", "415", "416", "417", "418", "419", "420", "421", "422", "423", "424", "425", "426", "427", "428", "429", "430", "431", "432", "433", "434", "435", "436", "437", "438", "439", "440", "441", "442", "443", "444", "445", "446", "447", "448", "449", "450", "451", "452", "453", "454", "455", "456", "457", "458", "459", "460", "461", "462", "463", "464", "465", "466", "467", "468", "469", "470", "471", "472", "473", "474", "475", "476", "477", "478", "479", "480", "481", "482", "483", "484", "485", "486", "487", "488", "489", "490", "491", "492", "493", "494", "495", "496", "497", "498", "499", "500", "501", "502", "503", "504", "505", "506", "507", "508", "509", "510", "511", "512", "513", "514", "515", "516", "517", "518", "519", "520", "521", "522", "523", "524", "525", "526", "527", "528", "529", "530", "531", "532", "533", "534", "535", "536", "537", "538", "539", "540", "541", "542", "543", "544", "545", "546", "547", "548", "549", "550", "551", "552", "553", "554", "555", "556", "557", "558", "559", "560", "561", "562", "563", "564", "565", "566", "567", "568", "569", "570", "571", "572", "573", "574", "575", "576", "577", "578", "579", "580", "581", "582", "583", "584", "585", "586", "587", "588", "589", "590", "591", "592", "593", "594", "595", "596", "597", "598", "599", "600", "601", "602", "603", "604", "605", "606", "607", "608", "609", "610", "611", "612", "613", "614", "615", "616", "617", "618", "619", "620", "621", "622", "623", "624", "625", "626", "627", "628", "629", "630", "631", "632", "633", "634", "635", "636", "637", "638", "639", "640", "641", "642", "643", "644", "645", "646", "647", "648", "649", "650", "651", "652", "653", "654", "655", "656", "657", "658", "659", "660", "661", "662", "663", "664", "665", "666", "667", "668", "669", "670", "671", "672", "673", "674", "675", "676", "677", "678", "679", "680", "681", "682", "683", "684", "685", "686", "687", "688", "689", "690", "691", "692", "693", "694", "695", "696", "697", "698", "699", "700", "701", "702", "703", "704", "705", "706", "707", "708", "709", "710", "711", "712", "713", "714", "715", "716", "717", "718", "719", "720", "721", "722", "723", "724", "725", "726", "727", "728", "729", "730", "731", "732", "733", "734", "735", "736", "737", "738", "739", "740", "741", "742", "743", "744", "745", "746", "747", "748", "749", "750", created_at: "2012-01-19 06:47:02", updated_at: "2012-01-19 06:47:02">

That’s a lot of noise. Most of the time, it just gets in the way when we’re trying to work with our records. If we ever wanted to inspect the content of such an attribute, we could just address it directly.

To clean this up in one shot across our entire application, we can override attribute_for_inspect. Put this in a file in lib/ somwhere and require it from your config/application.rb:

module ActiveRecord
  class Base
    alias_method :attribute_for_inspect_before_extension, :attribute_for_inspect
    def attribute_for_inspect(attr_name)
      value = read_attribute(attr_name)

      if value.kind_of?(Array) && value.length > 4
        (value[0..1] + ["... #{value.length - 4} elements ..."] + value[-2..-1]).inspect
      else
        attribute_for_inspect_before_extension(attr_name)
      end
    end
  end
end

This checks for any array attributes and gives them a shortened output if they contain more than five elements:

>> Gig.last
=> #<Gig id: 123, name: "ICELAB presents Ruby: A Rock Opera in 1.9.3 Thrilli...", starts_at: "2012-03-03 09:30:00", capacity: 750, available_ticket_numbers: nil, sold_ticket_numbers: ["1", "2", "... 746 elements ...", "749", "750"], created_at: "2012-01-19 06:47:02", updated_at: "2012-01-19 06:47:02">

Much neater. We get tidier inspect output without having to override ActiveRecord’s otherwise useful inspect altogether. Depending on your app, I’d imagine you could extend it in many different ways to clean up your time on the console.

Programa do Mução - A laje da casa de Will (30/01/2012)

2012-01-30T21:48:00+00:00

Programa do Mução - A laje da casa de Will (30/01/2012)

Arduino Air Swimmers Shark

2012-01-27T20:30:33+00:00

When SCALE approved my talk proposal, Fun with Linux and Devices, I had a challenge: I needed some good, visual Arduino demos that would work in front of an audience.

In particular, I wanted something that moved. A little toy truck? A moving penguin? A rotating sunflower? I fiddled with this and that, not fully satisfied with anything. And then suddenly I realized what I needed. Something cool. Something BIG. Something I'd been wanting an excuse to buy anyway.

An Air Swimmers Shark.

I'd seen these things on video, but never in person. They're available all over, even on Amazon, so I put in an order there and got a shark in a few days.

These things are ridiculous and cool. It's huge, about 5 feet long, and filled with helium. It takes maybe half an hour to assemble. It has a small motor to beat the tail, an infrared transmitter, and a weighted receiver that moves back and forth on a track to tilt the fish up or down as it swims.

Once it's assembled, you can get it filled with helium at a party store (which costs $3 to $6 depending on where you go). Once the shark is filled, you add clay as ballast until the shark is neutrally buoyant, neither rising nor sinking. It's quite sensitive: you'll find yourself needing to add or remove pea-sized chunks of clay as the temperature in the room changes, but being a little over- or under-ballasted doesn't hurt it much. With its tail beating, the shark really does look like it's swimming through the air.

My shark is named Bruce, after the mechanical shark used for the movie "Jaws". My Bruce, I'm happy to say, has been much more tractable than his famously intemperate namesake.

Okay, now how do we turn this ridiculous-but-cool thing into an Arduino project?

Hacking the transmitter

There were two possible approaches. First, mount an Arduino directly on the shark, and let it be totally self-directed. Second, patch the Arduino into the shark's transmitter and control it from Linux. I chose the second option, for several reasons. First, I was fairly sure it would be easier, and less invasive (the shark would still be usable with manual control). I also liked the idea of keeping the transmitter as a manual override, in case my control program didn't work right. Finally, I liked the idea of keeping a Linux machine in the loop -- the shark would actually be controlled by Linux, not just by the Arduino.

So the first thing I did was take the transmitter apart (4 Philips screws). Inside are 4 pushbuttons, for right, left, up, and down, and the circuit board is nice and simple. Whew -- this might be doable!

Four more screws and I had access to the back of the board, which was equally simple. Now I could get my voltmeter on the contacts while I pushed buttons.

It turned out the contacts (indicated with arrows on the photo) on the downstream side of each switch were normally high (4.5 volts -- the transmitter uses 3 AAA batteries). When I pushed the button, the contact went to ground. Okay, so what I needed was some way for the Arduino to ground those contacts at will.

First I needed to solder some wires to the contacts. (How do you tell which side of the pushbutton is the one you need to solder? Well, one side changes voltage when you press the button, and the other side stays constant. The one that changes is the one you need to connect to the Arduino, so the Arduino can change it too.)

I figured I needed 6 wires: ground, power, and one for each switch. (It turned out I didn't need the power wire, but I figured it didn't hurt to include it just in case.) I wanted to have a nice small connector on the side of the transmitter, but I couldn't find any 6-pin connectors that didn't look big and bulky, so I gave up and decided I'd just let my ribbon cable dangle from the transmitter. If I got a nice multi-colored one, maybe it would look festive.

I couldn't find any 6-conductor ribbon cable, so I got a wider one and separated 6 wires from the rest. Then I soldered the six wires to the appropriate places (marked by arrows in the photo). On the other end, I tinned the six wires with solder so I could plug the stranded wires into my breadboard.

Simulating button presses

I've done enough reading to know of three ways to simulate a button press. You can put a relay between the two contacts of the switch; you can do the same thing, but with an optocoupler (opto-isolator) instead of a relay; or you can do some magic with a transistor. I was fuzzy on the transistor magic bit, so a relay sounded easiest.

I played around with a relay and a spare switch and convinced myself I knew how to wire them up. Then it was off to my local parts store to buy four matched relays small enough to fit on my little mini breadboard.

There followed a several-day fiasco wherein I bought lots of relays that turned out not to be suitable, and got increasingly frustrated at how large and clunky all the available relays were. There are smaller ones, but I couldn't get them to work. And I learned that relays mostly come without documentation on which pin does which, so there's a lot of experimenting with each new type.

Frustrated, I tried some optocouplers I'd bought on a whim last year. No dice ... couldn't get them to work either.

Desperate, I turned to IRC, #arduino on Freenode. The folks there are mostly electronics wizards, and I'm sure my questions must have seemed very dumb, but they were patient with me, and pointed me toward a very simple circuit, LED4dummies, that was just what I needed. (They also suggested Wikipedia's Open collector article, but I found that less clear.)

It took me some experimenting with a transistor, an LED and a couple of resistors (I blew out a couple of transistors before I realized I had the R2 resistor in the wrong place) but eventually I got it working, and felt confident enough to try it with the real shark transmitter. The key was to simplify the circuit so it had no extra parts, then once it was working, add more parts to build it up to what I needed.

At left, the circuit I ended up with. For each button, I have one transistor and one resistor (I don't need the second resistor from the LED4dummies circuit, since that was just to keep the LED from burning out).

At right is the circuit assembled on a mini-breadboard on top of the proto-shield. Note that the ends of the ribbon cable are plugged in to a spare header I had lying around; a header makes a passable connector, so I can plug it in fairly easily right before a talk. The green and blue wires in the back are going to Arduino digital output pins 3 through 6 (leaving 0 and 1 for serial I/O). The red wires go from the transistors back to the ribbon cable wires that go to the shark's transmitter buttons.

The software side

Now I could make the shark think I'd pressed a button on its transmitter. How do I control that from Linux?

On the Arduino side, I wrote a simple program that reads and parses commands coming over the USB cable. So from the computer, I might send a line like L 300, and the Arduino would "press" the Left button for 300 milliseconds. I had already written something like this for a couple of other Arduino programs. That program is shark.pde.

On the Linux side, first I needed something that established a serial connection and sent commands to the Arduino. I wrote a Python class for that, shark.py. That let me send commands from the Python console to test the shark.

Then I needed something more visual, something I could show during a talk. In particular, the shark doesn't swim unless someone's pressing left, right, left, right buttons over and over. Of course I wanted the computer to handle that part.

So I wrote a little Python-GTK application that keeps the shark swimming, and lets me drag a mouse around to adjust its left/right up/down direction: sharkwindow.

Purely by coincidence, the week before SCALE, Scott Adams introduced a roboshark character: Dilbert, Jan 11 2012. Nice timing for the debut of my own roboshark!

Sadly, I don't have any photos or video of the shark in action. But if you're a LWN subscriber, there's an article on my talk with a couple of great pictures: Robots rampage (in a friendly way) at SCALE 10X. And you can see my slides and notes at Arduino notes.

São Roque, São Roque, São Roque!

2012-01-25T18:55:47+00:00

Quem tem medo de cachorro vai entender.

Programa do Mução - Carniça fazendo o bordão do Licurgo. Irre!

2012-01-24T22:15:11+00:00

Programa do Mução - Carniça fazendo o bordão do Licurgo. Irre!

Novela “Passeontem”

2012-01-24T22:04:45+00:00

Novela “Passeontem”

Corno, a gente se liga em você!

2012-01-24T22:01:47+00:00

Corno, a gente se liga em você!

Render Single-Line Markdown Text with Redcarpet

2012-01-24T00:00:00+00:00

At Icelab, we love Markdown. We use it wherever we can for text formatting. In a web app, the obvious place for it is in large text areas, where we can allow complete freedom of formatting. Headers, paragraphs, lists, it’s all good.

What about the formatting of text in single-line text fields? If our form entry is a single line, that’s usually how its text will be displayed in our interface. In this case, we probably want to avoid all the block-level elements that Markdown will let the user create.

This is easy to do using Redcarpet, a fantastic Markdown renderer for Ruby. It is fast and, importantly, it is modular: it allows us to define our own custom renderers. We can use this to create a renderer that ignores all the block-level Markdown elements. Put this somewhere in your Rails app (e.g., lib/redcarpet_renderers.rb):

module Redcarpet
  module Render
    class HTMLWithoutBlockElements < HTML
      include SmartyPants

      def initialize(opts = {})
        opts[:tables] = false
        super(opts)
      end

      # Regular markdown, just ignore all the block-level elements

      def block_code(code, language)
        code
      end

      def block_quote(quote)
        quote
      end

      def block_html(raw_html)
        raw_html
      end

      def header(text, header_level)
        "#{text} "
      end

      def hrule
        " "
      end

      def list(contents, list_type)
        " #{contents}"
      end

      def list_item(text, list_type)
        "* #{text}"
      end

      def paragraph(text)
        text
      end

      # Span-level calls

      def linebreak
        " "
      end

      # Postprocessing: strip the newlines

      def postprocess(document)
        document.gsub("\n", ' ').strip
      end
    end
  end
end

Now we can safely render the content from our single-line text fields using markdown. The user can still make things _emphasised_ or **bold** or even [linked](http://icelab.com.au/) and we don’t have to worry about unwanted block elements messing up our page layouts.

To use this renderer, just throw something like the following into your app/helpers/application_helper.rb file:

module ApplicationHelper
  def markdown(text)
    renderer = Redcarpet::Render::HTML.new({
      :filter_html => true,
      :hard_wrap => true
    })
    markdown = Redcarpet::Markdown.new(renderer, {
      :autolink => true,
      :no_intra_emphasis => true
    })

    markdown.render(text).html_safe
  end

  def markdown_line(text)
    renderer = Redcarpet::Render::HTMLWithoutBlockElements.new({
      :filter_html => true,
      :hard_wrap => true
    })
    markdown = Redcarpet::Markdown.new(renderer, {
      :autolink => true,
      :no_intra_emphasis => true
    })

    markdown.render(text).html_safe
  end
end

And it’s now a helper ready for use in our page templates. For example:

<div class="item">
  <h3 class="name"><%= markdown_line(item.name) %></h3>
  <div class="description">
    <%= markdown(item.description) %>
  </div>
</div>

The more Markdown we can use, the happier we are!

New Relic, Heroku and Rails Streaming Responses

2012-01-20T00:00:00+00:00

For Ticketscout’s payment processing, we use a custom Rails action with an HTTP streaming response. It shows the user a spinner and a “please wait” message, then periodically sends empty strings to the browser until the credit card transaction is completed. This streaming response keeps the request connection alive longer than the 30 second timeout period that is the default on the Heroku Cedar stack. The technique works well and ensures we don’t ever lose track of any slow-to-process credit card payments.

During development, we found that the streaming responses on Heroku did not work if we had the New Relic add-on enabled. Instead of seeing the page content incrementally delivered while the request ran, we’d see nothing until the request completed, then all the content delivered at once.

To deliver the feature on time, we simply disabled the add-on and released. Everything worked smoothly, but it was unnerving to run the app “blind” without New Relic’s useful metrics. This became especially problematic now that we’re wanting to pay some more attention to improving the app’s performance. I finally contacted New Relic support, and it turns out that a working solution is fairly simple: disabling the real user monitoring.

Turns out that the real user monitoring is currently incompatible with streaming responses. I disabled the automatic instrumentation in my newrelic.yml config file:

browser_monitoring:
  auto_instrument: false

I also switched off the “Enable end user monitoring” setting in the “Settings > Application” page. Then, after fresh deploy to Heroku, I re-enabled the add-on and we had both the metrics collection and streaming responses working together!

Granted, we’ve lost access to some useful real user metrics (like network load, page rendering and DOM processing times), but it’s better to have some metrics than none at all!

Down the track, we can probably get the best of both world’s by manually enabling the real user monitoring instrumentation in all the pages layouts except those used for the streaming responses. Hopefully further down the track, the automatic instrumentation will be compatible with streaming responses out of the box. In the meantime, it’s good to have our charts back!

Converting HTML pages into PDF

2012-01-17T19:30:11+00:00

I've long wanted a way of converting my HTML presentation slides to PDF. Mostly because conference organizers tend to freak out at slides in any format other than Open/Libre Office, Powerpoint or PDF. Too many times, I've submitted a tarball of my slides and discovered they weren't even listed on the conference site. (I ask you, in what way is a tarball more difficult to deal with than an .odp file?) Slide-sharing websites also have a limited set of formats they'll accept.

A year or so ago, I added screenshot capability to my webkit-based presentation program, Preso, do "screenshots", but I really needed PDF, not images.

Now, creating PDF from HTML shouldn't be that hard. Every browser has a print function that can print to a PDF file. So why is it so hard to create PDF from HTML in any kind of scriptable way?

After much searching and experimenting, I finally found a Python code snippet that worked: XHTML to PDF using PyGTK4 Webkit from Alex Dong. It uses Python-Qt, not GTK, so I can't integrate it into my Preso app, but that's okay -- a separate tool is just as good.

(I struggled to write an equivalent in PyGTK, but gave up due to the complete lack of documentation of Python-Webkit-GTK, and not much more for gtk.PrintOperation(). QWebView's documentation may not be as complete as I'd like, but at least there is some.)

Printing from QtWebView to QPrinter

Here are the important things I learned about QWebView from fiddling around with Alex's code to adapt it to what I needed, which is printing a list of pages to sequentially numbered files:

To print, you need to wait until the page has finished loading, so connect a function to SIGNAL("loadFinished(bool)"), then load(QUrl(url)).
That loadFinished function remains registered, so as you load new pages, it will be called each time. So you can load() the next URL as the last step in your loadFinished callback.
If you get confused about callbacks and connect more than one of them, bad things happen, and only the last page gets printed, or QApplication.exit() doesn't exit at all.

Things I learned about QPrinter():

All the examples I found online set the page size with lines like QPrinter.setPageSize(QPrinter.A4) or setPaperSize(QPrinter.A4) (setPageSize is apparently deprecated in favor of setPaperSize); but
If you want to set a specific size, you can do that with a line like QPrinter.setPaperSize(QSizeF(1024, 768), QPrinter.DevicePixel) The second argument (DevicePixel) is a unit, from this list.
That line gives you the right aspect ratio. But if you think "DevicePixels" means the size will correspond to pixels in your browser window (just because the documentation says so), you're sadly mistaken.
If you want a PDF page that actually corresponds to the size of your browser window, you can get it by calling QWebView.setZoomFactor(z) You'll have to experiment to find the right value of z; I found I needed about 1.24 if I wanted to capture my full 1366x768 slides, or exactly 2.0 if I wanted to restrict the saved PDF to only the 1024x758 part that shows up in the projector.

Anyway, it's a little hacky with that empirical zoom factor ... but it works! The program is here: qhtmlprint: convert HTML to PDF using Qt Webkit.

And it does produce reasonable PDF, with the text properly vectorized, not just raster screenshots of each page.

Printing the slides in the right order

Terrific -- now I can feed a list of slides to qhtmlprint and get a bunch of PDF files back. How do I print the right slides?

My slides are listed in order in an array inside a Javascript file, one per line. If I grep .html navigate.js, I get a list like this:

    "arduino.html",
    "img.html?pix/arduinos/arduino-clones.jpg",
    "getting_started.html",
    "img.html?pix/projects/led.jpg",
    //"blink.html",
    "arduino-ide.html",

To pass that to qhtmlprint, I only need to remove the commented-out lines (the ones with //) and strip off the quotes and commas. I can do that all in one command with a grep and sed pipeline:

qhtmlprint ` fgrep .html navigate.js  | grep -v // | sed -e 's/",/"/' -e 's/"//g' `

And voiaà! I have a bunch of fileNNN.pdf files.

Creating a multi-page slide deck

Okay, great! Now how do I stick those files all together into one slide deck I can submit to conference organizers?

That part's easy -- Ghostscript can do it.

gs -dNOPAUSE -sDEVICE=pdfwrite -sOUTPUTFILE=slidedeck.pdf -dBATCH file*.pdf

And now slidedeck.pdf contains my whole presentation, ready to go.

Moira para XFCE

2012-01-17T03:06:42+00:00

Baseado em um outro tema meu, só que para Openbox. O nome é inspiração de uma música da banda Madame Saatan (muito boa, por sinal).

Download @ deviantART

Perde quem?

2012-01-15T05:11:56+00:00

HTML and Javascript Presentations

2012-01-13T04:30:14+00:00

When I give talks that need slides, I've been using my Slide Presentations in HTML and JavaScript for many years. I uploaded it in 2007 -- then left it there, without many updates.

But meanwhile, I've been giving lots of presentations, tweaking the code, tweaking the CSS to make it display better. And every now and then I get reminded that a few other people besides me are using this stuff.

For instance, around a year ago, I gave a talk where nearly all the slides were just images. Silly to have to make a separate HTML file to go with each image. Why not just have one file, img.html, that can show different images? So I wrote some code that lets you go to a URL like img.html?pix/whizzyphoto.jpg, and it will display it properly, and the Next and Previous slide links will still work.

Of course, I tweak this software mainly when I have a talk coming up. I've been working lately on my SCALE talk, coming up on January 22: Fun with Linux and Devices (be ready for some fun Arduino demos!) Sometimes when I overload on talk preparation, I procrastinate by hacking the software instead of the content of the actual talk. So I've added some nice changes just in the past few weeks.

For instance, the speaker notes that remind me of where I am in the talk and what's coming next. I didn't have any way to add notes on image slides. But I need them on those slides, too -- so I added that.

Then I decided it was silly not to have some sort of automatic reminder of what the next slide was. Why should I have to put it in the speaker notes by hand? So that went in too.

And now I've done the less fun part -- collecting it all together and documenting the new additions. So if you're using my HTML/JS slide kit -- or if you think you might be interested in something like that as an alternative to Powerpoint or Libre Office Presenter -- check out the presentation I have explaining the package, including the new features.

You can find it here: Slide Presentations in HTML and JavaScript

Novas cores no terminal

2012-01-12T18:03:00+00:00

Text color: #CCCCCC
Bold color: #FFFFFF
Background color: #121212

#1: #484848
#9: #909090

#2: #C82727
#10: #ED152A

#3: #82B414
#11: #B6E354

#4: #FD971F
#12: #FEED6C

#5: #5664D6
#13: #8C9EFF

#6: #AC57B2
#14: #F16FFE

#7: #009DBF
#15: #8BDAEE

#8: #E0E0E0
#16: #F8F8F2

Scroll do Microsoft Wireless Mouse 5000 muito rápido?

2012-01-11T03:58:00+00:00

Microsoft Wireless Mouse 5000

Possui esse mouse e percebe que o scroll (a rodinha) está mais rápida que o meteóro de Pegasus? Basta desconectar o transceiver USB e conectar novamente. Nem precisa reiniciar o PC.

Microsoft Wireless Transceiver

Algumas pessoas tiveram esse problema no Windows. No meu caso aconteceu no Linux. Acredito que a solução sirva para os dois sistemas.

Parsing HTML in Python

2012-01-08T22:30:12+00:00

I've been having (mis)adventures learning about Python's various options for parsing HTML.

Up until now, I've avoided doing any HTMl parsing in my RSS reader FeedMe. I use regular expressions to find the places where content starts and ends, and to screen out content like advertising, and to rewrite links. Using regexps on HTML is generally considered to be a no-no, but it didn't seem worth parsing the whole document just for those modest goals.

But I've long wanted to add support for downloading images, so you could view the downloaded pages with their embedded images if you so chose. That means not only identifying img tags and extracting their src attributes, but also rewriting the img tag afterward to point to the locally stored image. It was time to learn how to parse HTML.

Since I'm forever seeing people flamed on the #python IRC channel for using regexps on HTML, I figured real HTML parsing must be straightforward. A quick web search led me to Python's built-in HTMLParser class. It comes with a nice example for how to use it: define a class that inherits from HTMLParser, then define some functions it can call for things like handle_starttag and handle_endtag; then call self.feed(). Something like this:

from HTMLParser import HTMLParser

class MyFancyHTMLParser(HTMLParser):
  def fetch_url(self, url) :
    request = urllib2.Request(url)
    response = urllib2.urlopen(request)
    link = response.geturl()
    html = response.read()
    response.close()
    self.feed(html)   # feed() starts the HTMLParser parsing

  def handle_starttag(self, tag, attrs):
    if tag == 'img' :
      # attrs is a list of tuples, (attribute, value)
      srcindex = self.has_attr('src', attrs)
      if srcindex < 0 :
        return   # img with no src tag? skip it
      src = attrs[srcindex][1]
      # Make relative URLs absolute
      src = self.make_absolute(src)
      attrs[srcindex] = (attrs[srcindex][0], src)

    print '<' + tag
    for attr in attrs :
      print ' ' + attr[0]
      if len(attr) > 1 and type(attr[1]) == 'str' :
        # make sure attr[1] doesn't have any embedded double-quotes
        val = attr[1].replace('"', '\"')
        print '="' + val + '"')
    print '>'

  def handle_endtag(self, tag):
    self.outfile.write('</' + tag.encode(self.encoding) + '>\n')

Easy, right? Of course there are a lot more details, but the basics are simple.

I coded it up and it didn't take long to get it downloading images and changing img tags to point to them. Woohoo! Whee!

The bad news about HTMLParser

Except ... after using it a few days, I was hitting some weird errors. In particular, this one:
HTMLParser.HTMLParseError: bad end tag: ''

It comes from sites that have illegal content. For instance, stories on Slate.com include Javascript lines like this one inside <script></script> tags:
document.write("<script type='text/javascript' src='whatever'></scr" + "ipt>");

This is technically illegal html -- but lots of sites do it, so protesting that it's technically illegal doesn't help if you're trying to read a real-world site.

Some discussions said setting self.CDATA_CONTENT_ELEMENTS = () would help, but it didn't.

HTMLParser's code is in Python, not C. So I took a look at where the errors are generated, thinking maybe I could override them. It was easy enough to redefine parse_endtag() to make it not throw an error (I had to duplicate some internal strings too). But then I hit another error, so I redefined unknown_decl() and _scan_name(). And then I hit another error. I'm sure you see where this was going. Pretty soon I had over 100 lines of duplicated code, and I was still getting errors and needed to redefine even more functions. This clearly wasn't the way to go.

Using lxml.html

I'd been trying to avoid adding dependencies to additional Python packages, but if you want to parse real-world HTML, you have to. There are two main options: Beautiful Soup and lxml.html. Beautiful Soup is popular for large projects, but the consensus seems to be that lxml.html is more error-tolerant and lighter weight.

Indeed, lxml.html is much more forgiving. You can't handle start and end tags as they pass through, like you can with HTMLParser. Instead you parse the HTML into an in-memory tree, like this:

  tree = lxml.html.fromstring(html)

How do you iterate over the tree? lxml.html is a good parser, but it has rather poor documentation, so it took some struggling to figure out what was inside the tree and how to iterate over it.

You can visit every element in the tree with

for e in tree.iter() :
  print e.tag

But that's not terribly useful if you need to know which tags are inside which other tags. Instead, define a function that iterates over the top level elements and calls itself recursively on each child.

The top of the tree itself is an element -- typically the <html></html> -- and each element has .tag and .attrib. If it contains text inside it (like a <p> tag), it also has .text. So to make something that works similarly to HTMLParser:

def crawl_tree(tree) :
  handle_starttag(tree.tag, tree.attrib)
  if tree.text :
    handle_data(tree.text)
  for node in tree :
    crawl_tree(node)
  handle_endtag(tree.tag)

But wait -- we're not quite all there. You need to handle two undocumented cases.

First, comment tags are special: their tag attribute, instead of being a string, is <built-in function Comment> so you have to handle that specially and not assume that tag is text that you can print or test against.

Second, what about cases like <p>Here is some <i>italicised</i> text.</p> ? in this case, you have the p tag, and its text is "Here is some ". Then the p has a child, the i tag, with text of "italicised". But what about the rest of the string, " text."?

That's called a tail -- and it's the tail of the adjacent i tag it follows, not the parent p tag that contains it. Confusing!

So our function becomes:

def crawl_tree(tree) :
  if type(tree.tag) is str :
    handle_starttag(tree.tag, tree.attrib)
    if tree.text :
      handle_data(tree.text)
    for node in tree :
      crawl_tree(node)
    handle_endtag(tree.tag)
  if tree.tail :
    handle_data(tree.tail)

See how it works? If it's a comment (tree.tag isn't a string), we'll skip everything -- except the tail. Even a comment might have a tail:
<p>Here is some  text we want to show.</p>
so even if we're skipping comment we need its tail.

I'm sure I'll find other gotchas I've missed, so I'm not releasing this version of feedme until it's had a lot more testing. But it looks like lxml.html is a reliable way to parse real-world pages. It even has a lot of convenience functions like link rewriting that you can use without iterating the tree at all. Definitely worth a look!

OpenSource Enthusiasts Need a Better Clipboard Tool

2012-01-06T01:02:55+00:00

Dear LazyWeb,

From time to time, I am required to use third party tools in my job where I may have to copy a bunch of information (separate fields of data) and paste them one at a time into a web form. Years ago, while working at a small web design firm, I was introduced to a tool for Windows called ClipMate. At that point in time, ClipMate proved to be a huge time-saver when copying customer data from our Access database and then pasting it into an online account setup form for a third party web-stats tracking service that we used on client websites. What sets ClipMate apart from other clipboard tools is that it not only stores a list of your clips (either text or images), but it will allow you to select a clip from where you would like to start pasting from, and then continue pasting down the list, one clip at a time. In the example that I stated previously, can you imagine going to the database, copying the client’s name, then pasting the name in the web form, then going back to the database to copy the next required chunk of information, and so forth…? Wouldn’t it be nice to be able to go to the database and copy, copy, copy, copy, copy…, then go to the web form and paste, paste, paste, paste….? ClipMate allows you to do this.

There are a handful of clipboard managers/organizers in the world of open source, but I have yet to find one that will do as I have described above. It has been some years now since I’ve used ClipMate, but recently, because of some repetitive copying and pasting that I’ve been doing in my job, I purchased a new license and am again happily plodding along with my friend ClipMate. However, this requires that I am using a Windows machine. Fortunately in my job, I have two separate production machines – on Linux and one Windows, so I am able to pick and choose the tools that work best for the task that I am trying to accomplish. Below are some screenshots of ClipMate in action:

I would like to have a Linux-friendly clipboard tool that will do as I have described – pasting successively from the list of clipboard elements. Seeing as how the clipboard tools that I’ve seen will allow you to select an item from the clipboard list and paste that item, it doesn’t seem like it would be all that difficult to implement a feature that would allow you to successively paste one item at a time from the list. I’m not sure that my programming skills are sharp enough to accomplish this at the moment, nor do I have the time to invest in trying to figure out how to implement such a feature, so I am hoping that some skilled individual will read my plea and decide to do what Open Sourcers do… grab some fine piece of software that already exists, make it better, and re-release it to the world.

Incidentally, I’ve tried running ClipMate under Crossover and Wine, and although the software did run, it somehow did not seem to be able to connect with my clipboard and so it was useless to me.

Maybe what I am looking for already exists and I just am not aware of it. Please feel free to comment and let me know if you have found a tool that will work as I have described, or if you have created something that will perform the desired task. Thank you!

A geeky HPR New Year's Eve

2012-01-05T12:00:35+00:00

I am not sure if it is just me, but I have always found New Year’s Eve to be problematic. Social doctrine dictates that I should probably celebrate the event by socialising in meatspace, which involves getting totally wankered on alcohol, dancing like a complete muppet, and freezing my tits off as I watch a squazillion pounds worth of gunpowder being shot into space. Now, call me a “boring old fart”, but personally that sounds like far too much complicated multitasking for one night.

So, when I read that the Hacker Public Radio crew were planning on holding a special 12 hour LIVE New Year’s Eve event, I danced with joy (well, not quite, but I was pretty excited.) I mean, what better way to celebrate the New Year than by joining some like-minded geeks for an open round-table discussion?

So, both Becky and I joined the event for a few hours and we spent our night chatting about all manner of things. Topics included some chatter about CrunchBang, mesh networks, drugs and naughty stuff, as well as other random subjects.

It was a fun event and I just wanted thank everybody involved for allowing Becky and I to be a part of it. It was definitely the geekiest New Year’s Eve that I have ever experienced and I really hope it becomes a regular event!

P.S. If you missed the event, but would like to listen to the discussions, the whole 12 hour show is being made available from the HPR site.

--
Tagged: crunchbang linux, hacker public radio, hpr

Open the X selection in a browser window, from any desktop

2012-01-04T06:00:17+00:00

Like most Linux users, I use virtual desktops. Normally my browser window is on a desktop of its own.

Naturally, it often happens that I encounter a link I'd like to visit while I'm on a desktop where the browser isn't visible. From some apps, I can click on the link and have it show up. But sometimes, the link is just text, and I have to select it, change to the browser desktop, paste the link into firefox, then change desktops again to do something else while the link loads.

So I set up a way to load whatever's in the X selection in firefox no matter what desktop I'm on.

In most browsers, including firefox, you can tell your existing browser window to open a new link from the command line: firefox http://example.com/ opens that link in your existing browser window if you already have one up, rather than starting another browser. So the trick is to get the text you've selected.

At first, I used a program called xclip. You can run this command: firefox `xclip -o` to open the selection. That worked okay at first -- until I hit my first URL in weechat that was so long that it was wrapped to the next line. It turns out xclip does odd things with multi-line output; depending on whether it thinks the output is a terminal or not, it may replace the newline with a space, or delete whatever follows the newline. In any case, I couldn't find a way to make it work reliably when pasted into firefox.

After futzing with xclip for a little too long, trying to reverse-engineer its undocumented newline behavior, I decided it would be easier just to write my own X clipboard app in Python. I already knew how to do that, and it's super easy once you know the trick:

mport gtk
primary = gtk.clipboard_get(gtk.gdk.SELECTION_PRIMARY)
if primary.wait_is_text_available() :
    print primary.wait_for_text()

That just prints it directly, including any newlines or spaces. But as long as I was writing my own app, why not handle that too?

It's not entirely necessary on Firefox: on Linux, Firefox has some special code to deal with pasting multi-line URLs, so you can copy a URL that spans multiple lines, middleclick in the content area and things will work. On other platforms, that's disabled, and some Linux distros disable it as well; you can enable it by going to about:config and searching for single, then setting the preference editor.singlelinepaste.pasteNewlines to 2.

However, it was easy enough to make my Python clipboard app do the right thing so it would work in any browser. I used Python's re (regular expressions) module:

#!/usr/bin/env python

import gtk
import re

primary = gtk.clipboard_get(gtk.gdk.SELECTION_PRIMARY)

if not primary.wait_is_text_available() :
    sys.exit(0)
s = primary.wait_for_text()

# eliminate newlines, and any spaces immediately following a newline:
print re.sub(r'[\r\n]+ *', '', s)

That seemed to work fine, even on long URLs pasted from weechat with newlines and spaces, like that looked like

http://example.com/long-
    url.html

All that was left was binding it so I could access it from anywhere. Of course, that varies depending on your desktop/window manager. In Openbox, I added two items to my desktop menu in menu.xml:

  <item label="open selection in Firefox">
    <action name="Execute"><execute>sh -c 'firefox `xclip -o`'</execute></action>
  </item>
  <item label="open selection in new tab">
    <action name="Execute"><execute>sh -c 'firefox -new-tab `xclip -o`'</execute></action>
  </item>

I also added some code in rc.xml inside <context name="Desktop">, so I can middle-click or control-middle-click on the desktop to open a link in the browser:

      <mousebind button="Middle" action="Press">
        <action name="Execute">
          <execute>sh -c 'firefox `pyclip`'</execute>
        </action>
      </mousebind>
      <mousebind button="C-Middle" action="Press">
        <action name="Execute">
          <execute>sh -c -new-tab 'firefox `pyclip`'</execute>
        </action>
      </mousebind>

I set this up maybe two hours ago and I've probably used it ten or fifteen times already. This is something I should have done long ago!

2011 in Review

2012-01-04T00:00:00+00:00

Each time I come to these posts, the year just past seems more activity-filled than ever. 2011 has been no exception. Another big year, with quite a few notable firsts for me:

A stake

In July I became a director and equal partner at Icelab, along with Michael and Max. My time working at the lab so far has been immensely satisfying. I am privileged to work with such a great team, and look forward to having Icelab as the foundation of many exciting future endeavours.

An app

For years I’d toyed with the idea of building an iOS app. It’s a platform I love to use, and one that will play an important role in the future of computing, and I wanted to be a part of it. Finally, with my attendance at Swipe Conference as motivation, I learnt Objective-C and Cocoa, and built and released an app (also thanks to the tireless design work of Max). Better still, it was an app for my favorite place on the internet, Decaf Sucks.

We released the app in August, just after Decaf Sucks' second birthday. Now, with a few months behind it, it looks like the release has been been quite successful. Personally, it’s found me a bunch of new coffee to try, and helped to take me well past 250 reviews. I hope it is a positive contribution for the coffee lovers of the world.

With this big first release under my belt, I feel empowered to create a whole bunch of fun new experiences on iOS. Hopefully there’ll be a lot more to share on this front in 2012.

A talk

In September, the whole Icelab crew travelled up to Sydney for Web Directions South, where Michael and I spoke about smart choices for smartphone apps (i.e., web or native, and tell me without the dogma).

This was my first major conference presentation and I was honored to be part of such an inspirational event. I found the whole experience to be really rewarding. I enjoyed the singular focus that it encouraged for a good few weeks of thought, and the extra depths that it encouraged me to plumb in order to give a comprehensive overview of the topic. Its an experience I’d definitely like to repeat. Conferences of the world, who would like to have me in 2012? ;–)

A move

If all this wasn’t enough, my wife and I capped off our year by packing up our lives in Australia and moving to the Philippines! I must say a little more about my amazing wife at this point: not only did she put up with me, look after me and keep me sane throughout my work on all of the above, she also shouldered most of the load for planning this move. I couldn’t have done half as much this year without her, and I’m lucky to be her husband.

Anyway, as I write this now, I look out our front window onto the lush green of a small park, surrounded by palm trees, goats grazing, roosters crowing, and the kids of the neighbourhood running around. I’m definitely in a whole new place, and I look forward to the many adventures I will have here, away from the computer screen.

And more

In 2011, Icelab grew from three to five people. Hugh and Andy are good humans to see everyday and are invaluable as members of the team. I got to witness a lot of great stuff come out of the lab this year, much of which I wish I could have worked on myself. I think this is a sign of a good workplace.

I did get to make one big release. In November we launched the new Corner Hotel site, along with a completely new ticket sales and box office management system, a Rails app that was practically my sole responsibility for the whole year. It’s one of Icelab’s biggest projects to date, and happily, everything’s gone pretty well since launch.

Aside from shipping new projects, we also kept-on-shipping with our coffee subscription service, Dispatch. It’s now over a year in the running, and we’ve had a lot of fun sending coffee around the country.

In the middle of 2012, my wife and I started eating paleo. It was a terrific decision. I’ve never felt healthier in my life, nor eaten better food. While it’s harder to follow closely here in the Philippines, we do our best, and I know that it will be an eating habit that I’ll keep for the long term.

And in one final notable event, in April we were lucky to witness the wedding of our good friends Hugh & Tomomi. Since moving to Canberra a few years ago, the friends I’ve made there really make it feel like home. We’ll be back before too long.

So that’s 2011. I finished the year feeling close to burnt out, but I’m proud of what I was able to achieve and I’ve now learnt the critical (if not already obvious) lesson that it’s best not to have many work commitments overlapping with an international house move. Now that we’ve settled here and had a short break over Christmas, I’m ready to go again and looking forward to a productive & venturesome 2012!

Good things

2012-01-03T12:39:41+00:00

Just recently, I have received a number of emails from people who have discovered CrunchBang. The following, by Paul McClellan, is just one example:

Wanted to tell you that after using linux as a desktop & server for 12+ years, I never found a linux distro that I liked for the packages, the stability & the aesthetics all at once. At most got two out of three. Until I found #! - I can’t tell you what an eerie yet delightful experience it was when I first started it up & found everything just so.

I hope to see you continue with this distro, while the donation is a tiny pittance, I hope to do more as time goes on & send like-minded folks along to try #!. I think there are many of us who are like me somewhat technical, yet have a sense of aesthetics and efficiency that recoils at either “bling” loaded desktops or worflow-interrupting instability.

My sincere esteem to you sir for making a Good Thing.

Now, I do not seek out affirmation that CrunchBang is a worthwhile project, but now-and-again it is nice to hear from people who seem to “get it”.

P.S. if anyone is reading this, I wish you a Happy New Year and I hope 2012 sends many good things your way! :)

--
Tagged: crunchbang linux

prabhupada01

2012-01-02T19:16:11+00:00

About a year ago I retired most of my computers, and am now left with only two: Loka, my old desktop (now with a new 23″ screen), and Savitar, a Thinkpad X41 I bought about 18 months ago.

Both computers run Debian Testing. My main desktop environments and window managers on Loka have been Xfce with Openbox and WindowMaker, which I rediscovered this year. On Savitar, I used only Openbox and Awesome.

Loka

Xfce 4.6 with Openbox, using the Alghattas Openbox and Gtk theme, and the Elementary icons. I don’t remember where I found the wallpaper.

Xfce 4.6 with Openbox, using the Alghattas Openbox and Gtk theme, the Elementary icons, and this wallpaper.

Xfce 4.8 with Openbox, using the Erthe Openbox and Gtk theme, the AnyColorYouLike icons, and part of this image as wallpaper.

Xfce 4.8 with Openbox, using the Erthe Openbox and Gtk theme, and the AnyColorYouLike icons.

WindowMaker, with the ShikiBlack WindowMaker theme, the Shiki-JetBlack Gtk theme, and the Moblin icons. I don’t remember where I found the wallpaper.

Savitar

Awesome, with the Children of the Earth Awesome theme, Gtk theme, and icons, and Caspar David Friedrich’s Monk am See.

Openbox with the Children of the Earth Openbox theme.

Brume_02

2012-01-02T18:19:46+00:00

I had written this a year ago, but somehow forgot to post it. But here are my screenshots from two years ago.

As I’ve been doing for a few years now, I am posting screenshots of my desktops of the past year, to amuse you (nearly everyone likes to look at screenshots), to document my changing aesthetics, and to capture my mood of the year as reflected in these images.

Below are the screenshots of the computers I’ve used in the past year. Aryaman is a Thinkpad 240 I bought on ebay for £10. I used it particularly when travelling, or when working in a library. I no longer use Mitra (my 11 year old Dell Inspiron 2500) and Yantra (my 10 year old Compaq Presario), but recently gave them away to others who found some good use for them. Loka is my old desktop.

Aryaman

Musca 0.9.24

Loka

Openbox 3.4.10 in Xfce 4.6, with the Fearless Openbox theme, the Umbra Gtk theme, and the Iris icon theme.

Pekwm 0.1.11, with the Elegance Pekwm and Gtk theme, and the Iris icon theme.

Pekwm 0.1.12, with an updated Groove Pekwm and Gtk theme, and the nuoveXT-1.7 icon theme.

Pekwm 0.1.12, with the Groove Pekwm and Gtk theme. The icons are from the Any Colour You Like icon theme.

Pekwm 0.1.12, with the Groove Pekwm and Gtk theme. The wallpaper is home made.

Openbox 3.4.10, with the Brume Openbox and Gtk theme.

Será, hein?!

2011-12-30T06:31:27+00:00

Tenho lá minhas dúvidas…

Plotting the Analemma

2011-12-30T04:00:54+00:00

My SJAA planet-observing column for January is about the Analemma and the Equation of Time.

The analemma is that funny figure-eight you see on world globes in the middle of the Pacific Ocean. Its shape is the shape traced out by the sun in the sky, if you mark its position at precisely the same time of day over the course of an entire year.

The analemma has two components: the vertical component represents the sun's declination, how far north or south it is in our sky. The horizontal component represents the equation of time.

The equation of time describes how the sun moves relatively faster or slower at different times of year. It, too, has two components: it's the sum of two sine waves, one representing how the earth speeds up and slows down as it moves in its elliptical orbit, the other a function the tilt (or "obliquity") of the earth's axis compared to its orbital plane, the ecliptic.

The Wikipedia page for Equation of time includes a link to a lovely piece of R code by Thomas Steiner showing how the two components relate. It's labeled in German, but since the source is included, I was able to add English labels and use it for my article.

But if you look at photos of real analemmas in the sky, they're always tilted. Shouldn't they be vertical? Why are they tilted, and how does the tilt vary with location? To find out, I wanted a program to calculate the analemma.

Calculating analemmas in PyEphem

The very useful astronomy Python package PyEphem makes it easy to calculate the position of any astronomical object for a specific location. Install it with: easy_install pyephem for Python 2, or easy_install ephem for Python 3.

import ephem
observer = ephem.city('San Francisco')
sun = ephem.Sun()
sun.compute(observer)
print sun.alt, sun.az

The alt and az are the altitude and azimuth of the sun right now. They're printed as strings: 25:23:16.6 203:49:35.6 but they're actually type 'ephem.Angle', so float(sun.alt) will give you a number in radians that you can use for calculations.

Of course, you can specify any location, not just major cities. PyEphem doesn't know San Jose, so here's the approximate location of Houge Park where the San Jose Astronomical Association meets:

observer = ephem.Observer()
observer.name = "San Jose"
observer.lon = '-121:56.8'
observer.lat = '37:15.55'

You can also specify elevation, barometric pressure and other parameters.

So here's a simple analemma, calculating the sun's position at noon on the 15th of each month of 2011:

    for m in range(1, 13) :
        observer.date('2011/%d/15 12:00' % (m))
        sun.compute(observer)

I used a simple PyGTK window to plot sun.az and sun.alt, so once it was initialized, I drew the points like this:

    # Y scale is 45 degrees (PI/2), horizon to halfway to zenith:
    y = int(self.height - float(self.sun.alt) * self.height / math.pi)
    # So make X scale 45 degrees too, centered around due south.
    # Want az = PI to come out at x = width/2.
    x = int(float(self.sun.az) * self.width / math.pi / 2)
    # print self.sun.az, float(self.sun.az), float(self.sun.alt), x, y
    self.drawing_area.window.draw_arc(self.xgc, True, x, y, 4, 4, 0, 23040)

So now you just need to calculate the sun's position at the same time of day but different dates spread throughout the year.

And my 12-noon analemma came out almost vertical! Maybe the tilt I saw in analemma photos was just a function of taking the photo early in the morning or late in the afternoon? To find out, I calculated the analemma for 7:30am and 4:30pm, and sure enough, those were tilted.

But wait -- notice my noon analemma was almost vertical -- but it wasn't exactly vertical. Why was it skewed at all?

Time is always a problem

As always with astronomy programs, time zones turned out to be the hardest part of the project. I tried to add other locations to my program and immediately ran into a problem.

The ephem.Date class always uses UTC, and has no concept of converting to the observer's timezone. You can convert to the timezone of the person running the program with localtime, but that's not useful when you're trying to plot an analemma at local noon.

At first, I was only calculating analemmas for my own location. So I set time to '20:00', that being the UTC for my local noon. And I got the image at right. It's an analemma, all right, and it's almost vertical. Almost ... but not quite. What was up?

Well, I was calculating for 12 noon clock time -- but clock time isn't the same as mean solar time unless you're right in the middle of your time zone.

You can calculate what your real localtime is (regardless of what politicians say your time zone should be) by using your longitude rather than your official time zone:

    date = '2011/%d/12 12:00' % (m)
    adjtime = ephem.date(ephem.date(date) \
                    - float(self.observer.lon) * 12 / math.pi * ephem.hour)
    observer.date = adjtime

Maybe that needs a little explaining. I take the initial time string, like '2011/12/15 12:00', and convert it to an ephem.date. The number of hours I want to adjust is my longitude (in radians) times 12 divided by pi -- that's because if you go pi (180) degrees to the other side of the earth, you'll be 12 hours off. Finally, I have to multiply that by ephem.hour because ... um, because that's the way to add hours in PyEphem and they don't really document the internals of ephem.Date.

Set the observer date to this adjusted time before calculating your analemma, and you get the much more vertical figure you see here. This also explains why the morning and evening analemmas weren't symmetrical in the previous run.

This code is location independent, so now I can run my analemma program on a city name, or specify longitude and latitude.

PyEphem turned out to be a great tool for exploring analemmas. But to really understand analemma shapes, I had more exploring to do. I'll write about that, and post my complete analemma program, in the next article.

MPD on Mac OS X

2011-12-27T19:12:37+00:00

My story really starts with Trackbox, an experiment of a perl script daemon I wrote long ago that worked like a headless jukebox daemon and allowed a client program to connect to it to provide a user interface to allow the user to browse the collection of music, manage playlists, play, pause, change volume, and all the other things you expect from a media player. It was pretty hacky, but it did the job. When Shank got MPD, the Music Player Daemon, working properly in raw C, though, I retired Trackbox. MPD is very similar to Trackbox: a super lightweight daemon that sits in the background, performing the duties of of music player. If I don’t need to interact with it, I don’t even need to have a client running — like when my linux tower boots and the init script starts it up. MPD resumes playing where it left off before I even launch X. I’ve obviously been a long time fan of even the idea of MPD.

I’ve been using a Mac increasingly over recent years. It started with a 15″ MacBook Pro for work, and eventually I got a 13″ one for personal use. My current and second Mac is a 13″ 2011 MacBook Air, and I’ve been frankly thrilled with it. Macs are great, but but I think a lot of folks endure the frustration of hunting for a great music player that simply doesn’t seem to exist.

Currawong started this excellent thread on head-fi which gives a really great overview and discussion on the most visible Mac music player options. I’ve tried most of them, but never really found one to my liking. Each was either buggy, lacking features, stupid expensive, or didn’t maintain a library, making practical use limited. I have gone back to iTunes every once in a while, and I’ve always been disappointed with crashing or general bloaty slowness. I finally went as far as to uninstall it from my system entirely, reclaiming hundreds of megabytes. (After removing the iTunes.app, I simply used the Preferences screen of “Software Update” to disable future update notifications.) I must say, I don’t miss it, not one little bit.

I finally decided to give MPD a go on the Mac. After using it for perhaps about a month now, I can happily say it is absolutely my new favorite. At home, I mount my server’s RAID array, and at work, I mount my off-site backup, a 3Tb external drive at my desk, so I always have my full collection of mostly lossless music readily at my disposal. I’ve even found a nice way to start mpd with different config files so I can aim it at the server or the local collection, depending on what I need. Being a global hotkey junkie, I’ve mapped restarting mpd with each config to a different key.

MPD is only as nice to use as the client you’re using. Theremin is nice native Mac one, but seems fairly unmaintained at this point. It scrobbles and has album art, but its main drawback for me is that it doesn’t allow browsing the collection by directory structure like most clients. I would recommend giving this one a try; it may fit the bill. There are countless others to choose from, though. I managed to get the excellent, GTK-based Sonata client to run under OS X with the help of MacPorts, but it is a bit kludgey. Client175 is a very nice, web-based one.

I’ve actually found myself liking the terminal-based ncmpcpp client best. It is jam-packed with features, remains blazingly quick, and even has the ability to add a random album, artist, or track to the playlist! I can understand why many folks might not like using a console client, but it works really well for me. Bind it to pop up with a global hotkey, and you have a slick, quick-launching client that you don’t even need running most of the time!

The other client I use is mpc, the command-line interface client. I barely ever actually use it on the command line, though, but through global hotkeys to toggle (play/pause) and play next track. I’ve also written a wrapper for mpc that allows me to play (or append to the playlist) random tracks on random albums. The most unique feature of my tool is the ability to specify “top-level” directories by a pre-designated “short code” and add random tracks or albums only from within one top-level dir at a time. The main directories of my collection are genres, so I can throw on some random ambient music or random drum ‘n bass. For example, with my Alfred extension, I do my ⌘+space to open him up, and type, “rt db” — immediately 10 random drum ‘n bass tracks are added, and the first one plays.

Honestly, this has revolutionized how I listen to my music. I often don’t even have the patience to drill down my folder structure or otherwise hunt for something to listen to. Now I can tell alfred “rt ch” and have random chill out beats. If I want to hear the entire album for one of the songs, ⌘⌥5 will play it.

I use and adore Alfred for many things including setting up these global hotkeys, and I feel it is worth plenty more than what Andrew is asking for it. But if you want something that is free, I recommend BetterTouchTool. Here is the key configuration I use on my Air:

⌘⌥1: play/pause toggle
⌘⌥2: next track
⌘⌥3: play 10 random tracks from entire collection (hold shift to append)
⌘⌥4: play 1 random album from entire collection (hold shift to append)
⌘⌥5: play the album the currently playing song belongs to (hold shift to append)
or all the above, holding Control as well causes the command to be directed at my server, which drives the living room speakers/headphone system
Control+⌥+n: launch ncmpcpp under iTerm2

(Note: I use Control+⌥+letter to launch / switch focus to my most often used apps. This makes me very efficient and I love it.)

MPD does the trick for me. It is slim, efficient, and invisible. I can do 90% of everything I need with a simple global hotkey, and Growl gives me feedback. When I actually do want to browse around or manage the playlist, ncmpcpp or other client lets me do that. There are some folks who look for an “audiophile” player, but the fact is that if the program can decode the file and feed it to the audio device in a “bit perfect” manner, every app should sound the same. I think there are apps that are broken and don’t do this quite right, and I also think there are people who get fall victim to the placebo effect. I really see no reason to worry about the sound quality of MPD — I trust it.

I’ve uploaded my mpct.php script to GitHub in case anyone is interested in it. If you use it, I would love to hear about your experience. If you have any questions about any of this, I would really love to hear from you in the comments!

Installing MPD

Installing MPD on OS X is really easy according to their instructions. Simply install homebrew with

/usr/bin/ruby -e "$(curl -fsSL https://raw.github.com/gist/323731)"

then use

brew install mpd

I really like this brew software manager! I can even install packages they don’t provide with something like

brew create http://unkart.ovh.org/ncmpcpp/ncmpcpp-0.5.8.tar.bz2

Calculating the Solstice and shortest day

2011-12-22T18:30:18+00:00

Today is the winter solstice -- the official beginning of winter.

The solstice is determined by the Earth's tilt on its axis, not anything to do with the shape of its orbit: the solstice is the point when the poles come closest to pointing toward or away from the sun. To us, standing on Earth, that means the winter solstice is the day when the sun's highest point in the sky is lowest.

You can calculate the exact time of the equinox using the handy Python package PyEphem. Install it with: easy_install pyephem for Python 2, or easy_install ephem for Python 3. Then ask it for the date of the next or previous equinox. You have to give it a starting date, so I'll pick a date in late summer that's nowhere near the solstice:

>>> ephem.next_solstice('2011/8/1')
2011/12/22 05:29:52

That agrees with my RASC Observer's Handbook: Dec 22, 5:30 UTC. (Whew!)

PyEphem gives all times in UTC, so, since I'm in California, I subtract 8 hours to find out that the solstice was actually last night at 9:30. If I'm lazy, I can get PyEphem to do the subtraction for me:

ephem.date(ephem.next_solstice('2011/8/1') - 8./24)
2011/12/21 21:29:52

I used 8./24 because PyEphem's dates are in decimal days, so in order to subtract 8 hours I have to convert that into a fraction of a 24-hour day. The decimal point after the 8 is to get Python to do the division in floating point, otherwise it'll do an integer division and subtract int(8/24) = 0.

The shortest day

The winter solstice also pretty much marks the shortest day of the year. But was the shortest day yesterday, or today? To check that, set up an "observer" at a specific place on Earth, since sunrise and sunset times vary depending on where you are. PyEphem doesn't know about San Jose, so I'll use San Francisco:

>>> import ephem
>>> observer = ephem.city("San Francisco")
>>> sun = ephem.Sun()
>>> for i in range(20,25) :
...   d = '2011/12/%i 20:00' % i
...   print d, (observer.next_setting(sun, d) - observer.previous_rising(sun, d)) * 24
2011/12/20 20:00 9.56007901422
2011/12/21 20:00 9.55920379754
2011/12/22 20:00 9.55932991847
2011/12/23 20:00 9.56045709446
2011/12/24 20:00 9.56258416496

I'm multiplying by 24 to get hours rather than decimal days.

So the shortest day, at least here in the bay area, was actually yesterday, 2011/12/21. Not too surprising, since the solstice wasn't that long after sunset yesterday.

If you look at the actual sunrise and sunset times, you'll find that the latest sunrise and earliest sunset don't correspond to the solstice or the shortest day. But that's all tied up with the equation of time and the analemma ... and I'll cover that in a separate article.

Fundamental-Hybrid para XFCE

2011-12-19T03:21:00+00:00

Tema pedido pelo netonardin.

Download @ deviantART