Systems: Because You Can’t Count That Fast


Me trying to explain a global SAP ERP network to kids.

On Thursday, it was bring your daughters and sons to work day at Genentech. My little Marlowe is only 6 weeks old so I didn’t bring her, but my VP shot me a note Wednesday saying she was bringing her two boys, knew of some other kids who would be there, and wondered if I could give a 30 minute talk on systems. I responded, “sure.”

But why the heck do you tell kids ranging from 6-12 years old about your corporate ERP system? While fascinating, I doubt they’d care about the usual things that I work on and I wouldn’t dare show them PowerPoint slides. So what to do?
Continue reading

Cleanup Unused Linux Kernels in Ubuntu

I update Ubuntu with a very simple script I call apt-update that looks like this:

$ cat ./apt-update 
sudo apt-get update; sudo apt-get dist-upgrade; sudo apt-get autoremove

Nothing too crazy there. It updates the apt-get cache, performs the upgrade, and then removes all the residual junk that’s laying around. Well, almost all. If you do this enough, eventually you’ll see the following (assuming you’ve got the default motd Ubuntu script running and you’re logging in from a terminal):

=> /boot is using 86.3% of 227MB

This is because that script I mentioned doesn’t consider old kernel images to be junk. However, unless you’ve got an abnormal /boot partition, it doesn’t take too many old images to fill it up.

A quick Google search found Ubuntu Cleanup: How to Remove All Unused Linux Kernel Headers, Images and Modules. The solution on the page had exactly what I’m looking for, however, I couldn’t take it at face value. While the article offers an adequate solution, it doesn’t offer much explanation. The remainder of this article explains the details for this one-liner noted in the article above:

$ dpkg -l 'linux-*' | sed '/^ii/!d;/'"$(uname -r | sed "s/\(.*\)-\([^0-9]\+\)/\1/")"'/d;s/^[^ ]* [^ ]* \([^ ]*\).*/\1/;/[0-9]/!d' | xargs sudo apt-get -y purge

Note: Only run this if you’ve rebooted after installing a new kernel.

Ick. Let’s dig into what’s going on here. The pipe characters are chaining a bunch of commands together. Each command’s output becomes the input for the next. Given that, let’s walk through what’s going on in 3 steps.
Continue reading

Controlling Movies in QuickTime Player X (10.1) With AppleScript

The following works for Quicktime Player X (10.1) in OS X 10.7 (Lion). It takes arguments from the command line and opens a movie, resizes it, sets the volume, and starts playing the movie (there are other properties you could add). Hopefully this is useful for someone like me who was trying to find examples of this, but only finding mounds of info for QuickTime Player 7 (and earlier), which doesn’t help much as many of the properties have changed.

-- AppleScript to open and control movie files in QuickTime Player X

-- The script is meant to run at the command line, like so:
-- $ osascript /path/to/movie.mp4 movie.mp4 800 60
-- If you don't want to use a shell, just remove this and 
-- the last line and manually populate the first 4 variables
on run argv
	
	set movieFile to item 1 of argv -- the full path with movie filename
	set movieName to item 2 of argv -- the file name only of the movie
	set desiredWidth to item 3 of argv -- the width of the movie on the screen
	set startOffset to item 4 of argv -- how many seconds into the movie to start
	
	-- this finds the right side of the screen, then offsets to the left
	-- the desired width of the movie
	tell application "Finder"
		set desktopBounds to bounds of window of desktop
	end tell
	set rightX to item 3 of desktopBounds
	set leftX to (rightX - desiredWidth)
	
	tell application "QuickTime Player"
		
		-- open the movie and bring it to the forefront (if other movies are open)
		open movieFile
		set lastOpenedWindow to (first window whose name contains movieName)
		set visible of lastOpenedWindow to true
		
		-- resize the movie using its original aspect ratio
		set movieBounds to bounds of lastOpenedWindow
		set widthOrigin to leftX
		set heightOrigin to 1
		set originalWidth to ((item 3 of movieBounds) - (item 1 of movieBounds))
		set originalHeight to ((item 4 of movieBounds) - (item 2 of movieBounds))
		set calculatedHeight to (round ((desiredWidth / originalWidth) * originalHeight) rounding down)
		set desiredWidth to (desiredWidth + widthOrigin)
		set calculatedHeight to (calculatedHeight + heightOrigin)
		set the bounds of lastOpenedWindow to {widthOrigin, heightOrigin, desiredWidth, calculatedHeight}
		
		-- if opening multiple movies, ensure they don't overlap
		set slideDown to 1
		repeat with aWindow in (get every window)
			set boundsOfAWindow to bounds of aWindow
			set heightOfAWindow to item 4 of boundsOfAWindow
			if (heightOfAWindow > slideDown) then
				if (name of aWindow does not contain movieName) then
					set slideDown to (heightOfAWindow + 1)
				end if
			end if
		end repeat
		set the bounds of lastOpenedWindow to {widthOrigin, (heightOrigin + slideDown), desiredWidth, (calculatedHeight + slideDown)}
		
		-- note, here we select a document, not a window
		set myMovie to document 1
		tell myMovie
			set audio volume to 0
			set current time to startOffset
			play
			activate -- makes QuickTime Player the frontmost application
		end tell
		
	end tell
end run

Integer Compression in Ruby (Base-10 to Base-62)

A few days ago I was thinking about all those link shortening sites and wondered how easy it would be to compress a base-10 number like 1,234,567,890 to something much smaller like 1LY7VK. Here’s what I came up with:

class IntegerCompressor
  
  CompressionCharacterSet = %w(0 1 2 3 4 5 6 7 8 9
  A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
  a b c d e f g h i j k l m n o p q r s t u v w x y z)
  
  def self.to_base
    CompressionCharacterSet.length
  end

  def self.compress(number_to_convert)
    digits_needed = Math.log(number_to_convert, IntegerCompressor.to_base).floor + 1
    compressed_number_string = ''
    previous_remainder = number_to_convert
    (digits_needed-1).downto(0) do |power|
      r=previous_remainder.divmod(IntegerCompressor.to_base**power)
      compressed_number_string << CompressionCharacterSet[r[0]]
      previous_remainder = r[1]
    end
    compressed_number_string
  end
  
  def self.decompress(compressed_number)
    power = 0
    base_10_integer = 0
    compressed_number.to_s.reverse.each_char do |digit|
      base_10_integer += ((IntegerCompressor.to_base**power)*CompressionCharacterSet.index(digit))
      power+=1
    end
    base_10_integer
  end
  
end

It seems to work:

$ irb
>> require '/path/to/file/integer_compressor.rb'
=> true
>> IntegerCompressor.compress 1234567890
=> "1LY7VK"
>> IntegerCompressor.decompress "1LY7VK"
=> 1234567890

If anyone has a more elegant solution, I'd be curious to see it.

(Thanks to Tamara for the log refresher.)

Symbolic Links to Image Folders in Rails 3.1

I recently tweaked the code for OmniNerd.com to upgrade it from a Rails 2.3 app to a Rails 3.1 app.  Online docs got me through everything but one item: symbolic links to folder of images (or assets in general).  On OmniNerd, users can upload images.  The site also auto-generates a lot of graphical content.  Our setup is like many, i.e., we deploy with Capistrano and use the shared folder to hold static user content that gets handed off between deployments.  We previously simply had a symlink setup such that:

/path/to/app/current/public/images/content --> /path/to/app/shared/user_uploaded

Continue reading

Ruby on Rails Diff Text to HTML and

This code is perfect if you have 2 text objects in your Rails application and you want to compare their differences in one of your HTML views. It’s 99% pure Ruby too, so if you alter the first line, you can use it for other purposes.

Only one thing to note: you must have diff installed. I’m using: diff (GNU diffutils) 2.8.1.

#set up some variables to reference later
temporary_directory = File.join(Rails.root, "tmp")
max_lines = 9999999 #needs to be larger than the most lines you'll consider
diff_header_length = 3

# text_old and text_new should be the values of the string objects to compare
# these are just example strings to show it works
text_old      = "line1\ndeleted line2\nline3\n\nline4\nline5"
text_new      = "line1\ninserted line2\nline3\n\nline4\nline5"

# since we're using diff on the file system, we'll save the text we want to compare
# and then run diff against the two files
file_old_name = File.join(temporary_directory,"file_old"+rand(1000000).to_s)
file_new_name = File.join(temporary_directory,"file_new"+rand(1000000).to_s)
file_old      = File.new(file_old_name, "w+")
file_new      = File.new(file_new_name, "w+")
file_old.write(text_old+"\n")
file_new.write(text_new+"\n")
file_old.close
file_new.close

# diff will give provide a string showing insertions and deletions.  We will
# split this string out by newlines if there are difference, and mark it up
# accordingly with html
lines = %x(diff -­-­­­­­­unified=#{max_lines} #{file_old_name} #{file_new_name})
if lines.empty?
  lines = text_new.split(/\n/)
else
  lines = lines.split(/\n/)[diff_header_length..max_lines].
  collect do |i|
    if i.empty?
      ""
    else
      case i[0,1]
      when "+"; then "<ins>"+i[1..i.length-1]+"</ins>"
      when "-"; then "<del>"+i[1..i.length-1]+"</del>"
      else; i[1..i.length-1]
      end
    end
  end
end

#clean up the temporary diff files we created
File.delete(file_new_name)
File.delete(file_old_name)

#return marked up text
lines.join("\n")</pre>
If you fire up RAILS_ROOT/script/console and paste that code in, it will return a nicely marked up string like this:
<pre lang="html">line1
<del>deleted line2</del>
<ins>inserted line2</ins>
line3

line4
line5

Use CSS to make your ins and del tags render however you like.

Skip All Rails Filters

It took me a while to figure this out, but it’s quite simple.  If you want to skip all of the filters a Rails controller will run, simply put the following at the top of your controller:

skip_filter filter_chain #both documented in the Rails API

For example, if your application controller defines a filter to check if a user is logged in, it makes sense that this filter might run for all controllers, except in rare cases.  In my case, I have a dynamic image controller that doesn’t require all of the overhead that most controllers do.  For that controller, I use the above to skip all of the filters.

Ruby On Rails RSS Reader

We moved our Athlo blog to a WordPress app to separate it completely from the main app. One interaction I wanted between the two though was that I wanted the most recent blog entries to show on the Athlo site. I thought that RSS would offer an easy solution so I started looking around to find out if I’d need a Rails plugin or something like that.

The solution was far simpler. And pure Ruby (man I love this language!).

require 'rss'
rss = RSS::Parser.parse(open('http://blog.athlo.com/feed/').read, false).items[0..MaxRSSItems-1]

That’s it. That simple call supplies you with a full array of all the items from the RSS feed. In my specific example, I’ve used a range to limit the results to the value of MaxRSSItems.

No plugins required. No Rails required. Ruby RSS will do what you need to read feeds. (That should be in a poem.)