Unixmonkey.net

I’ve been playing around with Shoes (shoooes.net) lately as a way to put a cross-platform graphical user interface (GUI) on some of my small purpose-built command-line ruby scripts.

I find that it is quite easy to get started with, and lends a lot of flexibility to the way your program is structured and displayed. However, the structure feels a little bit alien compared to everyday ruby, and there are some gotcha’s you need to keep in mind while developing for Shoes.

I feel I must preface this article by saying that Shoes has excellent documentation, _why (the lucky stiff) turns documentation into its own art form. The manual, “Nobody Knows Shoes” reads a lot like a comic book, full of _why’s own original artwork and clippings from old-timey photos and art, and is complimented by the documentation at help.shoooes.net

I had a bit of trouble at first getting ActiveRecord to interface with a database from a straight port from one of my console apps because I glossed over the parts of the manual that detail the tricky behavior of the garbage collector reaping predefined classes after the app’s initial load.

The fix is pretty simple. Stick all your classes in an external file (or many) and load them using ‘require’.

Anyhow, here is a barebones example of a working implementation for using ActiveRecord in Shoes:

# in foo.rb
class Foo < ActiveRecord::Base
end
 
#in app.rb
Shoes.setup do
  gem 'activerecord'
  require 'active_record'
  ActiveRecord::Base.establish_connection(
    :adapter   => 'sqlite3',
    :dbfile    => 'foos_db.sqlite3'
  )
  require 'foo'
end
Shoes.app do
  @foos = Foo.find(:all)
  para @foos
end

Now, this example requires there is an existing sqlite database with a foos table, change out the establish_connection parameters to connect to any other database. The gem ‘activerecord’ statment tells shoes to install the activerecord gem into the shoes ruby library if it isn’t already there.

If you don’t already have a database, and just want to use a db to act as a storage layer for your app, then you might want to use ActiveRecord::Schema.define to create a database and setup the tables the same way you do for Rails migrations.

Here is a more complete example of an app to keep track of notes using ActiveRecord as the backend. I like the “base class that inherits from Shoes” pattern, so I’m using that here.

# in note.rb
class Note < ActiveRecord::Base
end
 
# in app.rb
Shoes.setup do
  gem 'activerecord' # install AR if not found
 
  require 'active_record'
  require 'fileutils'
 
  ActiveRecord::Base.establish_connection(
    :adapter   => 'sqlite3',
    :dbfile    => 'shoes_app.sqlite3'
  )
 
  # create the db if not found
  unless File.exist?("shoes_app.sqlite3")
    ActiveRecord::Schema.define do
      create_table :notes do |t|
        t.column :message, :string
      end
    end
  end
 
end
 
class ShoesApp < Shoes
  require 'note'
 
  url '/', :index
 
  def index
    para 'Say something...'
    flow do
      @note = edit_line
      button 'OK' do
        Note.new(:message => @note.text).save
        @note.text = ''
        @result.replace get_notes  
      end
    end
    @result = para get_notes
  end
 
  def get_notes
    messages = []
    notes = Note.find(:all, :select => 'message')
    notes.each do |foo|
      messages << foo.message
    end
    out = messages.join("n")
  end
 
end
 
Shoes.app :title => 'Notes', :width => 260, :height => 350

Here’s a screenshot:

There you are; a cross-platform desktop app that doesn’t require a full-on build environment, and can be distributed with the source exposed for later improvements.

The first time this runs, it installs Activerecord, requires it, establishes a connection, creates the table unless one already exists. Then it shows a form to add notes followed by all the existing notes in the database. Adding a new note refreshes the notes shown.

This isn’t exactly a polished app with full CRUD, but should prove a good introduction to Shoes for someone used to working with ActiveRecord.

I’ve got some data in an excel file that I need to put in the database and its far too much to do by hand, what will I do?

Lets throw some ruby at the problem!

First, excel it too darn complicated and proprietary a format to even mess with unless you are creating something really worth it, so lets open that .xls with Excel or OpenOffice and do a File -> Save As -> .csv (comma separated values) to get a file that is easier to work with.

Now, we could write our own CSV parser since its such a simple format, but why futz with it when someone else has already put out a good library for that will likely be more error tolerant? Lets use FasterCSV, as its pretty well-known.

Install by issuing:

sudo gem install fastercsv

Now you can just fire up script/console of your Rails app and type in the below, or just put this in a database migration to slurp up all that good spreadsheet data.

The below assumes you have a ‘users’ table with fields name, address, and email that are also rows in your excel file. Adjust as necessary.

require 'fastercsv'
FasterCSV.foreach("#{RAILS_ROOT}/myfile.csv") do |row|
  record = User.new(
    :name    => row[0], # first column of csv file
    :address => row[1], # second column
    :email   => row[2]  # third
  )
  record.save
end

That’s pretty awesome; now how can I export that stuff in the database back out to Excel again?

Lets use Ruport, the Ruby report gem!

sudo gem install acts_as_reportable

Toss the require statement somewhere obvious (like environment.rb or above the model you want to export), and put ‘acts_as_reportable’ in your model declaration.

require 'ruport'
class User < ActiveRecord::Base
  acts_as_reportable
end

Now I can do this kind of stuff to export to a csv file (again with script/console, but a migration should work equally well):

content = User.report_table.as(:csv) # convert your model table to CSV

or

content = User.report_table_by_sql("SELECT name, address, email FROM users").as(:csv)

Then write that to a file like so:

file = File.open("#{RAILS_ROOT}/report.csv", "w") # open file
file.print(content) # print that csv content to the open file
file.close          # close the file

Open that CSV file with Excel and amaze the sales team, your boss, or whoever.

This is a narrow view of what we can do with FasterCSV and Ruport, but I’m sure you can see how you could build out a format.csv in a respond_to block in a Rails controller, or have a setter in your model that sucks in an uploaded CSV to create some records.

These are some pretty great libraries, and I’m very glad they were able to help me load, combine, query and output some data I’d been working with in a pinch.

I hope this post serves to help someone else in a similar situation.

The web is a generally free and open place for all types of communication, but if you put your email address on 1 website, you can expect an email-harvesting robot spider to find that address and send it to its spammer overlords.

Once on a spammer’s list, you can expect to get all kinds of interesting stock tips, products to enhance your manhood, and friendly letters from Nigerian diplomats.

If you simply have too little to do in the day, this can be a great way to meet new people and start a career in day trading. However, some of us are just too darn busy to stop what we are doing every 2/3rds of a second to check our email; but still need it for keeping in contact with friends, family, and business contacts.

From a few tips pulled from the web, I set to create a nice link helper for Ruby / Rails intended to display email links that work indistinguishably from regular mailto: links, and even gracefully downgrade for users without javascript.

Lets not even display the email address on the page at all, and use a little javascript to render the email address after the fact by breaking it up and putting it back together with javascript.

# Takes in an email address and (optionally) anchor text,
# its purpose is to obfuscate email addresses so spiders and
# spammers can't harvest them.
def js_antispam_email_link(email, linktext=email)
    user, domain = email.split('@')
    # if linktext wasn't specified, throw email address builder into js document.write statement
    linktext = "'+'#{user}'+'@'+'#{domain}'+'" if linktext == email 
    out =  "<noscript>#{linktext} #{user}(at)#{domain}</noscript>n"
    out += "<script language='javascript'>n"
    out += "  <!--n"
    out += "    string = '#{user}'+'@'+'#{domain}';n"
    out += "    document.write('<a href='+'ma'+'il'+'to:'+ string +'>#{linktext}</a>'); n"
    out += "  //-->n"
    out += "</script>n"
    return out
end

This is probably good enough for 90% of those robots, but you know if one spammer gets your address, he will likely share (or sell) your email to all his friends. The weak spot in this looks like the noscript version, lets fuzz that up a bit by converting to HTML character entities.

One of the earliest and simplest ways to obfuscate an email address is by converting each character into its HTML equivalent. This makes the source look nasty, but will be correctly rendered by the browser that the end-user is none the wiser.

An address like abc@example.com will look like this in the source:

abc@example.com

Let’s build a simple method to convert a plaintext string into something like the above. I’m going to cheat and only convert a-z and A-Z and leave @ signs, dots, dashes, etc. alone.

# HTML encodes ASCII chars a-z, useful for obfuscating
# an email address from spiders and spammers
def html_obfuscate(string)
  output_array = []
  lower = %w(a b c d e f g h i j k l m n o p q r s t u v w x y z)
  upper = %w(A B C D E F G H I J K L M N O P Q R S T U V W X Y Z)
  char_array = string.split('')
  char_array.each do |char|  
    output = lower.index(char) + 97 if lower.include?(char)
    output = upper.index(char) + 65 if upper.include?(char)
    if output
      output_array << "&##{output};"
    else 
      output_array << char
    end
  end
  return output_array.join
end

now in our js_antispam_email_link method we can “encrypt” the user and domain before sending to the browser like so:

def js_antispam_email_link(email, linktext=email)
  user, domain = email.split('@')
  user = html_obfuscate(user)
  domain = html_obfuscate(domain)
  ...

Not bad, but many spiders these days can still decode HTML entities and get at that address, so lets build up our defenses a bit more by adding some methods to really screw with those spiders.

We’ll write a method that encrypts a string with ROT13 and puts that on the webpage, and use some javascript to decrypt that on page display. ROT13 is a really simple cipher where you take characters a-z and shift them by half the alphabet.

This is a really simple one-liner borrowed from Jay Komineck

# Rot13 encodes a string
def rot13(string)
  string.tr "A-Za-z", "N-ZA-Mn-za-m"
end

Lets use this to really beef up our link helper by using some javascript that can decipher this. JS code taken from Allan Odgaard

string = '#{email}'.replace(/[a-zA-Z]/g, 
  function(c){ 
    return String.fromCharCode(
      (c <= 'Z' ? 90 : 122) >= (c = c.charCodeAt(0) + 13) ? c : c - 26
    );
  });

Now we’ve got some pretty strong defense against those pesky robots and by using simple HTML character encoding and lightweight ROT13 ciphering it shouldn’t be too taxing on your webserver to spit out a page with a few emails on it. Less sophisticated browsers still get the contact info and everyone is a little bit happier to come home to a (relatively) clean inbox.

Here’s the whole shebang put together, put this in application_helper.rb if using rails:

# Rot13 encodes a string
def rot13(string)
  string.tr "A-Za-z", "N-ZA-Mn-za-m"
end
 
# HTML encodes ASCII chars a-z, useful for obfuscating
# an email address from spiders and spammers
def html_obfuscate(string)
  output_array = []
  lower = %w(a b c d e f g h i j k l m n o p q r s t u v w x y z)
  upper = %w(A B C D E F G H I J K L M N O P Q R S T U V W X Y Z)
  char_array = string.split('')
  char_array.each do |char|  
    output = lower.index(char) + 97 if lower.include?(char)
    output = upper.index(char) + 65 if upper.include?(char)
    if output
      output_array << "&##{output};"
    else 
      output_array << char
    end
  end
  return output_array.join
end
 
# Takes in an email address and (optionally) anchor text,
# its purpose is to obfuscate email addresses so spiders and
# spammers can't harvest them.
def js_antispam_email_link(email, linktext=email)
  user, domain = email.split('@')
  user   = html_obfuscate(user)
  domain = html_obfuscate(domain)
  # if linktext wasn't specified, throw encoded email address builder into js document.write statement
  linktext = "'+'#{user}'+'@'+'#{domain}'+'" if linktext == email 
  rot13_encoded_email = rot13(email) # obfuscate email address as rot13
  out =  "<noscript>#{linktext}<br/><small>#{user}(at)#{domain}</small></noscript>n" # js disabled browsers see this
  out += "<script language='javascript'>n"
  out += "  <!--n"
  out += "    string = '#{rot13_encoded_email}'.replace(/[a-zA-Z]/g, function(c){ return String.fromCharCode((c <= 'Z' ? 90 : 122) >= (c = c.charCodeAt(0) + 13) ? c : c - 26);});n"
  out += "    document.write('<a href='+'ma'+'il'+'to:'+ string +'>#{linktext}</a>'); n"
  out += "  //-->n"
  out += "</script>n"
  return out
end

I hope this helps out somebody out there, please leave a comment if you have any suggestions.

If you tried to update rubygems today like me, its possible you got hit with a nasty error message running gem after the successful update to rubygems-1.0.1.

root@localhost ~:#gem -v
/usr/bin/gem:23: uninitialized constant Gem::GemRunner (NameError)

gem_runner needs loading, so lets fix that by adding a require at the top of/usr/local/lib/site-ruby/1.8/rubygems.rb

...
require 'rubygems/rubygems_version'
require 'rubygems/defaults'
require 'rubygems/gem_runner' ## add this line
require 'thread'
...

There we go, now lets try:

root@localhost ~:# gem -v
1.0.1

Hooray! Working rubygems!

Update:

I found this is due to a problem with just the apt packages, and re-downloading and installing the tar.gz from rubyforge will also solve the problem.