Installing Rails 2.5.0 on Mac OS High Sierra

Since I've just switched to my new laptop running High Sierra, I thought it would be good to update my rails software. The latest version is 2.5.0. I downloaded it and compiled it and got an error message when I tried to install rails.

yo:~ $ gem install rails
ERROR:  While executing gem ... (Gem::Exception)
    Unable to require openssl, install OpenSSL and rebuild Ruby (preferred) or use non-HTTPS sources

I'm using macports, so I installed openssl with:

sudo port install openssl

Then I had to play around a bit to get the ruby installation to find this openssl. The winning command turned out to be:

yo:ruby-2.5.0 $ ./configure --prefix=/Users/maryh/Software/rubies/2.5.0 --with-opt-dir=/opt/local
yo:ruby-2.5.0 $ make
yo:ruby-2.5.0 $ make install

Uploading a File in a Rails Site to Use to Update

I want to upload a file to my rails webpage, but not store it in the database. I just want to upload it and then get the data out of it and update the database with the information in the uploaded file.

I have a settings controller that I use for most administrative tasks. So this is where I'm going to put the form for the file upload.


<%= form_tag({action: :upload}, multipart: true) do %>
	Upload xml file of abstracts: <%= file_field_tag 'abstracts' %>
<%= submit_tag %> <% end %>


 resources :settings, only: [:index, :edit, :update] do
    post 'upload', on: :collection


  def upload
    file_data = params[:abstracts].tempfile
    flash[:notice] = "File uploaded"
    redirect_to settings_path

Then the method match_abstracts is where I do all the work I want to do on the file. Right now, all I have is the information matching names in the uploaded file with names stored in the database.

  def match_abstracts(file_data) do |file|
      @doc = Nokogiri::XML(file)
      elems = @doc.xpath("//Speaker")
      @applicants = Applicant.without_abstract
      @applicants.each do |applicant|
        elems.each do |e|
            if (('FirstName').text == applicant.firstname) && ('FamilyName').text == applicant.lastname))
     "TITLE: #{'Title').text}"
     "ABSTRACT: #{'Content').text}"

This just prints to the log the information that I'll need to use to update the database. Next I need to write another method, probably called update_applicant with the commands to add this information to the applicant's record.

Using Ruby on XML Files

I need to copy some data from one system to another. On the first system, I can download an xml file. The second system is a ruby on rails website. So I'd like to them upload the xml file to the site and have it enter in the data. But since I'm not that familiar with working with xml files, I thought I'd work on it from the command line first.

In ruby, there's a gem called nokogiri which works with xml (or html) files. Here's the sample xml file that I'm using:

  Diversity, diversity
  This is all about diversity.
  This is all about diversity.
   [email protected]
   [email protected]
  Diversity and Inclusion 
  Diversity and Inclusion 
  Higgs, higgs, higgs
  I've got *something to say* out the **Higgs.**
  I've got *something to say* out the **Higgs.**
   [email protected]
   The White House
   [email protected]
   The White House
  Higgs Physics
  Higgs Physics
  Oh the Lovely Higgs
  Some interesting info about how pretty the Higgs is.
  Some interesting info about how pretty the Higgs is.
   [email protected]
   [email protected]
  Higgs Physics

And here's the script I wrote:


require 'nokogiri'"Abstracts.xml") do |file|
	@doc2 = Nokogiri::XML(file)
	elems = @doc2.xpath("//Speaker")
	myfirst ='Abe'
	mylast = 'Lincoln'
	elems.each do |e|
		if ('FirstName').text == myfirst) && ('FamilyName').text == mylast)
			print "#{'FirstName').text} " 
			print "#{'FamilyName').text}\n"


First, I open the file and open it as an xml file with nokogiri. I then put in elems all the xml fields labelled "Speaker". I then cycle through elems looking for the string "Abe" and "Lincoln". If I find them both, I print out the field "Title" and "Content" which are both in the parent field from "Speaker" and then I print out the first and family names.

This is very basic and there are a lot more things that I need to do. I need to match the speaker info with names stored in a model in my rails app. And then update that entry with the title and content if I find a match.

Using Ruby on CSV Files

I'm working with some different systems for a conference I'm working on. We're using RegOnline to handle the registrations. One of the things that I need to do is get a list of people who have registered on RegOnline and post them on our conference website. Our conference site is plain html and I don't want to have to type in each person who registers. I can download a csv file from RegOnline. I want to write a script that reads the downloaded csv file and makes an html file for the website. I hadn't done much with csv files before, but here's a basic script that pretty much does what I want. (I still have to put html tags in, but that should be pretty easy.)

require 'csv'

unless ARGV.count == 1
	puts "SYNTAX:  generate_list_of_participants "
	puts ""
	puts "Expecting the csv file downloaded from regonline."

inputfile = ARGV.first

my_csv ="#{inputfile}", { headers: true })
# my_csv now has an array where each line in the file is an entry
# headers: true means the first line is headers and we don't want them
my_csv.each do |e|
	if e[4] == 'Other'
		puts "#{e[1]} #{e[2]} #{e[5]} #{e[6]}"
		puts "#{e[1]} #{e[2]} #{e[4]} #{e[6]}"

The first part of the script is just giving the syntax and checking that there's a filename on the command line. The second part, reads the entire file into and array (of arrays) called my_csv. The bit headers:true means that the first line in the csv file is a header row and it won't read that line in.

Then I just cycle through the array and print out the fields that I want. My csv file has a job in field e[4]. If it's other, I want to print what they typed in for their job (in field e[5]) instead of the 'Other' string.

That's pretty much it. There are some other things that I need to do, but to get started, this does the job.

Things to do:
-sort the array in alphabetical order on field e[2] (lastname)
-check that the attendee has actually paid, if they're submitting a check or wire transfer

Paperclip with Spoof File Checks

I have a rails site that accepts pdf files as uploads. Based on previous experience, I have found that trying to validate on content-type causes lots of problems. There are many browsers that send weird content types with file names. For pdf files, they should always be using 'application/pdf' and most browsers do. However, this morning I got email from someone who couldn't upload his file in one browser. He told me he used a different browser and it then worked. I searched the log files and found a bunch of error messages, but the main bit was this:

INFO -- : [paperclip] Content Type Spoof: Filename Tom_Smith_CV.pdf (x-unknown/pdf from Headers,
 content type discovered from file command: application/pdf. See documentation to allow this combination.

If I understand this correctly (and there's no guarantee of that), it's saying that the file is being uploaded with a content-type of x-unknown/pdf, but the file command (file --mime-type -b <file>) is returning application/pdf.

I believe that this mismatch is caused by the client browser being configured incorrectly. (And yes, it is. Check out this website.) Why? Because I was able to duplicate the problem by editing mimeTypes.rdf in my firefox profile directory. I simply changed anything that showed application/pdf to x-unknown/pdf and restarted the browser. I then got the same error.

This check is part of a new spoof detector in Paperclip version 4 and up. (I'm using 4.3.2.) Now based on lots of problems I've had before with validating content type, I put the do_not_validate_attachment_file_type :cv line in my model. Then I write a special validator that basically just checks the filename that it's a pdf file. This isn't ideal, but it has worked for me, until now.

This page explained that the spoof detector is not affected by the do_not_validate_attachment_file_type helper. That same page also gave me the workaround to turn off the spoof detector for now. I made a new initializer called paperclip.rb that contains:

require 'paperclip/media_type_spoof_detector'
module Paperclip
  class MediaTypeSpoofDetector
    def spoofed?

This basically just returns a false value for anything checking against the spoof detector.

I realize that none of this is ideal. However, I really don't understand things enough to know how to set up paperclip to properly do file validations while also accepting all the files that we should accept.

Declarative Authorization attribute rules

I have a website where I use declarative authorization to determine who is allowed to do what. One thing that I need to allow is for people to change their own passwords. It's pretty simple, my model/controller is users. Here's what I needed to do:


class UsersController < ApplicationController
  filter_access_to :all
  filter_access_to :edit, :update, :attribute_check => true


role :user do
	has_permission_on [:users], to: [:edit, :update] do
		if_attribute :id => is { }

Some of my user accounts are local (where I store the password) and some are on an ldap server, where the password is stored elsewhere. I'm only allowing those people who are local to change their password. And I put the form behind a 'Profile' link that I'm showing in the navigation bar. So this is the bit I added to the view to show it only for local_accounts.


<% if current_user.local_account %>
  • <%= link_to "Profile", edit_user_path( %>
  • <% end %>

    controllers/application_controller.rb (holds current_user method)

    def current_user_session
     return @current_user_session if defined?(@current_user_session)
      @current_user_session = UserSession.find
    def current_user
     return @current_user if defined?(@current_user)
      @current_user = current_user_session && current_user_session.record

    I haven't had to do this because, until recently, all of my users were being cleared off the ldap server. But now, we're having some cases where I need people not affiliated with my employer needing access to some pages.

    Making a PDF File and Automatically Attaching It

    I'm writing a rails site that collects information from an applicant and references from other people that will then be reviewed by a different group of people. For the programmer, it's easy to present the information from the database on a webpage and then provide links to uploaded documents. However, this is not necessarily the easiest way for the reviewers to read all the information. Based on other sites that do the same type of job, I've learned that reviewers want to be able to download a single file with all the information in it. I've put together various hacks in the past to do this, but have recently come up with a proper way. The following isn't complete, but it presents the basics of how to do this.

    There are three models that I'm using.
    1. Applicant - This is where the applicant initially fills out all of their info and uploads a CV.
    2. Reference_Document - We require each applicant to give us the names of two people who will write references for them. These uploaded documents are stored in reference_document. Each applicant has_many reference_documents.
    3. Application - This is the single model that just has an uploaded file that is a pdf created by combining the applicant's info, CV and any reference_documents provided. Each applicant has_one application.

    There are three gems that I'm using to create the application. Paperclip is to upload and store files. Prawn is used to create a pdf file with all the information that was stored in the database. And CombinePDF is used to combine all the pdf files into a single one that will get stored in application.

    Applicant Model (partial)

    class Applicant < ActiveRecord::Base
    	has_one :application, dependent: :destroy
    	has_many :reference_documents, dependent: :destroy
    	has_attached_file :cv, path: "#{Rails.root}/storage/:attachment/:id/:filename"
    	validates_presence_of :cv_file_name
    	validates_format_of :cv_file_name, with: /pdf\Z/i, message: "must be a PDF file"
    	do_not_validate_attachment_file_type :cv

    ReferenceDocument Model (partial)

    class ReferenceDocument < ActiveRecord::Base
      belongs_to :applicant
      has_attached_file :doc, path: "#{Rails.root}/storage/:attachment/:id/:filename"
    	validates_presence_of :doc_file_name
    	validates_format_of :doc_file_name, with: /pdf\Z/i, message: "must be a PDF file"
    	do_not_validate_attachment_file_type :doc

    Application Model

    class Application < ActiveRecord::Base
      belongs_to :applicant
      has_attached_file :info, path: "#{Rails.root}/storage/application/:attachment/:id/:filename"
      do_not_validate_attachment_file_type :info

    Let's just look at the applicants_controller because that's where most things happen. (When references are uploaded, similar methods will be called to update the application.)

    The create method does most of the work. (Later, update will repeat most of this as well.)

     def create
        @applicant =
          redirect_to thanks_path
          render :new

    Let's look at the generate_application method.

      def generate_application(applicant)
        @applicant = applicant
        info = make_pdf_of_info(@applicant)
        pdf =
        pdf << CombinePDF.load("#{info}")
        pdf << CombinePDF.load("#{Rails.root}/storage/cvs/#{}/#{@applicant.cv_file_name}")
        unless @applicant.reference_documents.empty?
          @applicant.reference_documents.each do |ref|
            pdf << CombinePDF.load("#{Rails.root}/storage/docs/#{}/#{ref.doc_file_name}")
        end "#{Rails.root}/tmp/#{@applicant.fullname_no_spaces}_travel.pdf"
        @application = Application.create!(info:"#{Rails.root}/tmp/#{@applicant.fullname_no_spaces}_travel.pdf"), applicant_id:

    The first step is to make a pdf with whatever info is stored in the database. Looking at that method:

      def make_pdf_of_info(applicant)
        @applicant = applicant
        @file = "#{Rails.root}/tmp/#{@applicant.fullname_no_spaces}.pdf"
        Prawn::Document.generate("#{@file}") do |pdf|
          pdf.text "#{@applicant.fullname}"
          ... (more lines like above)
        return "#{@file}"

    Along with creating the file, it also returns the name of the file created. The generate_application method can then use this file name, along with the stored CV file and any references to make the final application. Once we have this file, the last line in the method, "uploads" it to the Application model along with the applicant's id.

    That's basically it. We now have a single file with all the information for an applicant. As references come in, we'll run the same methods, but will have to change the create! to an update! because each applicant only has one application.

    Multiple Authentication Methods with Authlogic

    I have a website where users need to login to be able to comment on pictures. For the most part, users will use their university credentials to login. However, I've found a few recently retired people who would be able to help us identify people in the pictures. But since they are retired, their university credentials are no longer valid. So for these people, I'd like to be able to make a local account for them to login. My rails app uses authlogic to check credentials. And I've been authenticating against the university ldap server for a while without any problems, but mixing in local accounts is going to require a few changes.

    Normally, I'd just edit the user model with the following.

    	acts_as_authentic do |c|
    		c.validate_password_field = false
    		c.crypto_provider = Authlogic::CryptoProviders::SCrypt
    		c.logged_in_timeout				= 2.hours
    	def valid_password?(password)
    		ldap_settings = YAML.load_file("#{Rails.root.to_s}/config/ldap.yml")[Rails.env]
    		ldap_settings[:host] = ldap_settings['host']
    		ldap_settings[:port] = ldap_settings['port']
    		ldap_settings[:encryption] = { method: :simple_tls } if ldap_settings['ssl']
    		ldap_settings[:auth] =
    			{ method: :simple, username: "uid=#{self.username}, #{ldap_settings['base']}", password: password }
    		ldap =

    And the fields that my user model has are:

      create_table "users", force: :cascade do |t|
        t.string   "username",          limit: 255
        t.string   "firstname",         limit: 255
        t.string   "lastname",          limit: 255
        t.string   "persistence_token", limit: 255
        t.datetime "last_request_at"
        t.string   "role",              limit: 255
        t.datetime "created_at",                                    null: false
        t.datetime "updated_at",                                    null: false

    The default way that authlogic checks passwords is to run a method called valid_password? and return true if the password is good and false if not. To check against an ldap server, I just wrote a new valid_password? method (shown above) with the settings for my ldap server in a file called ldap.yml. The ldap.yml file looks something like this:

        port: 636
        base: ou=people,dc=test,dc=example,dc=com
        ssl: true
        port: 636
        base: ou=people,dc=example,dc=com
        ssl: true

    Now, in order to use local accounts as well, I need to make a few changes to the user model. First, I need to determine if the account is a local or an ldap account. Since by the time I was starting this, I already had about 100 ldap accounts, but no local accounts, I added a field to the user model called local_account. If it's true, it's a local account and if not, it's ldap. This way, I didn't need to change anything on the ldap accounts I already created. Secondly, I needed to add the fields that authlogic needs to store passwords in the database. These were strings: crypted_password and password_salt.

    $ rails g migration add_fields_to_users crypted_password:string password_salt:string local_account:boolean

    After I migrated these in, I had to change my new user form to accept a username and password for local users. Since I don't anticipate making many of these, I didn't do anything fancy with the form. It looks like this:

    <%= form_for(@user) do |f| %>
      Univeristy id: <%= f.text_field :username %>
    Firstname: <%= f.text_field :firstname %>
    Lastname: <%= f.text_field :lastname %>
    Role: <%= f.collection_select :role, User::ROLES, :to_s, :humanize %>

    If person doesn't have a University id, use their email address and be sure to fill out the following. Most importantly, be sure to check the local_account box. (The account name doesn't have to be an email address, but since University ids are never email addresses, they'd be safe to use.

    Local_account? <%= f.check_box :local_account %>
    Password: <%= f.password_field :password %>
    Password confirmation: <%= f.password_field :password_confirmation %>
    <%= f.submit 'Submit' %> <% end %>

    Now, how should I change valid_password? to check locally for the password if the local_account value is true? After reading the code for a bit, I found that authlogic itself calls a method valid_password? to check the password. All I need to do is change my valid_password? to first ask if local_account is true. If it is, call the authlogic valid_password?. If it's not, call my valid_password?. The first problem is obvious in that it's confusing that we're calling the same method, but in different locations. After a bit more searching, I found that authlogic has already thought of this. They have a method called verify_password_method which will let me change the name of my method to something else. So to fix all of this, here's what I changed.

    In the user_session.rb file

    class UserSession < Authlogic::Session::Base
    	verify_password_method :good_password?

    In the user.rb file, I changed my valid_password? method to good_password? and then call the authlogic valid_password? method if local_account is true.

    def good_password?(password)
    	if local_account == false
    		ldap_settings = YAML.load_file("#{Rails.root.to_s}/config/ldap.yml")[Rails.env]
    		ldap_settings[:host] = ldap_settings['host']
    		ldap_settings[:port] = ldap_settings['port']
    		ldap_settings[:encryption] = { method: :simple_tls } if ldap_settings['ssl']
    		ldap_settings[:auth] =
    			{ method: :simple, username: "uid=#{self.username}, #{ldap_settings['base']}", password: password }
    		ldap =
    		# Here checking local accounts in the database
    		valid_password?(password, true)

    This works exactly as I wanted. I still haven't set things up for people to recover their local password if they forgot it. But since I think that there will be maybe 2-4 people who get local accounts, I can just ask them to email me and I can set it to something manually.

    Crazy Error

    I just updated some rails webpages to 4.2.4. On one system, I didn't have any trouble. But on the other one, things worked fine on my laptop, but gave all sorts of random errors on my production system. I started keeping a list to see if I could figure out what was going on. Here are some of the errors that I was seeing.

    Assigning rec to a user
    D, [2015-09-16T13:19:32.760059 #9687] DEBUG -- :   Applicant Load (0.5ms)  SELECT  `applicants`.* FROM `applicants` WHERE `applicants`.`id` = ? LIMIT 1  [["id", 1]]
    E, [2015-09-16T13:19:32.760913 #9687] ERROR -- : Mysql::Error: : SELECT  `applicants`.* FROM `applicants` WHERE `applicants`.`id` = ? LIMIT 1
    I, [2015-09-16T13:19:32.763318 #9687]  INFO -- : Completed 500 Internal Server Error in 34ms (ActiveRecord: 4.4ms)
    F, [2015-09-16T13:19:32.765518 #9687] FATAL -- : 
    ActiveRecord::StatementInvalid (Mysql::Error: : SELECT  `applicants`.* FROM `applicants` WHERE `applicants`.`id` = ? LIMIT 1):
      app/controllers/reference_documents_controller.rb:44:in `update'
    Clicked on Admin
    D, [2015-09-16T13:20:57.053182 #9687] DEBUG -- :   SQL (0.4ms)  UPDATE `users` SET `last_request_at` = ?, `updated_at` = ? WHERE `users`.`id` = ?  [["last_request_at", "2015-09-16 18:20:57"], ["updated_at", "2015-09-16 18:20:57"], ["id", 2]]
    E, [2015-09-16T13:20:57.053291 #9687] ERROR -- : Mysql::Error: : UPDATE `users` SET `last_request_at` = ?, `updated_at` = ? WHERE `users`.`id` = ?
    D, [2015-09-16T13:20:57.053591 #9687] DEBUG -- :    (0.1ms)  ROLLBACK
    I, [2015-09-16T13:20:57.053942 #9687]  INFO -- : Completed 500 Internal Server Error in 8ms (ActiveRecord: 1.2ms)
    F, [2015-09-16T13:20:57.055476 #9687] FATAL -- : 
    ActiveRecord::StatementInvalid (Mysql::Error: : UPDATE `users` SET `last_request_at` = ?, `updated_at` = ? WHERE `users`.`id` = ?):
      app/controllers/application_controller.rb:15:in `current_user_session'
      app/controllers/application_controller.rb:20:in `current_user'
    Clicked on Applicants
    D, [2015-09-16T13:21:27.031430 #9687] DEBUG -- :    (0.3ms)  SELECT `reference_documents`.`firstname`, `reference_documents`.`lastname` FROM `reference_documents` WHERE `reference_documents`.`applicant_id` = ?  [["applicant_id", 1]]
    E, [2015-09-16T13:21:27.031532 #9687] ERROR -- : Mysql::Error: : SELECT `reference_documents`.`firstname`, `reference_documents`.`lastname` FROM `reference_documents` WHERE `reference_documents`.`applicant_id` = ?
    I, [2015-09-16T13:21:27.032374 #9687]  INFO -- :   Rendered applicants/index.html.erb within layouts/application (7.1ms)
    I, [2015-09-16T13:21:27.032567 #9687]  INFO -- : Completed 500 Internal Server Error in 27ms (ActiveRecord: 12.1ms)
    F, [2015-09-16T13:21:27.034208 #9687] FATAL -- : 
    ActionView::Template::Error (Mysql::Error: : SELECT `reference_documents`.`firstname`, `reference_documents`.`lastname` FROM `reference_documents` WHERE `reference_documents`.`applicant_id` = ?):
    Clicked on Upload Recommendation
    D, [2015-09-16T13:22:25.859508 #9687] DEBUG -- :   SQL (0.3ms)  UPDATE `users` SET `last_request_at` = ?, `updated_at` = ? WHERE `users`.`id` = ?  [["last_request_at", "2015-09-16 18:22:25"], ["updated_at", "2015-09-16 18:22:25"], ["id", 2]]
    E, [2015-09-16T13:22:25.859610 #9687] ERROR -- : Mysql::Error: : UPDATE `users` SET `last_request_at` = ?, `updated_at` = ? WHERE `users`.`id` = ?
    D, [2015-09-16T13:22:25.859970 #9687] DEBUG -- :    (0.1ms)  ROLLBACK
    I, [2015-09-16T13:22:25.860197 #9687]  INFO -- : Completed 500 Internal Server Error in 6ms (ActiveRecord: 1.0ms)
    F, [2015-09-16T13:22:25.861710 #9687] FATAL -- : 
    ActiveRecord::StatementInvalid (Mysql::Error: : UPDATE `users` SET `last_request_at` = ?, `updated_at` = ? WHERE `users`.`id` = ?):
      app/controllers/application_controller.rb:15:in `current_user_session'
      app/controllers/application_controller.rb:20:in `current_user'

    I had run rake rails:update to get to these pages and thought that perhaps I had screwed something up during the update. So I basically rewrote the entire app from scratch and I still got the above random errors. All I knew was that it was something to do with mysql.

    Finally, I looked at another app that does pretty much the same thing, but for a different group. When I closely examined the Gemfile.lock files, I realized that the working app was using the mysql2 gem (0.3.20) and the app with random errors was using the mysql gem (2.9.1). I then changed to the mysql2 0.3.20 gem and changed my database.yml file to reflect that. My random errors seem to have gone away, but I'm still testing.

    Did a bit of looking around and found that I should have never used the mysql gem. It hasn't been updated in years. I guess I didn't realize that I had been using this app for that many years. Anyway, now I know to stick with mysql2.

    Links vs Buttons

    Now that I'm writing more websites, I'm trying to make them better. A lot of it has to do with making them look better so that people have a better (UX) user experience. I've been using bootstrap more, which is nice. But now I need to get into more nuts and bolts stuff.

    The question then is, when do you use a link and when do you use a button. Links are easy. If something is just navigation, use a link. If I'm looking at a page and want to move to the edit page, I should have a link, probably called edit. Buttons are for things that would change the database behind the site.

    A simpler way to think of the difference between the two is to use a link if that's not really going to do any harm to the website. If there is a chance to do some damage to something, use a button.

    By default, rails uses links for just about everything. Specifically, scaffolding defaults to using a link to delete an entry. Based on the above information, I should be using a button there. So what do I need to change.

    Here's what the basic delete link looks like:

    <%= link_to 'Delete', applicant, method: :delete, data: { confirm: 'Are you sure?' } %>

    To change this to a button, I'd user this:

    <%= button_to "Delete", { action: :destroy, id: }, method: :delete, data: { confirm: 'Are you sure?' } %>

    Note in both of these lines, I have to specify that the method we want to use is delete. In rails, by default, a link_to method looks for /applicants/new (for example) using GET. The button_to method looks for /applicants/new using POST.