All Aboard Ruby On Rails

Ruby On Rails

Last night I posted the code for the next blog installment which is using the Ruby on Rails web framework for building your blog application. You can find this demo code in the new GitHub repository at http://github.com/deanpearce/MyBlogRails. And please do look at it, play with it, and tell me what I’ve done wrong as usual :). I have spent time over the past few weeks familiarizing myself with Ruby and how its gems, such as Rails, extend the functionality of the language. For those of you who are new to Ruby on Rails, it’s important to understand that Rails is a Ruby gem, meaning that the gem extends the language to make Ruby a friendly language for web development. As per usual, I am assuming that you are caught up in at least the fundamentals of the Ruby programming language, and have followed the installation guide (for Linux as usual :)) at http://wiki.rubyonrails.org/getting-started/installation/linux. The process is rather straightforward, and even easier on Ubuntu as most (if not all) the packages can be installed using Aptitude to get the development process going without needing to wait for builds.

Overall, this iteration of the project got easier, since the HTML::Mason and Rails ERB views are very similar in construction. It became an exercise in substitute the correct language code while keeping existing logic in place. For my own sanity, I added a users controller so I can control accounts through a web UI once you are authenticated (hint: this will be a lead up to a potential second round, integrating user accounts and external services such as Gravatar :P). Compensating for the time I spent in developing the basic template for Catalyst (a grand total of 3 of the 35 minutes), it works out that from scratch, development of a blog application took about 20 minutes in Ruby on Rails, which is stunning! But that isn’t without a few pitfalls that I will go over later in the article.

What I will be covering:

  • Generating your first Rails Application
  • Framing the structure using scaffolding
  • What comes built-in to Rails (and why the way I used sessions isn’t the brightest)

The Installation Process

This section is pretty short, and overall I have no complaints! Using Aptitude to install on Ubuntu, the process from opening the Wiki to having the default “You’re Riding On Rails” page was maybe 10 minutes, significantly faster than Catalyst. And because of the way we will be playing with Ruby on Rails, we won’t be doing our database creation! Well, we will be using a MySQL database, but we will use the rake tool to handle the complex parts of the database creation and generation for us.

Creating the App

Whoa! We have already cut a lot of the time out from the Catalyst application. The install process was a few simple apt-gets (in Ubuntu, a few simple makes everyone else) and we can skip the database creation phase! This is great for us since we are trying to shave minutes off development. In the real world, one would want to plan all of this out beforehand, such as planning your controllers rather than your database, but for our purposes we will do this ad-hoc. Switch to your favourite development directory and type:

  • pearce@marvin:~/Projects$ rails -d MyBlogOnRails
  • create …

Great, the application was generated! You might be wondering what the -d flag is. By default RoR (Ruby on Rails) uses an SQLite database, the -d flag tells it to build as if we are going to use a MySQL database. The flag may be -d or -D depending on the source you install and version, so keep an eye out for that. Very similar to Catalyst, and as we’ll discover, pretty much the same structure all the popular web frameworks use. Now that we have an application, let’s start creating with it.

Configuring the App

The first thing is we need to do is give the app some credentials for the database. Ruby on Rails differentiates between development, testing and production. The names are pretty self-explanatory and for our needs we will be using the generated default of development. Switch to your project directory MyBlogOnRails and edit config/databases.yml:

  • pearce@marvin:~/Projects$ cd MyBlogOnRails
  • pearce@marvin:~/Projects/MyBlogOnRails$ vi config/database.yml

development:
adapter: mysql
database: myblog_development
username: root
password: ****
host: localhost

This will tell RoR that our database, which we will call myblog_development, can use those credentials at that host. There are sections for production and testing also in the file, and you can edit those at your leisure, but for our purposes we will only be using development. That’s it for setup, nothing to make or build or chmod. Let’s continue on to Grand Central: the implementation of the model, view and controller.

Grand Central: MVC Implementation

If so far your RoR experience has been good, this is where things may switch tracks for some or go express for others. There are some tips and tricks that are necessary for a smooth experience that can be read in the Ruby on Rails Wiki such as naming convention of controllers and types to specify when generating the controller and such. If you have this knowledge in your head, Rails will turn your development express. If you’re like me and don’t like reading documentation before you start hacking around (though I always end up reading it, and improving what I do :)) then this might be the thing that derails your project (I’m on a roll with these puns :D). The big holdups for me were the following:

  • When using the generator name your controllers in the singular (it will automatically handle making things plural as needed), else you will encounter “dependency missing” message hell.
  • The documentation on what is going on when things go wrong at this point is horrible! Nine out of ten sources (sites, docs, comments) say “oh well regenerate your app” and that’s it!
  • For in-depth projects you will want to plan your database either independently and write your own controllers, or plan your controllers (down to the order things will be created).

Once you get over that, it’s fairly smooth sailing. First attempts at it were causing premature manual baldness and I am really surprised that for such a large and vibrant community these basic problems aren’t explained more often (and in ways that are easy to search for on a search engine).

Controller

So now let’s generate the controllers that we will need for this project:

  • pearce@marvin:~/Projects/MyBlogOnRails$ ruby script/generate scaffold post title:string body:text
  • pearce@marvin:~/Projects/MyBlogOnRails$ ruby script/generate scaffold user username:string password:string email:string

At first this may look a little confusing, why are we telling our controller what we need to include, and what is this scaffolding? This will be explained after we finish generating everything 🙂

And that’s it for generating code! What, I bet you thought it was going to be more. Scaffolding is a cool tool included in Rails to generate most of the code needed to get a full MVC application up and running. It creates the model, view files and the controllers all in a nice setup for you to hack away on. For the advanced developer, or very custom components it isn’t too helpful, but for beginners its a great way to see a tangible application, right now with not one line of code! We really are slowly coding ourselves out of jobs aren’t we :P. If you are going to be generating the model on your own, you can add the parameter –skip-migrations to the generate script, which will still generate all the code for you, but running db:migrate will not attempt to create the tables that you need. One important thing to note is that if you want the default action to be the posts controller, you will need to manually edit the routes.rb file in the config directory map.root :controller => “post” right before the end directive in the file, and you will need to delete the default index.html.erb template. This will ensure the root action is the post controller and that you won’t accidentally run it with the index.html.erb file.

The next big step will be to actually create the database and tables for our application, and now that our model is defined, we can use the rake tool included in rails:

  • pearce@marvin:~/Projects/MyBlogOnRails$ rake db:create
  • pearce@marvin:~/Projects/MyBlogOnRails$ rake db:migrate

And if we’ve given the correct permissions to our database user we should get messages saying it has created the database, and a message for each table it creates. As of now, we’re done! Well, not yet 🙂 but if you want to see your RoR app, you can run:

  • pearce@marvin:~/Projects/MyBlogOnRails$ script/server

Which will start a default Mongrel server on port 3000. Navigate to this and you will get the scaffold generated interface! This is all so exciting, a fully running app that you can technically “post” to with no code at all. But as you can see, no security or style either. Let’s take it to the next step and look at sessions and creating a controller for logging in:

  • pearce@marvin:~/Projects/MyBlogOnRails$ ruby script/generate controller user login

This creates our new controller along with the login action/views. Notice we are not doing scaffolding here as the implementation will be done in pure manual code and can be found in the GitHub repository under the controller and views of the same name. In principal we want to do the same thing as the Catalyst version: only allow posting when logged in and a basic login/logout system that just gives us access to the new post page and logout button.

The big helper here is the built-in session management provided by Rails. This means no fiddling around with authentication like in Catalyst, we can immediately use the session call from within our application like <%=session[:username]%>. This is exactly how it was implemented in the blog application, and while it seems good at first, you realize that we depend on the session not being forged and solely on the username :P. In a real system this might not fly as reasonable authentication, but for our purposes it’ll be like Fort Knox. Just as easy as we can set a session variable using session[:key] = val we can use reset_session to erase our session. I won’t go into code detail as I’m assuming you know at least a tiny bit about Ruby, or you have checked out the project and investigated the controller_user.rb file in app/controllers. Once you rework your templates using Ruby syntax such as @post.each do |post| and setup your session management you will be good to go.

Final Thoughts

Once again, it amazes me that it took far long to write (and find time and motivation for :P) writing the blog about the application than it did to actually write it. Ruby on Rails does one thing and does it exceedingly well: turns ideas into applications quickly with minimal code. Like any framework though, once you get more complicated than the basic single table layouts and non-scaffold actions things get a bit more complicated. Ruby on Rails sped things up by allowing me to skip figuring out where to place the model and the views, and allowed me to skip some basic templating and routing, since this was all automatically generated.

Pros

  1. If I thought Catalyst was wickedly fast at prototyping, this is like that star being hurled across space at 2.5 million KM/h. Plain and simple, it’s fast.
  2. Routing (chaining and paths in Catalyst) is cake as long as you don’t need to make modifications to the default values. Things get a bit more when you have to edit the routes.rb file, but the advantage is in that file you can see all the actions and controllers and exactly what they do. I like that, but I also like to know what the controller is doing in the actual controller file. So pluses and minuses to each system.
  3. Default templating is mind numbingly easy, and since HTML::Mason and the ERB files (along with the templating in Cake and JSP pages) they are all very similar in structure. In fact, with simple language replacement, you could migrate templates across systems with almost no work at all. And the scaffolding in Rails allows me to not even need to know how to make a default template (though I do) but just build on what it generates.
  4. Gems are great, I mean Rails, much like Catalyst, is just a module.

Cons

  1. Documentation for anything before actually writing custom controllers, and modules and such is a nightmare, they have great installation guides and how-to walkthroughs online, but anything that sways from the path, or funny errors because you forgot something minor are hard to find documentation for from my experience. Also, the default course of action most people provide is “oh well regenerate your application”, which in the case of being like 3 command line entries in isn’t bad, but let’s say something went awry once much deeper into the project, regenerating/restarting is unthinkable.
  2. The gems are a great feature, but the downside is they are harder to find and not often organized into just a repo. Some are built and from Aptitude, some are from individual sites, there are lots on RubyGems.org, but it just doesn’t feel as well organized. Maybe I’m just too new to see the elegance of the system, or I missed Gems 101, but it doesn’t feel as solid as Perl in that sense.
  3. Mongrel is nice but when the default suggestion is to load balance it I start to question performance a bit 🙂 much like Catalyst, memcached plugs into Rails easily with Cache.put and Cache.get, so it’s probably a good idea to integrate that into your code, and to anticipate using Mongrel Cluster on Apache (or some other httpd) for maximum performance. The upside is this allows for very easy distribution across servers.

So that’s it, I might have some updates to the post in future weeks, or after feedback. As usual, leave a comment below, or email me and I will respond to you. I will not be posting packages in my posts anymore in favour of uploading them to GitHub or to my local repository. Next on the list is CakePHP, which I look forward to as PHP is my secret addiction and I want to finish playing with it and put a post together on that. A friend of mine suggested that I add Django, which is a Python web framework to my list of things to try, so that has been added to the plate for after CakePHP!

Get a Reaction With Catalyst

Hi all, it’s been a while since I’ve updated my blog, but with jobs and projects and plain old laziness it’s hard to keep up with everything. For months now I’ve been playing with Catalyst at work, Ruby on Rails for development and back in the day some CakePHP to feed my PHP addiction. Also I’ve played with Java using Tomcat to produce web apps at previous jobs, and for the fun of it trying out different technologies. I’ve finally decided to make use of this wasteful time and put it together into a series of blog posts!

Today, I will be discussing Catalyst which is an MVC web application framework written in Perl. Free to use, just search CPAN for Catalyst for more information. If you’re adventurous and know Perl, go ahead and load CPAN and use “install Task::Catalyst” or on the Catalyst page, Matt Trout has posted a wonderful script to speed up installation. I’ll cover that part in a bit, but let’s say it takes longer than a coffee break. There will be a code link at the bottom of the article where you can download my 35 minute Catalyst application to play with!

What I will be covering:

  • The process from start to finish creating a new shell of a Catalyst application
  • A tiny bit about the plugins for Catalyst and which ones I found useful
  • Framing up your first application using HTML::Mason
  • How to properly enjoy the fact that from start to finish, you have a paginating blog with a decent login system that you can post to in under 30 minutes (after Catalyst initial install)

The Installation

Here it gets a bit technical, but I’ll explain it some as I go. I’ve cut out a lot of the processing text for sanity’s sake (denoted by ” …” at the end of a line). Besides, if you’re serious/already have done the install, who needs to see the auto-generated text anyways :))

Creating the Database

This is the backbone of the application, our database. Now, I’m still addicted to PHP like crack, so I still love my PHP tools like PHPMyAdmin. Wonderfully easy to get databases up and running. Organize a database for yourself, I setup a very simplistic (mostly self explanatory) set of tables: Users (id, username, password, email) and Entries (id, title, body, posted). It’s best to do your database architecture before we get started as we will be using it right away for model generation. The models can be regenerated at any time, but I recommend getting your database right the first time and then to tweak as needed. First we need a database:

  • mysql> create database MyBlog;
  • Query OK, 1 row affected (0.37 sec)
  • mysql> exit

Creating the App

App creation couldn’t be easier with Catalyst (basically the same as Ruby on Rails and CakePHP). You give the generator script the name of your app and off it goes creating the scaffolding you need to get your app going. It’s important to note that on some operating systems the call is simply “catalyst” instead of “catalyst.pl” (Windows if I am not mistaken, but then who’s using that for development anyways :))

  • pearce@marvin:~$ catalyst.pl MyBlog
  • created “MyBlog” …

Configuring the App

After running this, you will need to follow what the screen tells you and run the Makefile. It will take care of getting everything up and running, but not chmod’ing the scripts to be executable strangely enough. Just chmod the .pl files in the scripts directory that was generated.

  • Change to application directory and Run “perl Makefile.PL” to make sure your install is complete
  • pearce@marvin:~$ cd MyBlog
  • pearce@marvin:~/MyBlog$ perl Makefile.PL
  • include /home/pearce/MyBlog/inc/Module/Install.pm …
  • pearce@marvin:~/MyBlog$ chmod +x script/*.pl

The Meat and Potatoes: MVC Implementation

Here is where Catalyst shines in my opinion. It has nice, clean and clear steps for creating the Model, View and Controller. All functionality can be found in the script/myblog_create.pl script, and you can consult it for further functionality. We are going for speed, so let’s create the components as fast as possible. We want a model that comes from our MySQL database without any additional coding, a basic Blog controller (that we can use instead of Root) and we want to use HTML::Mason for the templating. This is done with 3 easy commands as shown.

Model

Don’t forget to change the “root” and “****” to your username and password for your database!

  • pearce@marvin:~/MyBlog$ script/myblog_create.pl model MyBlog DBIC::Schema MyBlog::SchemaClass create=static dbi:mysql:MyBlog root ****
  • exists “/home/pearce/MyBlog/script/../lib/MyBlog/Model” …

Controller

  • pearce@marvin:~/MyBlog$ script/myblog_create.pl controller Blog
  • exists “/home/pearce/MyBlog/script/../lib/MyBlog/Controller”
  • exists “/home/pearce/MyBlog/script/../t”
  • created “/home/pearce/MyBlog/script/../lib/MyBlog/Controller/Blog.pm”
  • created “/home/pearce/MyBlog/script/../t/controller_Blog.t”

View

  • pearce@marvin:~/MyBlog$ script/myblog_create.pl view Mason Mason
  • exists “/home/pearce/MyBlog/script/../lib/MyBlog/View”
  • exists “/home/pearce/MyBlog/script/../t”
  • created “/home/pearce/MyBlog/script/../lib/MyBlog/View/Mason.pm”
  • created “/home/pearce/MyBlog/script/../t/view_Mason.t”

We are now prepped to run our app! We can fire it up by typing “script/myblog_server.pl” which will fire up a Catalyst running server on port 3000. So if you navigate to http://localhost:3000 you will be able to see your app up and running! The hard part is over, the next steps will be creating the template files in the root directory and creating the actions. I will go over them broadly but it’s a lot of concepts to cover. The CPAN repository has great tutorials and cookbooks, but I’ll share with you what I didn’t know when I started with Catalyst.

  1. In typical fashion, the controller looks for the template file of the same name as the page you are requesting. So /login will look for “login” in the root directory that was generated. To change the template (and keep in typical naming fashion you would want something like “login.mason” or “login.mas” or perhaps more general “action.mason” which has parameters. All depends on your necessities. You can change the template for an action by changing the stash template $c->stash->{template} = “myfile.mason”.
  2. You can create an autohandler file in the root which is executed before any of the action templating. This allows you to insert a header/footer on every page automatically. Simply include the line “% $m->call_next;” where appropriate in the file to load the requested template file there.
  3. Chaining an action is amazing. If you don’t understand chaining, please read up on it elsewhere, its fundamental to most MVC designs, and you won’t get very far (sanely anyways) without it. It allows you to pass parameters via URL and handle subpages easily. It’s made up of the Chain (to a previous action), the PathPart (the actual action you want to associate with) and the Arguments (0 or more, Capture them if you are going to have subpages after the arguments). So something like “sub login : Chained(“/”) : PathPart(“login”) : Args(0)” will result in the action /login going to the sub named login.
  4. Mason files are very similar to TT, to ERB to JSP. Similar tags, similar structure, you just need to get used to the <%args> parameter at the top of the file, as well as some interesting syntax. You can embed code right in the page, and while not always best practice, it is a godsend for customization.
  5. Always leave -Debug on in MyBlog.pm when developing. It shows errors in detail in the browser, it shows errors in the console. You won’t be able to understand what’s going on without it at least while learning.

Final Thoughts

Hard to believe that this blog post took way longer to write than it did to finish up the demonstration blog application. The application from start (creating the db) to finish (creating the tarball) took 35 minutes. A fairly impressive feat, but hardly unbelievable if you are familiar with Perl and some MVC architecture. Here’s a look at some of the pros and cons I faced both in speed developing this app and in general developing with Catalyst:

Pros

  1. Wickedly fast prototyping, churning out a functional application in minutes.
  2. Growing community, quite a large amount of documentation on Catalyst and its default model DBIx.
  3. Easy to use chaining functionality (in comparison to Ruby on Rails routes file and others).
  4. Intuitive templating (not so much a pro, but a definite make it or break it situation).
  5. Integrates well with CPAN modules
  6. Has a wide variety of plugins available. The one I make use of is called AuthenCookie which handles all your cookie based authentication for you. This is available on CPAN and needed to run the demo. Explore, people write modules frequently and it might shave minutes or hours of your project, with the good feeling of having a community of eyes looking at it for you, keeping it up to date and speedy.

Cons

  1. Catalyst itself is a nightmare to install. I mentioned this earlier, but seriously, get the installation script or write your own that presses “Enter” over and over :). It has a good 75 modules to install, more if you’re starting from a fresh Perl install. Just make sure you have unzip configured for CPAN, write permissions everywhere and that your build tools are working, run the script and go for lunch. It will take upwards of an hour over a mediocre internet connection to get Catalyst going. The upside is once you have it, it’s there forever and easy to update/remove.
  2. You have to like Perl. And I don’t mean just “yeah me and Perl are good pals” but I mean “..must…have…Perl…to…live” kind of situation. Okay, maybe not that much, but an intimate knowledge of Perl (and its special variables) will save you time and headaches. With DBIx::Class I find myself mapping result sets, or iterating using $_ quite frequently. Frankly, you don’t need much more than that, a basic understanding of hashes and how Perl differentiates between a ref and an “object” (for lack of a conceptually easier term). You can pick it up as you go, and the framework covers most of the basics, but you will want to pick Perl up to get the most out of it.
  3. You will need some sort of memory caching service (like memcached) to keep any very large scale (or computationally complex) system running at peak performance. This is more of a scalability issue plus overhead than a design issue. Probably best not to be wasting cycles recomputing a homepage anyways.

So please download the package [MyBlog.tar.gz] and play with the app. It is fully functional, albeit kind of silly in terms of security and design, but it would be a good first effort. You get to see templating, parameters, the stash, DBIx (the model we are using, it is an interesting beast, please read up on CPAN about it). And please leave comments/corrections/suggestions in the comments as usual! And remember this is the first in a series of posts that will discuss Ruby on Rails and PHP, and dive into why Java is terrifying for any basic web application.

Update: I have now seen the goodness of GitHub and have started a repository. You can view the code for this and future projects at http://github.com/deanpearce/MyBlogCatalyst!

Update: My colleague and code guide Yanick has informed me that I could have saved myself a lot of time by disabling running the tests when building the modules. It makes good sense that we could save some time here, especially if we don’t anticipate any issues (such as on a fresh install with core prerequisite packages and programs installed such as build-essential and unzip installed). For this I’m a bit ashamed I didn’t think of this while installing, and from initial reinstall tests, goes from a lunch break to grabbing a coffee. Thanks Yanick!

DupliFinder Alpha Release

As promised, here is a compiled installer as well as the Visual C# 2008 source code for your enjoyment. A few things to note: I can’t guarantee that it will work on your system, I can’t guarantee that it will work as advertised, and that all crashes have a reason. For developers I can’t guarantee that the source code is readable :). Hopefully you’ll see through my poor commenting and naming conventions and see the potential for this code :P. Anyways, this is by no means a finished product, but I am releasing it to one keep myself on schedule, and two make a duplicate file finding program available for free (finally!). The installer may only be 64-bit (the publish options in Visual Studio are so confusing!) so if that’s the case you will need to compile it yourself. Mature releases will have a 64 and 32-bit build available (with probably a real and better install system).

As usual, please leave feedback relating to functionality or utter destruction in the comments below, and remember it’s alpha software so use at your own risk! It won’t delete random files, but don’t think it won’t actually delete all the files when you select them with the checkbox and choose Erase!

In Development Land Again: DupliFinder, Duplicate Image Removal

Interesting enough, the more stressed I become the more I desire to program. And not just random programming, but programming to solve a problem (namely one of me :)). What I’ve been working on over the past couple days is a nice Windows utility to work with and manage duplicate files. Right now it works on the head of the file (64KB) or entire file for comparison, but future versions (such as the one I’m working on now) are going to include audio stream matching (to compare audio regardless of the format or compression) and video frame matching along with a host of other options. Here’s a screen shot of it in action:

DupliFinder Duplicate File Finder

DupliFinder Duplicate File Finder

As you can see the UI is almost complete, allowing you to select a root folder and you can scan current and sub directories optionally. The only problem with it at the moment is it’s single-threaded, so when it is given a large folder to process it feels as if the program has frozen, but don’t worry it’s crunching away in the background.

The feature I’m adding right now is a preview/compare pane for easy verification that the file is the same, as well as giving the user to option to delete the “duplicate” or “original” in case of file confusion. This will be ready to go for the v1.0 release of the utility.

If anyone is interested, I can package up a preview version that has some quirks and the single-threaded problem. I had looked all over for a program that would allow you to delete duplicate images and other files and it’s ridiculous the amount of money those scam-like software companies try and charge for such simple (but effective) software! Anyways I’m contributing back to the community by releasing this under GPLv2. On top of handling photos (for which I’ve seen some software cost $30US!) it will handle all sorts of stream and data related information.

For any programmers out there reading this, it was programmed using Visual C# 2008 on a Windows 7 development box, so all sorts of shiny Windows 7 features may start peeking through soon :).

If you have any comments, please feel free to leave them below, and if you would like to play with the source or the compiled version of this app, just let me know in the comments and I will fire off some more information and a package. That’s all for now!