Building a Rails Gmail Client Outside-In

New Article: Building a Rails Gmail Client Outside-In/

For all the new projects that I start with my stakeholders I have been pushing Outside–in software development and Specification by example. Specification by Example really improves the collabora…

Advertisements

Rails Common Commands

A list of frequent Rails command line commands.

  • rvm use ruby-1.9.2-p0 - ruby version manager; switches to Ruby 1.9.2
  • rvm use ruby-1.9.2-p0 --default - ruby version manager; sets 1.9.2 as default
  • rvm use system - ruby version manager; switches to Ruby 1.87
  • rails new - creates a new Rails application
  • rails server [s] - launches WEBrick web server
  • rails generate [g] - lists available generators
  • rails generate controller --help - provides usage documentation
  • rails generate model --help - provides usage documentation
  • rails generate migration --help - provides usage documentation
  • rails destroy controller [Name] - undo generate controller
  • rails destroy model [Name] - undo generate model
  • rails generate scaffold [Name] --skip --no-migration - scaffold skipping existing files
  • rake db:migrate - runs database migrations
  • rake db:test:clone - clones current environment's database schema
  • rake db:test:purge - empties the test database
  • rake routes - list of all of the available routes
  • rake -T - list of rake commands
  • git init - creates git repo
  • git add . - adds all files to working tree
  • git commit -m "Initial commit" - commits files to repo
  • git status - status of the working tree
  • git push origin master - merges local repo with remote
  • git checkout -b new-dev - creates topic branch
  • git checkout master - switches to master branch
  • git merge new-dev - merges new-dev branch into master branch
  • git checkout -f - undo uncommitted changes on working tree
  • git branch - list branches
  • git branch -d modify-README - deletes branch
  • git mv README README.markdown - renames files using move command
  • heroku create - creates app on Heroku servers
  • git push heroku master - pushs app on to Heroku servers
  • heroku rake db:migrate - runs database migrations on Heroku servers
  • heroku pg:reset SHARED_DATABASE --confirm [app name] - deletes database file
  • heroku db:push - transfer an existing database to Heroku.
  • heroku logs - get logs.
  • rails console - command line interface to Rails app
  • rails dbconsole - command line database interface
  • bundle install - installs gems from Gemfile

Rubyist OS X Dev setup

I just bought a shiny new MacBook Pro with a 2.4 GHz Core i5, 8 GB Memory, on OS X 10.63 and thought I would share my setup.

TextMate

Head over to http://macromates.com/ and download TextMate. Once we have installed TextMate we are going to configure it. From the TextMate menu go to:

TextMate >> Preferences >> Fonts & Colors. Then select the “Twilight” theme.

For smaller projects where we do the markup ourselves we are going to use Haml, which we will install shortly, TextMate does not include this bundle by default, so we need to add it. Open up terminal and enter the following:

cd /Applications/TextMate.app/Contents/SharedSupport/Bundles
svn co "http://svn.textmate.org/trunk/Bundles/Ruby%20Haml.tmbundle/"

Back in TextMate go to Bundles >> Bundle Editor >> Reload Bundles.

Terminal

Next on our list is to configure Terminal.

The first thing we want to do is setup the bash prompt for Git. This cool trick will show our checked out topic branch right at the prompt:

Open a Terminal session and type the follow:

sudo mate ~/.bash_login 

To configure we will copy the following into our .bash_login file:


   RED="\[33[0;31m\]"
     YELLOW="\[33[0;33m\]"
 	  GREEN="\[33[0;32m\]"
       BLUE="\[33[0;34m\]"
  LIGHT_RED="\[33[1;31m\]"
LIGHT_GREEN="\[33[1;32m\]"
      WHITE="\[33[1;37m\]"
 LIGHT_GRAY="\[33[0;37m\]"
 COLOR_NONE="\[\e[0m\]"

function parse_git_branch {

  git rev-parse --git-dir &> /dev/null
  git_status="$(git status 2> /dev/null)"
  branch_pattern="^# On branch ([^${IFS}]*)"
  remote_pattern="# Your branch is (.*) of"
  diverge_pattern="# Your branch and (.*) have diverged"
  if [[ ! ${git_status}} =~ "working directory clean" ]]; then
    state="${RED}⚡"
  fi
  # add an else if or two here if you want to get more specific
  if [[ ${git_status} =~ ${remote_pattern} ]]; then
    if [[ ${BASH_REMATCH[1]} == "ahead" ]]; then
      remote="${YELLOW}↑"
    else
      remote="${YELLOW}↓"
    fi
  fi
  if [[ ${git_status} =~ ${diverge_pattern} ]]; then
    remote="${YELLOW}↕"
  fi
  if [[ ${git_status} =~ ${branch_pattern} ]]; then
    branch=${BASH_REMATCH[1]}
    echo " (${branch})${remote}${state}"
  fi
}

function prompt_func() {
    previous_return_value=$?;
    # prompt="${TITLEBAR}$BLUE[$RED\w$GREEN$(__git_ps1)$YELLOW$(git_dirty_flag)$BLUE]$COLOR_NONE "
    prompt="${TITLEBAR}${BLUE}[${RED}\w${GREEN}$(parse_git_branch)${BLUE}]${COLOR_NONE} "
    if test $previous_return_value -eq 0
    then
        PS1="${prompt}➔ "
    else
        PS1="${prompt}${RED}➔${COLOR_NONE} "
    fi
}

PROMPT_COMMAND=prompt_func

Next let’s make terminal a little prettier. From the Terminal menu:

Terminal >> Preferences >> Startup; New window settings: "Homebrew".
Terminal >> Preferences >> Settings; Font select "Andale Mono 18pt".

Now that we have TextMate and Terminal configured we are going to setup our development tools.

Development Tools

XCode Tools

Go to http://developer.apple.com/mac/ and download the latest version of XCode. Once it is downloaded simply run the installer.

Git

Next we will download the git-osx-installer, then run it.

RubyGems

From terminal we need to update our RubyGems:

sudo gem install rubygems-update
sudo update_rubygems

SSH

Still at terminal we will run:

ssh-keygen -t rsa -C "cgabaldon@gmail.com"
cat ~/.ssh/id_rsa.pub | pbcopy

GitHub

Then we will navigate over to GitHub.com >> Account Settings to add our RSA key that we copied to the clipboard.

GitHubAccountSettings by Carlos Gabaldon
GitHubAccountSettings, a photo by Carlos Gabaldon on Flickr.

MySQL

We are next heading over to download mysql-5.1.47-osx10.6-x86_64.dmg, and then once downloaded run the installer.

After MySQL is installed we will install the MySQL preference pane which gets added to the OS X System Preferences. To install we simply double clicking on the MySQL.prefPane contained within the MySQL install package. This gives us a convenient way to start and stop our MySQL server from the System Preferences.

Back to terminal to add our newly installed MySQL server to our path:

cd
sudo mate .bash_login # add: export PATH=/usr/local/mysql/bin:$PATH

Then last, but not least we need to install our ruby mysql gem:

sudo gem install mysql -- --with-mysql-config=/usr/local/mysql/bin/mysql_config

Sinatra

sudo gem install sinatra

Rails

sudo gem install rails

Other Gems

sudo gem install sequel
sudo gem install haml
sudo gem install do_sqlite3
sudo gem install cheat 
sudo gem install cucumber 
sudo gem install dancroak-twitter-search 
sudo gem install faker 
sudo gem install geokit 
sudo gem install giraffesoft-timeline_fu 
sudo gem install github 
sudo gem install heroku 
sudo gem install json 
sudo gem install mislav-hanna 
sudo gem install nokogiri 
sudo gem install passenger 
sudo gem install rack 
sudo gem install railsmachine 
sudo gem install RedCloth 
sudo gem install redgreen 
sudo gem install reek 
sudo gem install rest-client
sudo gem install right_aws
sudo gem install right_http_connection 
sudo gem install rr 
sudo gem install rspec 
sudo gem install technicalpickles-le-git 
sudo gem install thoughtbot-factory_girl
sudo gem install thoughtbot-paperclip 
sudo gem install thoughtbot-shoulda 
sudo gem install webmat-git_remote_branch 
sudo gem install webrat 
sudo gem install wirble 

Capistrano

To simplify deployment we need to install Capistrano:

sudo gem install capistrano
sudo gem install capistrano-ext

FireFox

We of course need to install FireFox, by downloading and installing.

FireBug

Then Firebug and YSlow to help us on our front end engineering work.

Quicksilver

To help us keep our hands off the mouse we will install Quicksliver so we can exercise our keyboard foo. To install simply extract the tarball and drag to Applications.

Update the mappings to be Apple+Space for Quicksilver and Option+Space for Spotlight.

Fluid

Finally, we will install Fluid to help us turn our commonly visited sites into apps. Once downloaded drag to Applications.

Virtual Box

All production applications get deployed to a LAMP server, so we need a place to test out various configurations locally. Say hello to virtualization! We are going to use VirtualBox; download and install.

Evernote

Never forget anything; install Evernote and sign up.

Concentrate

Multitasking does not work when trying program, distractions will kill you. We are going to install Concentrate to apply the Pomodoro technique to keep focused and productive.

Things

We have a lot of Things to get done, so we need a great task management tool.

Growl

Most of the cooler OSX applications support notifications, Growl is the system for doing that.

Pencil

Pencil is one of the coolest UI prototyping tools I have used since a Sharpie.

Hack

We are now ready to do some serious hacking..

Django LAMP

Over the past year and a half I have been using WordPress to publish my web site & blog. Prior to WordPress I was using Calabro which is build with Python on TurboGears. WordPress is great, but does not give me all the control I need. I have recently decided to move back into the world of freelance web development and therefore want to do more with my main web site carlosgabaldon.com. I wanted to build something that showcases my specialties which are Python, Django, JavaScript, semantic markup, and jQuery so this new web site will be built with all of these great tools. I also thought it would be a good opportunity to document the process that I follow when starting a new project.

I like to get integration and deployment tasks figured out early since these tasks contain the most risk. Here is where I usually start:

  1. Setup the the code repository.
  2. Create the initial project structure; so we have something to deploy and iterate over.
  3. Deploy the initial project to the production web server; Apache on a Linode VPS.
  4. Script the the subsequent deployment process for continuous deployment.

Today I am going to go over step number 3; deploying a LAMP server.

Server

The first thing you are going to need is to get a virtual private server (VPS) or dedicated server account at a hosting company such as Linode or SliceHost. I am using Linode running Ubuntu 10.04 LTS (Lucid Lynx).

SSH

Once you have your server you will need to open a console and SSH into your new server:

$ ssh root@{your.ip.address}

Prerequisites

Now that we are logged in we need to install the prerequisites:

$ apt-get update
$ apt-get upgrade
$ apt-get install libapache2-mod-python python-mysqldb
$ apt-get install python-setuptools
$ apt-get install mysql-server
$ apt-get install subversion

Django

We want to be on the bleeding edge so we are going to install the trunk version of Django. First we checkout django-trunk to /usr/local/lib:

$ cd /usr/local/lib/
$ svn co http://code.djangoproject.com/svn/django/trunk/ django-trunk

Then create our symbolic links to point to our new trunk version of Django:

$ ln -s `pwd`/django-trunk/django /usr/lib/python2.6/dist-packages/django
$ ln -s `pwd`/django-trunk/django/bin/django-admin.py /usr/local/bin

Also, while we are in this directory we will create a symbolic link to the Django admin media:

$ ln -s `pwd`/django-trunk/django/contrib/admin/media /var/www/media
$ cd

Git

We need to install Git and then clone the project repository. To keep things simple for this tutorial we are going to put the web site under the Apache root.

$ apt-get install git-core
$ cd /var/www
$ git clone git://github.com/{YourGitHubAccout/your_django_project}.git

Apache

Next we need to configure Apache. I will be using modpython as the python host for Django. Since Django has support for WSGI we could use modwsgi, but I am going to stick with modpython for this tutorial since it has a proven track record in large scale production environments. The only draw back to using modpython over modwsgi is that an Apache restart is required after each deployment. Also, to keep this tutorial simple we are going have Django sever up the static meda files. This is not an optimal setup for a high traffic web site, but for the initial setup it works fine.

To configure Apache will open the default vhost and edit it to look like below:

$ nano /etc/apache2/sites-enabled/000-default
<VirtualHost *:80>
        ServerName {your.domain}.com
        DocumentRoot /var/www
        <Location "/">
                SetHandler python-program
                PythonHandler django.core.handlers.modpython
                SetEnv DJANGO_SETTINGS_MODULE {your_django_project}.settings
                PythonOption django.root /{your_top_level_folder_name
                PythonDebug On
                PythonPath "['/var/www/'] + sys.path"
        </Location>

        <Directory /var/www/>
                Options Indexes FollowSymLinks MultiViews
                AllowOverride None
                Order allow,deny
                allow from all
        </Directory>
        ErrorLog /var/log/apache2/error.log
</VirtualHost>

For the changes to take effect we need to restart Apache

$ /etc/init.d/apache2 restart

MySQL

We need configure the database. Connect to the mysql server using your mysql user name and password.

$ mysql -u root -pXXXX
$ create database {your_db_name};
$ nano /var/www/{your_django_project}/settings.py # update database connection info

Finally, we have Django create the tables.

$ cd /var/www/{your_django_project}
$ ./manage.py syncdb

If everything was done right we can open a browser and point it to http://{your.ip.address}/home/ and we should be running. The final step would be to open your DSN manager (i.e. godaddy) where your domain is registered and point the @ record to {your.ip.address}

Best practices for running a software development team

  1. Build solid relationships with all of  your stakeholders.

    • Customers
    • Product Management
    • Executives
    • Business Partners
    • Business Units
    • Customer Support
    • Operational Support
  2. Create a single intake process for all product requests.

  3. Partner with stakeholders when prioritizing requests.

    • Make sure they are part of any product steering committee meetings.
  4. Always prioritize severity 1 & 2  bugs before accepting any product feature requests from your stakeholders.

  5. Scalability, availability, and performance enhancements should be prioritized very high in relation to product feature requests.

    • Categorizing them as bugs also is way to make sure that they fall high on the priority list.
  6. Do weekly reviews of all production & QA bugs.

  7. Do monthly reviews of production & QA logs to do trending for security issues, performance issues, and capacity planning.

  8. The development team should have real-time visibility to system level stats from the production environment.

    • Use tools like Ganglia and Cacti to give the team a real-time view into production.
  9. Every person on the development team should spend time sitting with the customer support team listening in on support calls to get a deeper view into customer issues.

    • Every 2-3 months for a few hours is a good schedule.
  10. Define a clear escalation plan for production outages/issues.

    • Who is the first line of defense? Who do they escalate to? How do they classify the issue? When should development get involved? How should data be captured for later root cause analysis? Pull data from the HR system for the escalation up the management chain, do not waste time maintaining separate team lists of contact information. Update you HR system to contain all contact info then pull the data!
  11. Every person on the development team should be trained on how to do root cause analysis.

  12. After any and every production incident there should be a root cause analysis performed with clear action items and associated owners to prevent the issue from re-occurring.

    • This is not a witch hunt to find someone who to blame, but rather a way to identify what process or lack of process created the environment for the issue to have occurred.
  13. Hire a technical writer to setup a wiki with a well designed information architecture for all product documentation.

    • Run books, architectural artifacts, knowledge base, etc..
  14. Ensure that there is clear requirements traceability back to the originating stakeholder request.

  15. There should be design traceability from the high level design to the detailed design to the completed code.

  16. Focus on eliminating waste in all SDLC processes.

    • A good place to start is with any process outside of analysis and writing the code, ask the question is the process adding value?
  17. Balance time to market vs. perfect architecture.

    • What is a reasonable architecture to solve the problem today. If cost, resources, or time constrains the architectural design then have a story of how to undo the trade offs made in the design to meet the time to market demands. What is the plan for getting rid of the technical debt?
  18. Make data driven decisions, but do not require that every decision be made with complete data.

    • Identify the experts in the organization and trust them to make the right decisions based on past experiences. For example, picking the right initial architecture should be based expert judgment, past experience, and some amount of prototypes. But scaling that architecture over time should be completely data driven from the metrics collected on the production systems. Always be measuring, analyzing, and optimizing based on the data.
  19. Everyone on the development team should have a professional development plan.

    • This plan is a contract between the employee and management to ensure that the organization will provide all required resources and opportunities for the employee to realize the career goals.
  20. Foster a culture that supports a healthy balance between work and family life for everyone within the organization.

    • Let people go home to their families at reasonable times. Ensure that people are taking vacations on a regular basis. Value results driven work over duration driven work.
  21. Create a respectful work environment where:

    • employees are valued
    • communication is polite and courteous
    • people are treated as they wish to be treated
    • conflict is addressed in a positive and respectful manner
    • disrespectful behavior and harassment are addressed
  22. Every 2-3 months get the entire organization together to discuss the product roadmap, discuss successfully delivered work, answer questions, motivate the teams, and above all to publicly praise and reward the star team members within the team.

Agile development and low tech tools

I just recently started working on large software development project that the management team decided to use SCRUM over some of the existing waterfall process used throughout the rest of the organization. When I first heard the news I thought great now the team can focus on building working software rather than following processes that only make the leadership team feeling like they have control over the project. The problem is that the team is spending large amounts of their time keeping information up to date in VersionOne the tool that was chosen by the management team to manage this project. VersionOne is a very popular tool that many successful companies have standardized on for SCRUM or XP, but from my experience effective agile development teams always choose low tech tools for managing the development.

What do I mean by low tech tools? Low tech tools are tools that do not require the team to sit in front of the computer to use; simple things like big visible charts or information radiator and index cards. Why is this so important? Because the team already spends a large amount of time sitting in front of the computer developing and testing the software, why make them spend all their remaining time updating and managing user stories, burn down charts, and tasks chained to the that same machine.

Low tech tools have a way of connecting people with the ideas and concepts in a much more clear, tangible, and meaningful way. For example, a team can sit down and write all the user stories on index cards then with some large wall space tape the index cards to the wall to create a product backlog. From that the team can do sprint planning by simply moving cards from the backlog section of the wall to the next planned sprint; all with no dual core’s required!

In a large organization or working with some remote team members the low tech way can be difficult and the organization may require that the planning be documented in a more formal project management or tacking tool. To address this have the PM on the team be responsible for keeping the low tech tools in sync with the high tech tools. They are the ones that need to communicate the status of the project to many different types of stakeholders so it makes the most sense for them to manage the communication; and leave the team to focusing on the code…

How to build a product

  1. Developers, product management, and customers brainstorm ideas.

    What problem are we trying to solve?
    What market opportunity are we trying to meet?

  2. Developers & product management write the core user stories.
  3. Developers build an end-to-end product skeleton with mocks to any external systems.
  4. Developers spec out the core product API.
  5. Developers and product management iterate over the product skeleton adding the core user stories.
  6. Partnering with the customer; developers and product management push the product to market as quickly as possible.
  7. From customer feedback developers and product management enhance, rewrite, or create new user stories and apply those stories back into the product.
  8. Repeat 1-7

Meetings

It seems that in all my years of software development every company I work at seems to feel the need to pull people off of real work to have a meeting. Where they want to discuss some useless topic that just makes managers feel like the team is being productive by communicating status or issues. So what are good guidelines for meetings? Should we be having meetings? Here are my 3 golden rules:

  1. Meetings should never be scheduled; you should strive to create an environment where meetings just happen. How do you do this? Create open development environments where there are no offices or cubicles just groups of desks with lots of couches and whiteboards. In this environments meetings become organic and happen only when people need to talk.
  2. You should never have meetings to communicate status; that is why you use wiki’s and bug tracking tools. People who want status should just pull the status from those tools. It does not matter if you are standing up while you are giving status in a meeting or you call it agile. Status or issues should be an asynchronous activity where the people who need to know should be pulling those from the team, not slowing them down with a synchronous meeting.
  3. You should only have meetings for productive things like brainstorming new ideas, doing use cases, group design, discussing code, or getting the team together for drinking beer.

Why working in a large org sucks

Great quote by Paul Graham which explains why working in a large software development organization sucks.

..we can get a portrait of the “normal” world. It’s populated by people who talk a lot with one another as they work slowly but harmoniously on conservative, expensive projects whose destinations are decided in advance, and who carefully adjust their manner to reflect their position in the hierarchy.

http://www.paulgraham.com/kate.html