While investigating existing automated tools for mobile TDD, we have run into a well-documented and supported library by Xamarin. It’s called calabash, comes with iOS and Android support and has a wide scope of supported user actions, gestures and expectations. The only drawback we found significant is lack of native Rspec support (cucumber only for now). This post provides a simple tutorial for using calabash with Rspec to test your iOS applications.
Overview
Calabash library including сucumber step definitions is aggregated within single gem
calabash-cucumber. Basic tutorial for integrating your project with Calabash testing tool is well described
here. We use mixed approach for initial setup by adding
Calabash pod and manually copying main project target in Xcode. As cucumber is a primary framework coming out of the box, some additional steps are required to use it with Rspec.
Setting up Rspec environment
First of all install rspec gem and initialize your project:
gem install rspec //or use Gemfile
rspec --init
This will generate the following file structure:
- ./spec folder for all your test examples.
- ./spec/spec_helper.rb file for all configuration and initial setup stuff.
Using predefined cucumber setup and a bit of source investigation, we came up with the following definitions within spec_helper.rb:
require 'sim_launcher'
require 'calabash-cucumber/operations'
include Calabash::Cucumber::Core
include Calabash::Cucumber::TestsHelpers
include Calabash::Cucumber::WaitHelpers
include Calabash::Cucumber::KeyboardHelpers
RSpec.configure do |config|
config.treat_symbols_as_metadata_keys_with_true_values = true
config.run_all_when_everything_filtered = true
config.filter_run :focus
config.order = 'random'
config.before do
@calabash_launcher = Calabash::Cucumber::Launcher.new
unless @calabash_launcher.calabash_no_launch?
@calabash_launcher.relaunch
@calabash_launcher.calabash_notify(self)
end
end
config.before(:each) do
element_exists('view')
end
config.after do
unless @calabash_launcher.calabash_no_stop?
calabash_exit
@calabash_launcher.stop if @calabash_launcher.active?
end
end
end
A before hook was set up to wait for any view to appear – such trick is required to avoid stupid failures because of initial simulator startup delay. There are a few additional method definitions required to make this setup work:
def escape_quotes(str)
str.gsub("'", "\\\\'")
end
#### emulate cucumber reports
def embed(path, type, label)
p "storing report: #{path} (type: #{type}, label:#{label})"
end
You can add them to your spec_helper.rb.
Sugarize it
Basic ruby modules included in calabash-cucumber gem have predefined steps for
- querying elements
- performing actions/gestures
- matching expectations
We have added simple wrapper methods to make test examples readable and DRY. Here are some examples that are used in our test scenarios:
Step definitions
module StepHelper
WAIT_TIMEOUT = (ENV['WAIT_TIMEOUT'] || 30).to_f
STEP_PAUSE = (ENV['STEP_PAUSE'] || 0.5).to_f
def wait_to_see_view(expected_mark)
wait_for_elements_exist([marked_view_query_string(expected_mark)], :timeout => WAIT_TIMEOUT)
end
#Touches
def touch_the_button(name)
touch("button marked:'#{name}'")
sleep(STEP_PAUSE)
end
def touch_view(expected_mark)
touch(marked_view_query_string(expected_mark))
sleep(STEP_PAUSE)
end
def go_back
touch("navigationItemButtonView first")
sleep(STEP_PAUSE)
end
#Waiting
def wait_for_seconds(seconds)
sleep seconds.to_f
end
#Inputs
def enter_text_into_textfield_by_placeholder(text_to_type, placeholder)
touch("textField placeholder:'#{placeholder}'")
keyboard_enter_text(text_to_type)
sleep(STEP_PAUSE)
end
def enter_text_into_textfield_by_label(text_to_type, label)
touch("textField marked:'#{label}'")
keyboard_enter_text(text_to_type)
sleep(STEP_PAUSE)
end
def marked_view_query_string(expected_mark)
"view marked:'#{expected_mark}'"
end
...
end
Custom matchersRSpec::Matchers.define :show_view do |expected_mark|
match do
(element_exists( "view marked:'#{expected_mark}'" ) or
element_exists( "view text:'#{expected_mark}'"))
end
failure_message_for_should do
"No visible view marked '#{expected_mark}' found"
end
failure_message_for_should_not do
"Found visible view marked '#{expected_mark}'"
end
end
RSpec::Matchers.define :show_error_alert do |expected_error_message|
match do
error_message = query("view className:'UIAlertView' label className:'UILabel'", :text).first
error_message == expected_error_message
end
failure_message_for_should do
"No alert with text '#{expected_error_message}' found"
end
failure_message_for_should_not do
"Found visible alert with text '#{expected_error_message}'"
end
end
...
Conclusions
Adding Rspec support for your acceptance testing using calabash library is really simple. Approach described above helped us to build and support application test coverage for wide variety of flows and interactions. Here is a short example on login flow test scenario:
...
shared_examples_for "login error" do
it do
should show_error_alert(alert_error_message)
should_not show_view('Asset list zero case')
end
end
...
describe "login error" do
before do
navigate_to_login_screen
stub_api_request_with_error("POST", "/users/sign_in", error_message);
submit_login_form
end
context "wrong credentials" do
let(:error_message) {'login_failed'}
let(:alert_error_message) {wrong_email_or_password_error_message}
it_should_behave_like "login error"
end
context "server exception/unavailable" do
let(:error_message) {'some_error_message'}
let(:alert_error_message) {server_unavailable_error_message}
it_should_behave_like "login error"
end
end
...
Using Rspec instead of cucumber gives you a better opportunity to setup context of test examples. It’s also very important to use any CI server you prefer as it takes some time for full test coverage to run. Do not hesitate to add your own step definitions and matchers to keep your test examples readable, DRY and clean.