Using Page Objects for More Readable Feature Specs

Posted by John Wood on Jun 1, 2015 5:24:36 PM

Feature specs are great for making sure that a web application works from end to end. But feature specs are code, and like all other code, the specs must be readable and maintainable if you are to get a return on your investment. Those who have worked with large Capybara test suites know that they can quickly become a stew of CSS selectors. It can make the tests difficult to read and maintain if CSS selectors are sprinkled around through the suite. This is even more problematic if the name of the CSS selector doesn’t accurately reflect what the element actually is or does.

Introducing Page Objects

This is where Page Objects come in. Per Martin Fowler:

A page object wraps an HTML page, or fragment, with an application-specific API, allowing you to manipulate page elements without digging around in the HTML.

Page Objects are great for describing the page structure and elements in one place, a class, which can then be used by your tests to interact with the page. This makes your tests easier to read and understand, which in turn makes them easier to maintain. If used correctly, changing the id or class of an element that is used heavily in your feature specs should only require updating the page object, and nothing else.

Let’s look at an example. Here is a basic test that uses the Capybara DSL to interact with a page:

Let’s take a look at at that same spec re-written to use SitePrism, an outstanding implementation of Page Objects in Ruby:

In the SitePrism version of the spec, the majority of the CSS selectors have moved from the test into the CompanyOfficePage class. This not only makes the selectors easier to change, but it also makes the test easier to read now that the spec is simply interacting with an object. In addition, it makes it easier for other specs to work with this same page since they no longer need knowledge about the structure of the page or the CSS selectors on the page.

While SitePrism’s ability to neatly abstract away most knowledge about CSS selectors from the spec is pretty handy, that is only the beginning.

Repeating Elements, Sections, and Repeating Sections

Page Objects really shine when you completely model the page elements that your spec needs to interact with. Not all pages are as simple as the one in the above example. Some have repeating elements, some have sections of related elements, and some have repeating sections of related elements. SitePrism contains functionality for modeling all of these. Defining the proper repeating elements, sections, and repeating sections in your page object allows you to interact with those elements and sections via SitePrism APIs that make your specs much easier to write, read, and understand.

Testing for Existence and Visibility

SitePrism provides a series of methods for each element defined in the page object. Amongst the most useful of these methods are easy ways to test for the existence and visibility of an element.

Using the Capybara DSL, the best way to check for the existence of an element is to expect that a CSS selector that uniquely identifies the element can be found on the page:

This becomes a bit easier with SitePrism, as we no longer have to use the CSS selector if our page object has an element defined:

Visibility checks are also simpler. Asserting that an element exists in the DOM, and is visible, can be a bit convoluted using the Capybara DSL:

Using SitePrism, this simply becomes:

The above code will wait until #blah is added to the DOM and becomes visible. If that does not happen in the time specified by Capybara.default_wait_time, then an exception will be raised and the test will fail.

Waiting for Elements

Using the above visibility check is a great way to ensure that it is safe to proceed with a portion of a test that requires an element be in the DOM and visible, like a test that would click on a given element. Waiting for an element to be not only in the DOM, but visible, is a great way eliminate race conditions in tests where the click target is dynamically added to the DOM.

Using Capybara Implicit Waits

By default, SitePrisim will not implicitly wait for an element to appear in the DOM when it is referenced in the test. This can lead to flakey tests. Because of this, we highly recommend telling SitePrism to use implicit waits in your project.

Summary

SitePrism is a fantastic tool. If you write feature specs, I highly recommend you check it out. It will make your specs eaiser to write, and maintain.

 

Topics: rails, testing, capybara

Tips and Tricks for Dubugging and Fixing Slow/Flaky Capybara Specs

Posted by John Wood on Apr 23, 2015 2:23:20 PM

In a previous post, I wrote about how the proper use of Capybara’s APIs can dramatically cut back on the number of flaky/slow tests in your test suite. But, there are several other things you can do to further reduce the number of flaky/slow tests, and also debug flaky tests when you encounter them.

Use have_field(“.some-element”, with: “X”) to check text field contents

Your test may need to ensure that a text field contains a particular value. Such an expectation can be written as:

This can be a flaky expectation, especially if the contents of #some-text-field are loaded via an AJAX request. The problem here is that this expectation will check the value of the text field as soon as it hits this line. If the AJAX request has not yet come back and populated the value of the field, this test will fail.

A better way to write this expectation would be to use have_field(“.some-element”, with: “X”):

This expectation will wait Capybara.default_wait_time for the text field to contain the specified content, giving time for any asynchronous responses to complete, and change the DOM accordingly.

Disable animations while running the tests

Animations can be a constant source of frustration for an automated test suite. Sometimes you can get around them with proper use of Capybara’s find functionality by waiting for the end state of the animation to appear, but sometimes they can continue to be a thorn in your side.

At UrbanBound, we disable all animations when running our automated test suite. We have found that disabling animations have stabilized our test suite and made our tests more clear, as we no longer have to write code to wait for animations to complete before proceeding with the test. It also speeds up the suite a bit, as the tests no longer need to wait for an animations to complete.

Here is how we went about disabling animations in our application: https://gist.github.com/jwood/90e7bf6873774055b169

Use has_no_X instead of !have_X

Testing to make sure that an element does not have a class, or that a page does not contain an element, is a common test to perform. Such a test will sometimes be implemented as:

Here, have_css will wait for the element to appear on the page. When it does not, and the expression returns false, it will be negated to true, allowing the expectation to pass. However, there is a big problem with the above code. have_css will wait Capybara.default_wait_time for the element to appear on the page. So, with the default settings, this expectation will take 2 whole seconds to run!

A better way to check for the non-existence of an element or a class is to use the has_no_X matcher:

Using to_not will also behave as expected, without waiting unnecessarily:

No sleep for the fast feature test

Calls to sleep are often used to get around race conditions. But, they can considerably increase the amount of time it takes to run your suite. In almost all cases, the sleep can be replaced by waiting for some element or some content to exist on the page. Waiting for an element or some content using Capybara’s built in wait functionality is faster, because you only need to wait the amount of time it takes for that element/content to appear. With a sleep, your test will wait that full amount of time, regardless.

So, really scrutinize any use of sleep in a feature test. There are a very small number of cases, for one reason or another, where we have not been able to replace a call to sleep with something better. However, these cases are the exception, and not the rule. Most of the time, using Capybara’s wait functionality is a much better option.

Reproduce race conditions

Flaky tests are usually caused by race conditions. If you suspect a race condition, one way to reproduce the race condition is to slow down the server response time.

We used the following filter in our Rails application’s ApplicationController to slow all requests down by .25 seconds:

Intentionally slowing down all requests by .25 seconds flushed out a number of race conditions in our test suite, which we were then able to reliably reproduce, and fix.

Capture screenshots on failure

A picture is worth a thousand words, especially when you have no friggin idea why your test is failing. We use the capybara-screenshot gem to automatically capture screenshots when a capybara test fails. This is especially useful when running on CI, when we don’t have an easy way to actually watch the test run. The screenshots will often provide clues as to why the test is failing, and at a minimum, give us some ideas as to what might be happening.

Write fewer, less granular tests

When writing unit tests, it is considered best practice to make the tests as small and as granular as possible. It makes the tests much easier to understand if each test only tests a specific condition. That way, if the test fails, there is little doubt as to why it failed.

In a perfect world, this would be the case for feature tests too. However, feature tests are incredibly expensive (slow) to setup and run. Because of this, we will frequently test many different conditions in the same feature test. This allows us to do the expensive stuff, like loading the page, only once. Once that page is loaded, we’ll perform as many tests as we can. This approach lets us increase the number of tests we perform, without dramatically blowing out the run time of our test suite.

Sharing is caring

Have any tips or tricks you'd like to shrare? We'd love to hear them in the comments!

Topics: rails, testing, capybara

Fix Flaky Feature Tests by Using Capybara's APIs Properly

Posted by John Wood on Apr 9, 2015 7:38:01 AM

A good suite of reliable feature/acceptance tests is a very valuable thing to have. It can also be incredibly difficult to create. Test suites that are driven by tools like Selenium or Poltergeist are usually known for being slow and flaky. And, flaky/erratic tests can cause a team to lose confidence in their test suite, and question the value of the specs as a whole. However, much of this slowness and flakiness is due to test authors not making use of the proper Capybara APIs in their tests, or by overusing calls to sleep to get around race conditions.

The Common Problem

In most cases flaky tests are caused by race conditions, when the test expects an element or some content to appear on the page, but that element or content has not yet been added to the DOM. This problem is very common in applications that use JavaScript on the front end to manipulate the page by sending an AJAX request to the server, and changing the DOM based on the response it receives. The time that it takes to respond to a request and process the response can vary. Unless you write your tests to account for this variability, you could end up with a race condition. If the response just happens to come back quick enough and there is time to manipulate the DOM, then your test will pass. But, should the response come a little later, or the rendering take a little longer, your test could end up failing.

Take the following code for example, which clicks a link with the id "foo", and checks to make sure that the message "Loaded successfully" displays in the proper spot on the page.

There are a few potential problems here. Let’s talk about them below.

Capybara’s Finders, Matchers, and Actions

Capybara provides several tools for working with asynchronous requests.

Finders

Capybara provides a number of finder methods that can be used to find elements on a page. These finder methods will wait up to the amount of time specified in Capybara.default_wait_time (defaults to 2 seconds) for the element to appear on the page before raising an error that the element could not be found. This functionality provides a buffer, giving time for the AJAX request to complete and for the response to be processed before proceeding with the test, and helps eliminate race conditions if used properly. It will also only wait the amount of time it needs to, proceeding with the test as soon as the element has been found.

In the example above, it should be noted that Capybara’s first API will not wait for .message to appear on the DOM. So if it isn’t already there, the test will fail. Using find addresses this issue.

The test will now wait for an element with the class .message to appear on the page before checking to see if it contains "Loaded successfully". But, what if .message already exists on the page? It is still possible that this test will fail because it is not giving enough time for the value of .message to be updated. This is where the matchers come in.

Matchers

Capybara provides a series of Test::Unit / Minitest matchers, along with a corresponding set of RSpec matchers, to simplify writing test assertions. However, these matchers are more than syntactical sugar. They have built in wait functionality. For example, if has_text does not find the specified text on the page, it will wait up to Capybara.default_wait_time for it to appear before failing the test. This makes them incredibly useful for testing asynchronous behavior. Using matchers will dramatically cut back on the number of race conditions you will have to deal with.

Looking at the example above, we can see that the test is simply checking to see if the value of the element with the class .message equals "Loaded successfully". But, the test will perform this check right away. This causes a race condition, because the app may not have had time to receive the response and update the DOM by the time the assertion is run. A much better assertion would be:

This assertion will wait Capybara.default_wait_time for the message text to equal "Loaded successfully", giving our app time to process the request, and respond.

Actions

The final item we’ll look at are Capybara’s Actions. Actions provide a much nicer way to interact with elements on the page. They also take into account a few different edge cases that you could run into for some of the different input types. But in general, they provide a shortened way of interacting with the page elements, as the action will take care of performing the find.

Looking at the example above, we can re-write the test as such:

click_link will not just look for something on the page with the id of #foo, it will restrict its search to a link. It will perform a find, and then call click on the element that find returns.

Summary

If you write feature/acceptance tests using Capybara, then you should spend some time getting familiar with Capybara’s Finders, Matchers, and Actions. Learning how to use these APIs effectively will help you steer clear of flaky tests, saving you a whole lot of time and aggravation.

Topics: rails, testing, capybara

Testing at UrbanBound

Posted by John Wood on Mar 26, 2015 9:57:46 AM

Testing is a very large part of how we build UrbanBound. We test at various phases in our software development lifecycle, and we have tests that target different levels of the application’s stack. Testing reassures us that the application will behave as we expect it to, ensures that it continues to behave as we expect it to as we change it, and catches problems early, making them easier and cheaper to fix.

User Testing Our Designs

The product team here at UrbanBound strives to design a product that is intuitive, easy to use, and solves the problem at hand. This is very easy to get wrong. Often, teams will make assumptions around the knowledge that the user has of the problem, or how they will interact with the product. Many times, these assumptions are incorrect.

By testing our designs with people who are, or potentially could be users of our application, we find out if we’re wrong at the earliest possible phase in the process. This lets us quickly correct our mistakes while still in the design phase of a feature, instead of spending time and money developing the solution, only to find out after the feature has been launched that the design is a failure.

How we conduct user testing, and the people we recruit as testers, both differ depending on the feature being designed. Often, the prototype being tested can simply consist of a series of static screens that are linked together. This sort of prototype is cheap to build, and is great for testing a user’s understanding of the product and some simple flows through the product. For features that involve a large amount of user interaction, we’ll build a fully interactive prototype in JavaScript. This sort of prototype lets us test the user’s interaction with the product, in addition to their understanding of it and the product’s flow. While it’s much more expensive to build, the costs pale in comparison to the costs of building and delivering a bad solution, and having to go back to the drawing board after a feature has launched and failed.

Backend Unit Tests

UrbanBound is powered by a Rails application on the backend. Like other Rails apps, the application consists of controllers, models, and a host of support classes (such as service objects, policies, serializers, etc). We write unit tests to verify that each of these classes behaves as expected in isolation. Unit tests are great to write when creating or modifying a class, as it lets you know immediately if your code is working properly. It also gives you feedback with regards to how other modules will interact with your code. This feedback helps you define a public interface that is clear and simple. In addition, unit tests act as a great safety net for catching regressions introduced into the application.

We use rspec as the framework for our unit tests, factory_girl for setting up test data, and database_cleaner to help us clean up after each test runs.

Frontend Unit Tests

Like many modern web applications, UrbanBound’s frontend is a single page JavaScript application. Because our frontend is an app itself with a wide array of classes and components, it is important that we test it like we test our backend application. We do this for the same reasons we unit test our backend code.

Our frontend tests are driven by karma, and written using the Jasmine testing framework. We also use the great set of extensions to Jasmine provided by jasmine-jquery.

Feature Tests

Unit tests are great for making sure that modules work in isolation. But, simply making sure the wheels turn and the engine starts doesn’t mean the car will drive. We feel that, especially with a single page JavaScript app on the frontend, it is important to have a suite of tests that exercise the entire stack. This is where our feature tests come in.

Feature tests can be difficult to write and maintain. Because feature tests usually interact with a web server running in a different process or thread, issues related to timing can pop up. If you’re not careful, your test could be checking for the existence of an element on the page before the server has had time to process the response and insert that element into the DOM. This is especially problematic in frontends that make a lot of asynchronous requests to the server. However, there are some very mature tools out there that help with this problem, so you just need to invest some time into learning how to use them properly.

For feature tests, we use rspec + capybara to write our feature tests, in combination with either Selenium WebDriver or Poltergeist as the test driver. We prefer to use Poltergeist because it runs the tests in a headless browser, but sometimes need to fall back to Selenium for specific tests that don’t run quite right in Poltergeist. We also use SitePrism to model the pages in our application as Page Objects, which makes tests easier to read, and write.

Continuous Integration

What’s the point in having an automated test suite if you don’t run it every chance you get? We use a continuous integration service to run our entire suite each time a change is pushed to GitHub. If any tests fail, we are notified immediately so we can investigate why the test failed and fix the issue.

We use CircleCI for our CI server. Although it can be a bit pricy for more than few workers, it really is a fantastic service. It is simple to set up, incredibly flexible, and supports all sorts of  testing setups. They also provide the ability to SSH into the machine that is running your tests, which has proven to be incredibly useful for tracking down that odd case where a test passes locally, but fails consistently on the CI server.

Manual Testing

Automated testing can never completely replace manual testing. It can only help to keep the manual testing focused on the things that can’t be easily automated. We have a team of Quality Assurance Engineers that, in addition to writing feature tests, will manually test features before they are cleared for deployment and merged into the master branch. Our QA team, with their attention to detail, are able to find issues that automated testing alone would never catch. Issues with the visual layout of a page, of the “feel” of the application, are prime examples of where manual testing shines.

Our Testing Philosophy

Our goal is to make sure our application works, and continues to work as we change it. Our approach to testing helps us make this happen. But, we try to never forget that testing has a cost. It takes time and money to maintain a test suite. Therefore, there must always be a return on that investment to the business. Though it doesn’t happen often, there are certainly times where automating the testing of a certain feature just doesn’t make sense. We’re not dogmatic about making sure that 100% of our codebase is covered by an automated test. But, that being said, we automate as much as we practically can.

Topics: testing, Development & Deployment

Human Resources Today