Testing

Testing is a hugely important part of development. It's rare that a piece of software is so straightforward that we can be confident that it's working as expected as soon as it's developed.

People often believe this is a good testing procedure:

But in reality, websites are never such a conveniently linear process. People browse around, we expect sometimes payments might fail, people will arrive directly to product pages, products will be out of stock.

it's not unusual for testing to take as long as the creation of the product itself as test-fix-release cycle is repeated until all the discovered issues are ironed out.

Once this process is complete we can usually consider ourselves ready to deploy.

Testing types overview

There are four main types of testing:

• Smoke Testing: Manually having a bit of a poke around and seeing if it looks like it's on fire
• Unit Testing: Testing that the internals of each component are doing what they should be doing
• Integration Testing: Check that the individual components interact with others correctly. We won't go into this one in depth on this page.
• Acceptance Testing: Making sure that a delivered product functions according to a set of desired behaviours

The analogy below can help gain an understanding of the stages that lead up to acceptance testing.

During the process of manufacturing a ballpoint pen, the cap, the body, the tail and clip, the ink cartridge and the ballpoint are produced separately and Unit tested separately.

When two or more units are ready, they are assembled and Integration Testing is performed to make sure the cap fits on the stem, and that the ink cartridge plugs into the tip.

Acceptance Testing is then performed so as to confirm that the ballpoint pen is performing its desired function of transferring ink onto paper.

Smoke Testing

Smoke testing is a type of software testing that ensures that the most important functions work and won't cause any major issue. Fitting the name, developers will poke around to see if nothing catches fire. It's used as an indication to see if a build is stable enough to proceed with further testing.

None of these major functions are tested in depth, but enough to determine if there's anything fundamentally broken. If the Smoke Test passes, then the build can progress to further testing, On the other hand, if it fails, then further and more in depth testing might waste time as there are still problems to deal with first.

Advantages of Smoke Testing are:

• It exposes integration issues
• It uncovers problems early
• Provides a level of confidence that changes to the software have not adversely affected major areas (the areas covered by smoke testing, of course).

Acceptance Testing

Acceptance Testing's objective is to ensure that the product conforms to the business requirements and assess whether it is acceptable for delivery. It has no concern with the product's internal behaviour - it solely makes sure the product's external behaviour is correct.

The following is the definition provided by the ISTQB (International Software Testing Qualifications Board):

Acceptance testing: Formal testing with respect to user needs, requirements, and business processes conducted to determine whether or not a system satisfies the acceptance criteria and to enable the user, customers or other authorized entity to determine whether or not to accept the system.

Acceptance testing is classically done with a Quality Assurance team manually clickking around the site. They're usually experts at finding bugs by typing text into number fields, adding unexpected product combinations

Performing a comprehensive Acceptance Test can take a long time. And when changes are made, the previous round of testing becomes invalid

Behat Testing

Behat is a tool which provides a way to automate Acceptance Testing. It works by interacting with a website as though it were a human user.

It allows you to write instructions for actions you want to test, and what you'd expect to see when you do so. It then loads your website in a normal web browser, runs the tests and checks that the expected outcome happens. You can set this off every time a change is made.

The format used for this in Behat is called Gherkin. It uses instructions such as 'When I click on the text "Update Shopping Cart' 'Then I should see the quantity of the test product has changed'.

Below are 3 example of these tests being used against an ecommerce store.

Testing the Cart Page

The following specification states that the user should "be able to perform standard tasks in my basket", such as:

• adding something to a cart
• updating the quantity of the item
• removing an item
• seeing an error when a non numerical value is added in the quantity field.

It describes where the user should be, what they should be able to do and the given result of these. If the test passes, then we know that it's working as intended.

Feature:
  As a customer I should be able to perform standard tasks in my basket

  Background:
    Given I am on the homepage
    And I have an empty cart
    And I go to the test product page
    And I click on the text "Add to Cart"

  Scenario: I update the shopping cart quantity on a product
    Given I am on the cart page
    And I change the quantity of the test product in the cart to "2"
    When I click on the text "Update Shopping Cart"
    Then I should see the quantity of the test product has changed

  Scenario: I remove a single product from the cart
    Given I am on the cart page
    When I click on the Remove Product Item link
    Then I have an empty cart

  Scenario: There should be an error when a non numeric quantity is added
    Given I am on the cart page
    And I change the quantity of the test product in the cart to "eee"
    When I click on the text "Update Shopping Cart"
    Then I should see "This is a required field."

You can see a video of Behat running through these tests below:

Testing the Checkout

The next script tests if the user can access and progress through the checkout.

Feature:
  As a user I should be able to get to the checkout
  Background:
    Given I am on the homepage
    And I go to the test product page
    And I click on the text "Add to Cart"
    And I go to the checkout
    Scenario: Successfully create an order with a guest account
      Given I fill in a guest checkout shipping
      When I submit a guest checkout shipping
      And I submit my shipping address details
      Then I should see the order success page
    Scenario: Invalid Details on guest checkout
      Given I fill in a guest checkout shipping
      When I fill the shipping form field "username" with "behat"
      Then I should see "Please enter a valid email address (Ex: johndoe@domain.com)."
      And I fill the shipping form field "username" with "behat@example.com"
      When I fill the shipping form field "firstname" with " "
      Then I should see "This is a required field."
      And I fill the shipping form field "firstname" with "behat"
      When I fill the shipping form field "lastname" with " "
      Then I should see "This is a required field."
      And I fill the shipping form field "lastname" with "example"
      When I fill the shipping form field "street[0]" with " "
      Then I should see "This is a required field."
      And I fill the shipping form field "street[0]" with "123 Main Street"
      When I fill the shipping form field "telephone" with " "
      Then I should see "This is a required field."
      And I fill the shipping form field "telephone" with "01257898789"
      When I fill the shipping form field "city" with " "
      Then I should see "This is a required field."
      And I fill the shipping form field "city" with "Leeds"
      When I select "" from "country_id"
      Then I should see "This is a required field."

Testing the Homepage

Finally, the last script tests to see if a user can access the homepage within 2 seconds, and then start interacting with elements on the page.

Feature:
  As a user I can reach the homepage within 2 seconds
  And start interacting with elements on the page

  Background:
    Given I am on the homepage

  Scenario: Load the homepage and click on an element to test interactivity
    Given I see the welcome text
    When I press the mini cart button
    Then I should see "You have no items in your shopping cart."
    Then I close the mini cart

Tying the tests together

All three of these can also be run in one single test, as seen in the terminal on the right.

Further Pages on Behat

For more in depth information on behat testing, feel free to have a look at the following pages:

• An Introduction ot behat & BDD
• Automated Testing Stack
• Behat Folder Structure
• Behat Retry
• Example Testing Sites
• Gherkin
• Summary of Behat