Testing is a hugely important part of development. It's rare that a piece of software is so straightforward that we can be confident that it's working as expected as soon as it's developed.
People often believe this is a good testing procedure:
But in reality, websites are never such a conveniently linear process. People browse around, we expect sometimes payments might fail, people will arrive directly to product pages, products will be out of stock.
it's not unusual for testing to take as long as the creation of the product itself as test-fix-release cycle is repeated until all the discovered issues are ironed out.
Once this process is complete we can usually consider ourselves ready to deploy.
Testing types overview¶
There are four main types of testing:
- Smoke Testing: Manually having a bit of a poke around and seeing if it looks like it's on fire
- Unit Testing: Testing that the internals of each component are doing what they should be doing
- Integration Testing: Check that the individual components interact with others correctly. We won't go into this one in depth on this page.
- Acceptance Testing: Making sure that a delivered product functions according to a set of desired behaviours
The analogy below can help gain an understanding of the stages that lead up to acceptance testing.
During the process of manufacturing a ballpoint pen, the cap, the body, the tail and clip, the ink cartridge and the ballpoint are produced separately and Unit tested separately.
When two or more units are ready, they are assembled and Integration Testing is performed to make sure the cap fits on the stem, and that the ink cartridge plugs into the tip.
Acceptance Testing is then performed so as to confirm that the ballpoint pen is performing its desired function of transferring ink onto paper.
Smoke testing is a type of software testing that ensures that the most important functions work and won't cause any major issue. Fitting the name, developers will poke around to see if nothing catches fire. It's used as an indication to see if a build is stable enough to proceed with further testing.
None of these major functions are tested in depth, but enough to determine if there's anything fundamentally broken. If the Smoke Test passes, then the build can progress to further testing, On the other hand, if it fails, then further and more in depth testing might waste time as there are still problems to deal with first.
Advantages of Smoke Testing are:
- It exposes integration issues
- It uncovers problems early
- Provides a level of confidence that changes to the software have not adversely affected major areas (the areas covered by smoke testing, of course).
Acceptance Testing's objective is to ensure that the product conforms to the business requirements and assess whether it is acceptable for delivery. It has no concern with the product's internal behaviour - it solely makes sure the product's external behaviour is correct.
The following is the definition provided by the ISTQB (International Software Testing Qualifications Board):
Acceptance testing: Formal testing with respect to user needs, requirements, and business processes conducted to determine whether or not a system satisfies the acceptance criteria and to enable the user, customers or other authorized entity to determine whether or not to accept the system.
Acceptance testing is classically done with a Quality Assurance team manually clickking around the site. They're usually experts at finding bugs by typing text into number fields, adding unexpected product combinations
Performing a comprehensive Acceptance Test can take a long time. And when changes are made, the previous round of testing becomes invalid
Behat is a tool which provides a way to automate Acceptance Testing. It works by interacting with a website as though it were a human user.
It allows you to write instructions for actions you want to test, and what you'd expect to see when you do so. It then loads your website in a normal web browser, runs the tests and checks that the expected outcome happens. You can set this off every time a change is made.
The format used for this in Behat is called Gherkin. It uses instructions such as 'When I click on the text "Update Shopping Cart' 'Then I should see the quantity of the test product has changed'.
Below are 3 example of these tests being used against an ecommerce store.
Testing the Cart Page¶
The following specification states that the user should "be able to perform standard tasks in my basket", such as:
- adding something to a cart
- updating the quantity of the item
- removing an item
- seeing an error when a non numerical value is added in the quantity field.
It describes where the user should be, what they should be able to do and the given result of these. If the test passes, then we know that it's working as intended.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Feature: As a customer I should be able to perform standard tasks in my basket Background: Given I am on the homepage And I have an empty cart And I go to the test product page And I click on the text "Add to Cart" Scenario: I update the shopping cart quantity on a product Given I am on the cart page And I change the quantity of the test product in the cart to "2" When I click on the text "Update Shopping Cart" Then I should see the quantity of the test product has changed Scenario: I remove a single product from the cart Given I am on the cart page When I click on the Remove Product Item link Then I have an empty cart Scenario: There should be an error when a non numeric quantity is added Given I am on the cart page And I change the quantity of the test product in the cart to "eee" When I click on the text "Update Shopping Cart" Then I should see "This is a required field."
You can see a video of Behat running through these tests below:
Testing the Checkout¶
The next script tests if the user can access and progress through the checkout.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
Feature: As a user I should be able to get to the checkout Background: Given I am on the homepage And I go to the test product page And I click on the text "Add to Cart" And I go to the checkout Scenario: Successfully create an order with a guest account Given I fill in a guest checkout shipping When I submit a guest checkout shipping And I submit my shipping address details Then I should see the order success page Scenario: Invalid Details on guest checkout Given I fill in a guest checkout shipping When I fill the shipping form field "username" with "behat" Then I should see "Please enter a valid email address (Ex: email@example.com)." And I fill the shipping form field "username" with "firstname.lastname@example.org" When I fill the shipping form field "firstname" with " " Then I should see "This is a required field." And I fill the shipping form field "firstname" with "behat" When I fill the shipping form field "lastname" with " " Then I should see "This is a required field." And I fill the shipping form field "lastname" with "example" When I fill the shipping form field "street" with " " Then I should see "This is a required field." And I fill the shipping form field "street" with "123 Main Street" When I fill the shipping form field "telephone" with " " Then I should see "This is a required field." And I fill the shipping form field "telephone" with "01257898789" When I fill the shipping form field "city" with " " Then I should see "This is a required field." And I fill the shipping form field "city" with "Leeds" When I select "" from "country_id" Then I should see "This is a required field."
Testing the Homepage¶
Finally, the last script tests to see if a user can access the homepage within 2 seconds, and then start interacting with elements on the page.
1 2 3 4 5 6 7 8 9 10 11 12
Feature: As a user I can reach the homepage within 2 seconds And start interacting with elements on the page Background: Given I am on the homepage Scenario: Load the homepage and click on an element to test interactivity Given I see the welcome text When I press the mini cart button Then I should see "You have no items in your shopping cart." Then I close the mini cart
Tying the tests together¶
All three of these can also be run in one single test, as seen in the terminal on the right.
Further Pages on Behat¶
For more in depth information on behat testing, feel free to have a look at the following pages: