Skip to content

Regression Testing#
Author Juha-Matti Hietala
Last Updated 22.10.2025
Last Updated By Darlington Ikeri

Requirements#

  • Robot Framework and SeleniumLibrary installed
  • Basic knowledge of Robot Framework

What is Regression Testing#

Regression Testing is a set of test cases that are executed whenever changes are made to a product, in this case, the Prestashop site. Their purpose is to make sure that the changes made do not break any existing functionalities. These functions have already been tested to be working as intended, often manually, so redoing them would be very time consuming, especially because the number of test cases included in Regression Testing can be in the hundreds, even thousands if the project is large enough.

Notice that there is a difference between regression testing and retesting. As mentioned before, regression testing goes through cases already tested to be working, making sure no surprising bugs or defects have appeared. Retesting tests for specific defects or bugs, often found in the regression testing.

Example#

You have manually tested a feature to be working, for example that the links on the footer of the homepage lead to the correct URL. This is a simple task to be done manually once, but doing it every time a change is made to the code would be very time consuming; the bigger the project, the larger the number of updates and the number of required re-tests. Imagine you would need to do this manually for all those tens, maybe hundreds or even thousands of small tests individually, soon this would consume all your worktime. This is why test automation is so important.

After you’ve manually gone through this test case and noted it working as intended, and it is deemed as something that should be repeated after every update, create a robot framework script of it and add it to the regression test file that is run in your Gitlab CI/CD pipeline or locally. Now you only need to manually test it again if the automated test reports an error.

When to run regression tests#

In the previous example, the Regression Tests were run every time a change is made into the code, but there are other instances where it could / should be done. For example, before the product is launched for the public to make sure a final time that everything works, when some addon is added to the site like a payment method or after security updates. As some of these are done without direct changes to the code, you must be able to run the test file(s) manually from your workstation.

Why is regression testing important#

There are many benefits in regression testing, ranging from project management to development efficiency and all the way to the end user experience. Some examples but not all of the benefits:

Making sure existing functionalities still work#

Increase confidence that the existing functionalities still work after every change by thoroughly testing them.

Product quality#

By finding product defects before the update is released to the public, the image of the quality of your product stays intact.

Faster bug findings and fixes#

As the tests are run after every code change, you get fast feedback on what changes caused the bugs and the developers can fix them before they end up causing further problems.

End user experience#

Making sure no broken updates are released to the public you avoid negative feedback and possible lost sales, if something in the buying / paying functionalities would break.

Test Structure Explanation#

Each Robot Framework test file follows a modular structure for readability and scalability:

  • Settings: Defines the libraries and resources used, such as SeleniumLibrary for browser automation and XML for parsing responses.
  • Variables: Stores configurable values such as URLs, element locators, or credentials. This allows tests to be updated easily without changing the logic.
  • Keywords: These are reusable blocks of actions, similar to functions. They group related steps (like login or add to cart) into logical units.
  • Test Cases: Contain actual test scenarios that use the defined keywords. Each test case should be independent and self-contained to avoid dependencies between tests.

By organizing tests in this way, maintenance becomes easier and debugging failures is faster, especially as the regression suite grows.

Best Practices for Regression Test Design#

When expanding your regression suite, follow these best practices to ensure stability and maintainability:

  • Keep tests atomic: Each test should validate one functionality. Avoid chaining multiple features into a single test.
  • Use meaningful test names: Clearly describe what each test does, e.g. Sign In with Valid Credentials or Add Product to Cart.
  • Avoid hard-coded waits: Instead, use dynamic waits such as Wait Until Element Is Visible for better stability.
  • Isolate test data: Store test credentials, URLs, and product identifiers in a separate resource file or variable file.
  • Use headless browsers in CI/CD: This ensures tests run faster and don’t require a GUI environment.
  • Add teardown steps: Always include [Teardown] Close Browser to prevent resource leaks and ensure clean test environments.
  • Tag tests logically: Use [Tags] for organizing subsets of tests, e.g. smoke, checkout, or auth, to run specific groups as needed.

Following these practices makes it easier to manage large regression suites across multiple environments.

Example regression test file for Prestashop#

This is a simple and small regression test file for reference and to be built on. It includes tests for some links in the footer, signing in with correct credentials and adding a item to the cart. It doesn’t use a resource file so you can copy and paste it for testing, for information about resource files see this guide. Change ${BASE_URL} to your address and the sign in credentials in the "Sign in test" to a user in your shop. 

*** Settings ***
Documentation   Regression tests
Library  SeleniumLibrary
Library    XML


*** Variables ***
${BASE_URL}     https://fip-AAA-BBB-CCC-DDD.kaj.poutavm.fi/en/     #page to be tested
#PRODUCTS LINKS
${PRICES_DROP}    //div[@class='col-md-6 wrapper']/ul/li/a[contains(., 'Prices drop')]
${HEADER_TEXT}    //section[@id='main']/h1           #Page header for products pages
${NEW_PRODUCTS}    //div[@class='col-md-6 wrapper']/ul/li/a[contains(., 'New products')]
${BEST_SELLERS}    //div[@class='col-md-6 wrapper']/ul/li/a[contains(., 'Best sales')]
#SIGN IN
${SIGNIN_BUTTON}    class=user-info                   #Sign in button
${SIGNIN_SUBMIT}    id=submit-login
${LOGIN_EMAIL}        id=field-email
${LOGIN_PASSWORD}    id=field-password
${SIGN_OUT}            //div[@class='user-info']/a[contains(., 'Sign out')]
#ADD TO CART
${CLOTHES_TOP}        id=category-3                             #Link to clothes in top bar
${TSHIRT_HUMMINGBIRD}    //img[@alt = 'Hummingbird printed t-shirt']
${ADD_CART}            //div[@class='add']/button
${ADD_SUCCESSFULL}    //div[@class='modal-header']/h4[text()='Product successfully added to your shopping cart']



*** Keywords ***

Open Browser To Homepage
    Open Browser    ${BASE_URL}    headlessfirefox


Click Prices Drop Link In Footer
    Click Element        ${PRICES_DROP}
    Element Should Contain    ${HEADER_TEXT}    PRICES DROP

Click New Products Link In Footer
    Click Element    locator=${NEW_PRODUCTS}
    Element Should Contain    ${HEADER_TEXT}    NEW PRODUCTS

Click Best Sales Link In Footer
    Click Element    locator=${BEST_SELLERS}
    Element Should Contain    ${HEADER_TEXT}    BEST SELLERS


Click Sign In link
        Click Element    ${SIGNIN_BUTTON}

Type In Username
        [Arguments]    ${username}
        Input Text    ${LOGIN_EMAIL}    ${username}

Type In Password
        [Arguments]   ${password}
        Input Text    ${LOGIN_PASSWORD}    ${password}

Submit Login
        Click Element    ${SIGNIN_SUBMIT}
        Title Should Be    My account

Log Out
        Click Element    ${SIGN_OUT}
        Element Should Contain    ${SIGNIN_BUTTON}    Sign in

Click Clothes Link In Top Bar
        Click Element    ${CLOTHES_TOP}

Click Hummingbird T-Shirt
        Click Element    ${TSHIRT_HUMMINGBIRD}

Add To Cart
        Click Element    ${ADD_CART}
        Wait Until Page Contains Element   ${ADD_SUCCESSFULL}
        Wait Until Element Contains   ${ADD_SUCCESSFULL}    Product successfully added to your shopping cart
        Element Should Contain    ${ADD_SUCCESSFULL}    Product successfully added to your shopping cart


*** Test Cases ***
Footer links test
    Open Browser To Homepage
    Click Prices Drop Link In Footer
    Click New Products Link In Footer
    Click Best Sales Link In Footer
    [Teardown]      Close Browser

Sign in and out
    Open Browser To Homepage
    Click Sign In link
    Type In Username    working@email
    Type In Password    WorkinPassword
    Submit Login
    Log Out
    [Teardown]      Close Browser

Add item to cart    
    Open Browser To Homepage
    Click Clothes Link In Top Bar
    Click Hummingbird T-Shirt
    Add To Cart 
    [Teardown]      Close Browser

When run the output should look like this:

==============================================================================
Presta Regression :: Regression tests
==============================================================================    
Footer links test                                                     | PASS |
------------------------------------------------------------------------------    
Sign in and out                                                       | PASS |
------------------------------------------------------------------------------    
Add item to cart                                                      | PASS |
------------------------------------------------------------------------------    
Presta Regression :: Regression tests                                 | PASS |    
3 tests, 3 passed, 0 failed
==============================================================================    
Output:  Path_to\output.xml
Log:     Path_to\log.html
Report:  Path_to\report.html

To demonstrate how the output looks when a test fails, I changed the user password to a wrong one on the "Sign in and out":

==============================================================================
Presta Regression :: Regression tests
==============================================================================
Footer links test                                                     | PASS |
------------------------------------------------------------------------------
Sign in and out                                                       | FAIL |    
Title should have been 'My account' but was 'Login'.
------------------------------------------------------------------------------    
Add to cart                                                           | PASS |
------------------------------------------------------------------------------
Presta Regression :: Regression tests                                 | FAIL |
3 tests, 2 passed, 1 failed
==============================================================================
Output:  Path_to\output.xml
Log:     Path_to\log.html
Report:  Path_to\report.html

As can be seen, the "Sign in test" shows a fail and an explanation why it failed, in this case the "Sign out" link wasn't found because the sign in itself wasn't successful with the wrong password. After getting the report and seeing there is a fail, you would manually test only this failed test case and see what is wrong. For even more detailed information read the report.html file.

If you run the test locally and it has a failure, in VSCode you might need to manually press continue in the interface that appears at the top of VSCode window, when the test is running. To avoid this run the keywords with Run Keyword And Continue On Failure, for example the failed Test Case Sign in and out would look like: 

Sign in and out
    Open Browser To Homepage
    Click Sign In link
    Type In Username    working@email
    Type In Password    WrongPassword
    Run Keyword And Continue On Failure  Submit Login
    Run Keyword And Continue On Failure  Log Out
    [Teardown]      Close Browser
Running it in the CI/CD pipeline with the guide mentioned earlier, this is not needed.

Troubleshooting Common Failures#

Even well-written regression tests can sometimes fail due to environment conditions, timing issues, or application changes.
This section lists common problems you might encounter and their possible solutions.

Issue Possible Cause Solution
Element not found Page load delay, network latency, or changed locator Add dynamic waits like Wait Until Element Is Visible or update the element locator.
Login test fails intermittently Session cookies, cache interference, or stale browser state Use Delete All Cookies before login or start a new browser session for each test.
Add to cart test fails Timing issue with the confirmation modal or slow animation Add a short wait before verifying the success message (e.g., Wait Until Element Is Visible).
Browser does not open in CI No GUI or display server available Run tests using headlesschrome or headlessfirefox mode.
Tests pass locally but fail in pipeline Different browser versions, base URLs, or missing environment variables Check the CI configuration and use environment variables for credentials and URLs.
Test randomly fails on slower systems Resource contention or asynchronous loading Increase timeout values or use conditional waits instead of fixed sleep delays.

If the issue persists, increase the log level to capture more details:

robot --loglevel DEBUG tests/regression_tests.robot

Further Reading and Sources#