Introduction to Load Testing with Gatling – Part 4

By | May 6, 2016
Gatling logo, used with permission.

Gatling logo, used with permission.

Update 2020-04-07: This article has been updated for Gatling 3.3.1 as described here.

This is the fourth, and last, part in a series of posts in which I will present a written version of a workshop I recently held on load testing with Gatling. The target audience of the workshop was Java developers without any previous knowledge of Gatling and this series of articles is not different in this area.
Since I myself like concrete examples, the workshop was based a number of examples. A project containing all the examples is available on GitHub.
Earlier articles in the series are: Part 1part 2 and part 3.

Example 6 – Checks and Assertions

In this, the sixth, example I will show how to perform checks on responses and assertions on statistics of the entire simulation or part of it.

package se.ivankrizsan.gatling.simulations

import io.gatling.core.Predef._
import io.gatling.core.structure.ScenarioBuilder
import io.gatling.http.Predef._
import io.gatling.http.protocol.HttpProtocolBuilder
import io.gatling.http.request.builder.HttpRequestBuilder.toActionBuilder

/**
  * Example Gatling load test that sends one HTTP GET requests to a bad URL.
  * The resulting HTTP status code should be 404, which is verified in the simulation.
  * In addition, we also verify that the maximum response time during the simulation is less than 200 ms.
  * Run this simulation with:
  * mvn -Dgatling.simulation.name=HttpSimulation6 gatling:test
  *
  * @author Ivan Krizsan
  */
class HttpSimulation6 extends Simulation {
    val theHttpProtocolBuilder: HttpProtocolBuilder = http
        .baseUrl("https://computer-database.gatling.io")

    val theScenarioBuilder: ScenarioBuilder = scenario("Scenario1")
        .exec(
            http("Bad Request")
                .get("/unknown")
                /*
                 * Check the response of this request. It should be a HTTP status 404.
                 * Since the expected result is 404, the request will be verified as being OK
                 * and the simulation will thus succeed.
                 */
                .check(
                    status.is(404),
                    regex("Action not found").count.is(2)
                )
        )

    setUp(
        theScenarioBuilder.inject(atOnceUsers(1))
    )
        /*
         * This asserts that, for all the requests in all the scenarios in the simulation
         * the maximum response time should be less than 500 ms.
         * If this is not the case when the simulation runs, the simulation will considered to have failed.
         */
        .assertions(
            global.responseTime.max.lte(500),
            forAll.failedRequests.count.lte(5),
            details("Bad Request").successfulRequests.percent.gte(90)
        )
        .protocols(theHttpProtocolBuilder)
}

Note that:

  • The scenario “Scenario1” sends a HTTP GET request to /unknown relative to the base URL.
    There is no such URL and thus the request should return the 404 HTTP status code.
  • After the GET request there is a call to a method named check on row 31.
    As the parameter(s) to this method we supply one or more checks that we wish to perform on the response.
    In this example we want to assert that the HTTP status is 404.
  • Chained onto the invocation of the setUp method, there is a call to a method assertions on row 45.
    As parameters to this method we can make assertions on statistics from either all requests (global), each individual request (forAll), a group of requests (details) or a single named request (details).
  • On row 46 there is an assertion that the maximum response time of all requests in the simulation must not exceed 500 milliseconds.
  • On row 47 there is an assertion that for each request, the number of failures must be less than five.
  • On row 48 there is an assertion that for the request with the name “Bad Request”, the number of successful requests must be greater than 90%.

Run the simulation with the following terminal command:

mvn -Dgatling.simulation.name=HttpSimulation6 gatling:test

The result should be a successful simulation.

Tweak the parameters in the assertions and re-run the simulation or try to add additional checks or assertions and see what happens.

Example 7 – Multiple Checks and Verification of Request Payload

The seventh example shows how to make multiple checks on a response and, as one of check-types, make a check against the payload of the response.

package se.ivankrizsan.gatling.simulations

import io.gatling.core.Predef._
import io.gatling.core.structure.ScenarioBuilder
import io.gatling.http.Predef._
import io.gatling.http.protocol.HttpProtocolBuilder
import io.gatling.http.request.builder.HttpRequestBuilder.toActionBuilder

/**
  * Example Gatling load test that sends one HTTP GET requests to a URL.
  * The resulting HTTP status code should be one of the status codes in a list of expected status codes.
  * The response body is examined and the request is considered to have failed if the specified regular
  * expression is not matched.
  * Run this simulation with:
  * mvn -Dgatling.simulation.name=HttpSimulation7 gatling:test
  *
  * @author Ivan Krizsan
  */
class HttpSimulation7 extends Simulation {
    val theHttpProtocolBuilder: HttpProtocolBuilder = http
        .baseUrl("https://computer-database.gatling.io")

    val theScenarioBuilder: ScenarioBuilder = scenario("Scenario1")
        .exec(
            http("Request Computers List")
                .get("/computers")
                /* Several checks on the response can be specified. */
                .check(
                    /* Check that the HTTP status returned is 200 or 201. */
                    status.find.in(200, 202),
                    /* Check that there is at least one match of the supplied regular expression in the response body. */
                    regex("Computer database").count.gte(1)
            )
        )

    setUp(
        theScenarioBuilder.inject(atOnceUsers(1))
    ).protocols(theHttpProtocolBuilder)
}

Note that:

  • The status check on row 30 checks against a range of status codes.
    In the example the expected status codes are just three; 200, 201 and 202.
  • There is an additional parameter to the check method.
    As before, checks are added as parameters to the check method.
  • The additional parameter to the check method starts with a call to a method named regex.
    The regex method searches the response body for the supplied regular expression, in this case just the string “Computer database”.
    The methods chained onto regex first returns the number of occurrences of the search expression and then asserts that the number is greater than or equal to one.

Run the simulation with the following terminal command:

mvn -Dgatling.simulation.name=HttpSimulation7 gatling:test

The result should be a successful simulation.

Change the parameter to the gte method on row 32 to 5 and re-run the simulation.
There will be an error in the simulation that renders the following output to the terminal:

---- Errors --------------------------------------------------------------------
> regex(Computer database).count.greaterThanOrEqual(5), but actu      1 (100.0%)
ally 1 is not greater than or equal to 5
================================================================================

From the error message we can see that there were one single occurrence of the string, not five as the simulation expected.

Example 8 – Simulation Inheritance

Recall the first part of this series, in which I mentioned that Gatling simulations are Scala classes. With a background in object-oriented programming, I naturally start to think about whether it is possible to create some kind of template for a certain type of load-tests and then, through the use of inheritance, define load-tests for specific cases without having to duplicate the common parts.
My goal was to be able to only set values of instance variables in my subclasses. In order to accomplish this, I created this base-class for my load-test simulations:

package se.ivankrizsan.gatling.common

import io.gatling.core.Predef._
import io.gatling.core.structure.ScenarioBuilder
import io.gatling.http.Predef._
import io.gatling.http.protocol.HttpProtocolBuilder
import io.gatling.http.request.builder.HttpRequestBuilder.toActionBuilder

import scala.concurrent.duration._

/**
  * Abstract Gatling simulation class that specifies properties common to multiple Gatling simulations.
  * Certain simulation parameters have been extracted to class variables which allows for subclasses
  * to modify certain behaviour of the basic simulation by setting new values of these variables.
  *
  * @author Ivan Krizsan
  */
abstract class HttpSimulationBaseClass extends Simulation {
    /* Base URL of requests sent in scenario 1. */
    protected var scenario1BaseURL = ""
    /* Additional request path appended after the base URL in scenario 1. */
    protected var scenario1RequestPath = ""
    /* Final number of users in the simulation. */
    protected var finalUserCount = 10
    /* Time period after which the number of users is to reach the final number of users. */
    protected var userCountRampUpTime : FiniteDuration = (20 seconds)

    /**
      * Creates the HTTP protocol builder used in the simulation.
      */
    def createHttpProtocolBuilder(): HttpProtocolBuilder = {
        val httpProtocolBuilder = http
            .baseUrl(scenario1BaseURL)
            .acceptHeader("text/plain")
            .userAgentHeader("Gatling")
        httpProtocolBuilder
    }

    /**
      * Creates the scenario builder used in the simulation.
      */
    def createScenarioBuilder(): ScenarioBuilder = {
        val scenarioBuilder = scenario("Scenario1")
            .exec(
                http("myRequest1")
                    .get("/" + scenario1RequestPath)
            )
        scenarioBuilder
    }

    /**
      * Performs setup of the simulation.
      * Needs to be called from the constructor of child classes, after the instance
      * variables have been initialized, in order to create a scenario using the instance
      * variable values of the child classes.
      */
    def doSetUp(): Unit = {
        val theScenarioBuilder = createScenarioBuilder()
        val theHttpProtocolBuilder = createHttpProtocolBuilder()

        setUp(
            theScenarioBuilder.inject(rampUsers(finalUserCount).during(userCountRampUpTime))
        ).protocols(theHttpProtocolBuilder)
    }
}

Note that:

  • The HttpSimulationBaseClass is an abstract class that extends the Gatling class Simulation.
  • There are four instance variables defined in this class.
    scenario1BaseURL – This will be the parameter to the HttpProtocolBuilder method baseURL and the base URL of all requests in simulations inheriting from this class. Default value is the empty string.
    scenario1RequestPath – Additional URL path that will be appended to the base URL for the request in scenario one of simulations inheriting from this class. Default value is the empty string.
    finalUserCount – The maximum number of simulated users that the simulation will ramp up to. Default value is 10.
    userCountRampUpTime – A duration specifying the time during which the simulated users will ramp up.
  • In the Gatling 2.3 version of the example, there was a before block in which the doSetUp method was called.
    This no longer works with Gatling 3 and, as pointed out in the comments, there were problems with this approach even earlier.
  • There is a method named createHttpProtocolBuilder.
    As the name implies, this method creates a HTTP protocol builder. The contents of the methods should be familiar, but earlier the resulting HTTP protocol builder was assigned to an immutable instance variable directly. The instance variable scenario1BaseURL is used to set the base URL of the HTTP protocol builder. Note that in Scala, the return keyword can be omitted, as in this method.
  • There is a method named createScenarioBuilder.
    Not surprisingly this method creates a Gatling simulation scenario builder. In this method, the instance variable scenario1RequestPath is used as parameter to the get method.
  • Finally there is a doSetUp method.
    This method calls the two methods seen earlier to create a HTTP protocol builder and a scenario builder and then invokes the setUp method we have used in our load-simulations earlier. In this method, the instance variables finalUserCount and userCountRampUpTime are used as parameters when specifying the number of users and the ramp-up time for users. As will be shown below, this method must be called from the constructor of subclasses to this class after all instance variables have been set.

Nice, but we cannot run this as it is. Lets look at an example showing how to use such an abstract base-class for load-simulations:

package se.ivankrizsan.gatling.simulations

import se.ivankrizsan.gatling.common.HttpSimulationBaseClass

import scala.concurrent.duration._

/**
  * Example Gatling load test that is based on an abstract base class.
  * Run this simulation with:
  * mvn -Dgatling.simulation.name=HttpSimulation8 gatling:test
  *
  * @author Ivan Krizsan
  */
class HttpSimulation8 extends HttpSimulationBaseClass{
    scenario1BaseURL = "http://computer-database.gatling.io"
    scenario1RequestPath = "computers"
    finalUserCount = 4
    userCountRampUpTime = (5 seconds)

    /*
     * doSetUp needs to be called here, after the instance variables have been
     * initialized but before the check for a scenario has been performed.
     * The call to doSetUp() cannot be located to a before-hook since it is called too late.
     */
    doSetUp()
}

Note that:

  • The class HttpSimulation8 extends the abstract base-class we just created.
  • This class contains four assignments to the instance variables from the base-class.
  • The doSetUp method is called immediately after having assigned values to the instance variables.
    This is done in order to create a scenario before the check in the Simulation class that there is indeed a scenario.

Run the simulation with the following terminal command:

mvn -Dgatling.simulation.name=HttpSimulation8 gatling:test

The result should be a successful simulation with four requests being sent.

This concludes this series in which I introduced load-testing with Gatling.

Happy coding!

9 thoughts on “Introduction to Load Testing with Gatling – Part 4

  1. Freeman

    Very interesting buddy.

    A question, how i can inherit a Objet from another Scala file(another class)?

    In this Class i have a web service request

    Regards!!

    Reply
    1. Ivan Krizsan Post author

      Hello!
      I don’t think I entirely understand your question, but I guess that you want to access an instance field/property or method in an instance of another Scala class.
      Since Gatling simulations are Scala code, you could create an instance of the other class in the Gatling simulation and invoke the method, or access the property, on the instance from the Gatling simulation.
      Hope this helps!

      Reply
  2. suresh

    Hello,
    The inherited sample simulated does need any changes? when i tried to execute I am getting the below error.

    Caused by: java.lang.IllegalArgumentException: requirement failed: No scenario set up
    at scala.Predef$.require(Predef.scala:277)
    at io.gatling.core.scenario.Simulation.params(Simulation.scala:99)
    at io.gatling.app.Runner.run0(Runner.scala:81)
    at io.gatling.app.Runner.run(Runner.scala:64)
    at io.gatling.app.Gatling$.start(Gatling.scala:59)
    at io.gatling.app.Gatling$.fromArgs(Gatling.scala:43)
    at io.gatling.app.Gatling$.main(Gatling.scala:35)
    at io.gatling.app.Gatling.main(Gatling.scala)
    … 6 more
    I am using Gatling 2.3.1 version and running my command as below using maven
    mvn -Dgatling.simulationClass=simulations.HttpGETSimulation clean gatling:test

    Reply
    1. Ivan Krizsan Post author

      Hello!
      Have you tried running the unchanged example project from GitHub?
      It is difficult to tell what could be your problem from only looking at the enclosed stacktrace.
      Happy coding!

      Reply
    2. Poliana

      I got the same error…. So i removed the “before { doSetUp() }” from the HttpSimulationBaseClass and added the “doSetUp();” to the HttpSimulation8 class and it worked. I dont know why it isnt working into the before section.

      Reply
  3. Antonio

    It’s possible multi scenarios in the same simulation?
    Example:
    I have to execute the same test during 2H, but in each interval time (10min), I have to inject different rate user/second.
    I have tests with:
    setUp(
    scenario1.inject(atOnceUsers(1)).scenario2.inject(2)
    ).protocols(httpProtocolBuilder)

    but according to results, only executes the last scenario configuration, in example case, for 2 users.
    Someone could help me?

    Reply
  4. Varun Deshpande

    Hi,
    Nice article to explain the assertions. Is it possible to build a list of assertion outside and then pass.

    Reply
    1. Ivan Krizsan Post author

      Hello!
      I assume, without having checked in detail, that assertions(…) is some form of invocation, method invocation for instance. I assume that the parameter to assertions() can then be constructed in a manner that you see fit for your purposes – perhaps you want to implement an assertion language and store assertions in a database and, when running your load-tests, build the assertions from the data in the database.
      So I guess the answer to your question is yes, though I must admit that I have only used the basic technique outlined in these articles.
      Happy coding!

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *