Stress testing asynchronous REST service with Gatling

In this blog post, I am not going to deep dive into application performance testing theory nor methodology but rather focus on explaining more advanced features of Gatling performance testing tool on the relatively common scenario when testing asynchronous service. Asynchronous service we are going to test in this post, report generation service, has following API specification:

  1. A report is requested for a given userId via POST method on /report endpoint and in the response request tracking reportId is returned
  2. Report status is tracked via GET method on /report/{reportId} when response status code is 202 Accepted it means that report generation is still in progress. 5xx status code means error during generation. When the report is done service returns 200 OK and the generated report.

Gatling testing tool allows you to create any testing scenario. Testing synchronous service is pretty simple – call the service with the request payload, get the response and assert the content. Performance, in this case, is relatively simple: response time of single service, error rates etc. However, testing an asynchronous service is a bit more difficult as it cannot be done via a single call. Testing scenario needs to follow the structure:

  1. request a service with given parameters and store id for this particular request
  2. poll the request status for tracing the progress
  3. download result when the request is completed

Performance of the service cannot be measured directly as processing is no longer bounded by a single call. What we would like to measure is processing time since the report is requested until processing is complete.

Test scenario can be nicely transformed into test scenario in Gatling DSL:

 val generateReport = scenario("Generate report")
 .exec(requestReport)
 .exec(pollReportGenerationState)
 .exec(getGeneratedReport)

Each test step contains validations that assure correctness of test scenario for a given virtual user. Test scenario relies on Gatling sessions which allow us to carry over user specific data between test steps. In this particular case, we need user specific reportId in order to track request progress and finally collect the report. ReportId is parsed from JSON Report generation response :

 private val requestReport: HttpRequestBuilder = http("Request report generation")
 .post("/report")
 .queryParamMap(withApiKey)
 .body(StringBody(reportRequestBody(userId, reportParameter)))
 .asJSON
 .check(status is 202)
 .check(jsonPath("$..reportId").ofType[String].exists.saveAs("reportId"))

Probably most interesting part is polling the report generation state where we need to keep a virtual user in polling mode until the report is ready. Again we are going to use Session in combination with asLongAs looping operator:

 private def notGenerated(sess: Session): Validation[Boolean] = {
val reportInProgress = 202
val generationStatus = sess("generationStatus").asOption[Int].getOrElse(reportInProgress)
logger.debug(s"Report generation status: $generationStatus")
generationStatus == reportInProgress
}

private val pollReportGenerationState = asLongAs(notGenerated)(
pause(100.millis)
.exec(
http("Poll report generation state")
.get("/report/${reportId}")
.queryParamMap(withApiKey)
.asJSON
.check(status saveAs ("generationStatus"))
)
)

When working with asynchronous service another operator worth consideration is “tryMax”. The disadvantage of this operator is that request failed are counted towards failed requests what in our case would drastically falsify results. Silencing the request would erase this test step completely.

Collecting report when is generated is pretty straightforward. In order to collect statistics on generating report scenario – time to get the report, we need to wrap the part of the scenario in group combinator which will result in group statistics for that group.

val generateReport = scenario("Generate report").group("Generation report completion"){
exec(requestReport)
.exec(pollReportGenerationState)
.exec(getGeneratedReport)
}

In this blog post, we used more advanced features of Gatling performance testing tool when testing asynchronous service using virtual users sharing important data between test steps via Session. Complete code snippet can be found on my GitHub gist. If you use different way when testing asynchronous service or any suggestion how to support custom metrics in Gatling report let me know in comment section below this blog post.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s