Apache JMeter is a free, open-source load testing tool with a rich set of features that enable the creation of advanced, transactional test scripts. It is, I think, the leading open source load testing tool available but it is often seen as having a steep learning curve. This isn’t because JMeter is complicated, it’s actually very simple to use, but it does have a non-standard interface which can be initially confusing, especially to those used to using other load test tools. This post is an attempt to smooth this learning curve by explaining some of the core concepts.
So, the first thing you need to do is install JMeter. This is very easily done because JMeter runs in Java, and the chances are you’re reading this on a machine that has a Java Runtime Environment already present.
1. Download the latest version from here. For simplicity, chose one of the ‘binary’ files.
2. Create a directory, maybe `c:\apache-jmeter\v2-8\` or `/Applications/apache-jmeter-v2.8/`, you get the idea.
3. Unzip the file to this directory
That’s it. You now have JMeter installed. To run it either execute `/Applications/apache-jmeter-v2.8/bin/jmeter` or double click the `c:\apache-jmeter\v2-8\jmeter.bat` file.
The JMeter GUI
This will open the JMeter GUI which will look a lot like this:
Default JMeter GUI
Here is an example test file that you can download and open using JMeter. Contained in this test plan are some the basic elements for a load test against a typical web site. There are other features you might want to add in, such as correlation, loops etc., but this is a start to explain the interface.
The Tree View
An important concept to understand in JMeter is the use of the tree model. This is displayed in the panel to the left and it is the heart of how tests are put together in JMeter, it decides not only execution order but also the scope of different controls. In fact, it decides everything – in this one place you build the entire test. Here you can:
- add new requests
- perform correlation
- add verification checks
- define configuration for cookies, headers etc.
- define virtual users (threads)
- insert think time
- define execution loops
- add while / for loops
- add if statements
- define pacing
- add transactions
- configure rampup / duration values
- define scheduling
- configure groups of scripts
- decide execution order
- really, everything…
The tree view is a graphical representation of the test plan. JMeter uses XML to store your test plan and it is this XML that is being shown here. In the same way that XML uses parent child relationships to decide scope, JMeter decides the scope of each control based on its position in the tree.
You’ll notice that there are multiple Thread Groups in this example – loginAndBrowse & staticPages. JMeter uses Thread Groups to separate groups of requests (samples). Each group can be assigned a set of configuration options that decides how many threads (virtual users) are run, how many loops (iterations) are executed and also other values such as rampup and duration. Typically, each Thread Group runs in parallel and execution order in JMeter is simply top down within each Thread Group.
Configure a Thread Group
A Thread Group is best thought of as a script, each group is (normally) run in isolation and in parallel to other groups. In LoadRunner, a virtual user script is the same as a thread group. Inside a thread group, rather than a script view containing code, you have a sub tree containing graphically represented requests (samples) that are executed in order, from the top down. The next sampler (request) will not start until the previous sample is complete. This is how JMeter defines journeys through a site.
As previously mentioned the postion of an element in the tree decides its scope, Ie. it decides the other controls that are affected by it. For example, in our example there is a control called HTTP Cookie Manager. This control is telling JMeter to manage cookies – simply by including it you tell JMeter to react to any cookies sent by the server, keep them in memory for each user (thread) and send then back for subsequent calls, it is fully automatic. But in this case we have positioned it inside the loginAndBrowse thread group so it will only apply to samplers inside the same group. In other words, the scope of this control is limited to this Thread Group. We could also have put the HTTP Cookie Manager at the root of the testplan like this:
With the cookie manager at the root of the test plan it will apply to all samplers, in all thread groups.
Another example of scope is shown with this Response Assertion:
A Response Assertion Limited to one Sampler
Because this control is a child of a single sampler, the only request affected by it is its parent, the login request. You could place this assertion higher up the tree and it would check the response of multiple requests. You could even place it at the root of the testplan and it would check every single request. A use case for this is setting a negative assertion, you might have a generic error page for you site that returns something like: “Aw Shucks, Something Went Wrong”. The configuration shown below will check each page for this error text and only pass a request if it is NOT found. Note how the ‘Pattern Matching Rules’ are set to ‘NOT’.
An example of a global error check
A final example of scope is Timers. Timers are instructions to JMeter to wait for a period of time before continuing. Like everything (yes, everything) they follow the same rules of scope. If you do this:
Then this will NOT insert a delay inbetween the login and browse samplers. Instead, it will apply a random wait to every request within its scope. That is, the homepage, login, browse and logout request will all have a ransom wait applied to them. If you want a wait to be specific to a single sampler then do this:
But hey, maybe you want to have every request in this thread group wait for a random time before continuing. Maybe you want all requests in the whole test plan to have think time before them. All you have to do is drag and drop this timer control to the position you want it and the rest is done for you – welcome to the awesomeness of JMeter.
In load testing it’s always a good idea to pace your test so that you are running at a controlled rate. In LoadRunner you do this using Pacing, in JMeter you can use the Constant Throughput Controller. When added this will make sure the requests within its scope are only made once every X times per minute. Like everything in JMeter it follows the same scope rules so if you put it globally your test would, overall, only make X requests per minute, if you put it as a child of a single request then this individual request will only be made X times per minute.
By the way, have you noticed how scope applies to everything? That’s a very cool thing that you will come to appreciate. Not only can you reliably and quickly change the whole setup of a test by simply dragging and dropping controls, but because the interface is a visual tree-view, it’s ridiculously easy to do it as well.
So, an example of pacing is this:
Using a Constant Throughput Controller for Pacing
The homepage request is limited to 1 request per minute, per thread. Because all the requests in a thread group are run sequentially and because each request waits for the one before it, you are effectively pacing the whole script (thread group) for each user (thread). This is a useful way to set pacing for a particular journey.
Controlling Execution with Loops
This is so easy it’s silly. Want to add a loop to your test to make one request 5 times? Do this:
That’s an obvious example to explain the interface but you should know that you can do so much more. For example, you can make a request to a search page and then parse the response to get the url of every results link. Then, you can use a For Each Controller to dynamically call each of the results. Hey, you just built a web crawler!
You can have an IF statement that only executes its children if a certain condition is true. Or a While loop that will run until a condition becomes true.
You don’t have to write code for this stuff, it is all visual, and this makes adding this sort of control to your test plan very easy and very fast.
Listeners do two main things:
- let you visualise the test as it is running (or after it has run by importing the jtl file into them).
- Allow you to debug your test.
OK, so this is not JMeter’s strongest feature, JMeter is not pretty; it’s the geeky one in the corner with thick glasses. But you can make it slightly sexier using this plugin. Just copy the JMeterPlugins.jar file into the /lib/ext/ directory.
Debugging a Test
Use the View Results Tree Listener to debug your tests. It’s a very concise, graphical way to see what requests were made, the response you got for each call and the order they were made. There’s no need to trawl through logs file.
JMeter can be run in two modes, GUI & COMMAND LINE. There’s a good reason for this and, typically, they both have a purpose. The GUI is what you use to create, edit and debug tests with. It is a rich graphical representation of your test plan that – once you get your head round it – allows scripts to be put together in a very short time. You can also run complete tests from the GUI, this is possible, but typically it is better to execute the full, proper test run in COMMAND LINE mode. The reason for this is performance, JMeter will perform much better when not in GUI mode and this is important if you are running heavy load tests.
Running in GUI Mode
Erm, well, just press the green button shaped like an arrow. Or you can just use CTRL-R. It’s big red square or CTRL-. (period) to stop.
Running in CMD LINE Mode
[Note. I'm focusing on unix based systems here but the windows bat file follows very similar conventions.]
Less pretty, MUCH more powerful. Type:
./path/to/jmeter/bin/jmeter -n -t /path/to/my/test.jmx -l /path/where/to/put/results.jtl
./path/to/jmeter/bin/jmeter = run the jmeter process
-n = Run headless, without the GUI (I.e. n = non gui mode)
-t /path/to/my/test.jmx = tell jmeter which test plan to run
-l /path/where/to/put/results.jtl = put results here
When running at the command line you don’t need, nor want Listeners to be enabled. You can disable a control in the GUI by pressing CTRL-T. There’s one exception, you can use a Generate Summary Results listener to get output on the command line.
As ever, there’s a lot more available that just these basics. You can pass in custom variables, define custom properties. You can even integrate this command into a shell script and call it from your build server and hey presto, you have the beginnings of a CI system.
Full Details here. Play.