jmeter-ec2 is a simple, freely-available shell script that automates running Apache JMeter on Amazon EC2 or any other server. It does things like:
- Using Amazon’s API to launch instances
- Installing JAVA & JMeter
- Copying test files to each server
- Adjusting thread counts to ensure the load is evenly distributed over each host
- Editing the jmx for file paths to external data files
- Displaying real-time aggregated results from the test as it is running
- Downloading and collating all jtl files
- Terminating instances.
Jump to Example for an idiot-level step by step example.
UPDATE: The name ‘jmeter-ec2′ is actually something of a misnomer now as I recently added the ability to specify a list of already existing hosts that the the script will run jmeter on (via the REMOTE_HOSTS property). These hosts can be any number of linux based machines in any locations that you have access to, you are not limited to using Amazon EC2.
Even though ‘it’ has a ‘name’, it’s just a shell script. As a shell script, it is mainly designed to improve the efficiency of an otherwise repetitive task. It is not an application, it does not write tests scripts, identify bottlenecks or make you better at your job. About the most advanced thing it does do is aggregate the output from the Generate Summary Results listener to the screen, which is basic math. That said, it does save an awful lot of time.
[Source: https://github.com/oliverlloyd/jmeter-ec2 with usage instructions in the README file]
The Cloud and Load Testing
One of the things the Cloud is useful for is load testing; very large amounts of hardware can be used to generate load at minimal cost with the added benefit that, if your application you are testing is external to your corporate network, your tests will be run from a realistic location which prevents any problems with artificial bottlenecks occurring on your LAN. This type of testing, using externally located hosts, is increasingly common and JMeter is a superb open-source solution for facilitating this. There are also several excellent paid for options for running load tests from the Cloud. SOASTA are, in my opinion, developing a very interesting offering in this area – they even allow tests up to 100 threads/users for free using their Cloud Test Lite solution. But if you are comfortable using Apache JMeter, do not need advanced analytics or presentation and if you have a requirement for using multiple externally-hosted hosts for load testing then JMeter and EC2 will probably do the job.
There are still cases where a traditional test lab located internally can and should be used for load testing but there is currently a sea change happening in the industry where more and more functionality is being delivered via the SaaS or PaaS model and in my view our approach to Performance Testing needs to adapt to this. Fundamentally, if the application that is being tested is remotely hosted then the test rig should be also.
Using JMeter on Amazon EC2
Running JMeter in the Cloud is not as simple as you might think. Certainly, assuming you run JMeter from the command line then spinning up a single instance and running a test from there is not especially difficult, but running a test over multiple instances is time consuming.
Some problems with trying to run JMeter in the Cloud:
- If you want to use your local machine to control the test you have to navigate the joyous process of getting RMI to work over multiple subnets and firewalls to allow your local machine to control multiple remote (EC2) slaves – you can do this by tunnelling RMI communication and patching JMeter, it’s messy though. To workaround this issue you have to use a remote Master as well as remote Slaves.
- Even then, in master/slave mode when running high throughput tests JMeter will eventually reach an IO or network bottleneck that will affect the results (even using Batch mode). Too many processes trying to write to a single file at the same time inevitably start to queue. To be certain of avoiding this issue you have to not use Distributed mode and instead run multiple independent tests over n hosts that do not use the GUI and do not write results to a single master but instead run at the command line and keep results local. Then, after the test is complete, you have to collate everything. This approach has several annoying problems: a) excessive terminal windows, b) limited visibility on test progress as it happens, c) to preserve throughput the jmx file must be adjusted for the number of hosts in use and d) too much time is spent on repetitive tasks that could be automated.
jmeter-ec2 Shell Script – Details
To get around the problems listed above and improve my life/work balance I wrote a shell script and then GitHub made me invent a name for it.
This file contains several properties required for running the script. Most of these relate to your Amazon account and should already have been setup as part of your Amazon AWS registration. This file should have executable permissions (it is ‘run’ to set each property).
In place of using Amazon to create the hardware it is also possible to specify a list of hosts to run the test over. If the REMOTE_HOSTS property is populated in this file then the script will use these machines instead.
This is the main shell script. It should have executable permissions.
Used to install Java & JMeter on each slave instance (not to run locally!)
jmeter & jmeter.properties
These are edited versions of two of JMeter source files. They are included here to enable certain features of the script. After JMeter is installed, if these files are present in the root directory then they are uploaded in place of the default files.
The values changed from their defaults are:
HEAP="-Xms2048m -Xmx2048m" -
OPTIONAL (specific to your test)
NEW="-XX:NewSize=256m -XX:MaxNewSize=256m" -
specific to your test
jmeter.save.saveservice.output_format=csv - REQUIRED, DO NOT CHANGE
jmeter.save.saveservice.hostname=true - OPTIONAL (but useful in this context)
REQUIRED, DO NOT CHANGE
summariser.interval=15 - OPTIONAL (should be tuned)
The main change is to set the output format to csv. Without this the script will not be able to collate the results and they will be corrupted, but the test will still run. The post processing step requires that thread counts be present as it uses a column index to adjust them for the number of hosts used in the test.
- That the testplan to be run has a Generate Summary Results listener*. See here.
- That your JMeter testplan has a duration or loop count value set**. See here.
[*Without this no results will be displayed to the screen but the test will still run. No other listeners need to nor should be present.]
[**Without this the test will run forever or until you press CTRL-C. All testplans should also employ some form of pacing as best practice – load tests should not be run without some way to control the throughput. One way this can be achieved in JMeter is using the Constant Throughput Controller.]
Prerequisits specific to using Amazon:
- That you have an Amazon AWS account. See here.
- You have Amazon’s API tools installed on your machine. See here.
Step by Step Instructions:
STEP 1 - First create a directory called something like /Users/oliver/jmeter-ec2. This is the script home and it is where you will place your testfiles, project by project.
STEP 2 – Clone / Fork the files from GitHub placing the files into the directory just created. Tip: For simple read-only snapshots use the ZIP button to download.
Note. myproject.jmx is simply a dummy testplan but it demonstrates the structure required for the script to work.
Any jmx testplan used should have:
1. A Generate Summary Results listener.
2. Duration or Loop Count should be set to a fixed value. This is not essential as the test can be shutdown using CTRL-C, the script will capture this and process any results up to this point.
STEP 5 – If your testplan uses any external files then copy these into the /data directory under your project. For example. If your testplan foobar.jmx references a file mydatafile.csv then copy this into /foobar/data. During execution the script will automatically adjust any references in the jmx file to point to the remote version of this file.
STEP 6 – The final step is to update the values in the jmeter-ec2.properties file. If using Amazon then these need to correspond with your Amazon AWS account, the AMI you intend to use and the directory you created above. If not using Amazon then REMOTE_HOSTS needs to be set to a valid, comma-separated list of hostnames.
# This is a java stye properties file for the jmeter-ec2 shell script
# It is treated like a normal shell script and must have executable permissions
# See README.txt for more details about each property
LOCAL_HOMEis the directory you created on your computer.
PEM_PATHrelate to your Amazon account.
USERall relate to the AMI used.
REMOTE_HOSTSis an optional list of hosts to use in place of creating new ones.
can be left as shown.
The test can now be run; open a terminal and type:
$ cd /Users/oliver/jmeter-ec2
$ ./jmeter-ec2.sh myproject 1
If you copied your own foobar.jmx file into the /jmx directory in step 4 then use:
$ ./jmeter-ec2.sh foobar 1
1‘ is the count of hosts to request from Amazon, you can specify as many as your AWS account allows. If you try to launch more than you are allowed Amazon will only return the maximum possible according to your account limits, the script will adjust for this. If you are specifying your own hosts then this value is ignored.
If you have more than one project in the /jmeter-ec2 directory then in the command you would replace ‘myproject’ with ‘anotherproject’. For example, if you wanted to run anotherproject using 5 Amazon instances you would type:
$ ./jmeter-ec2.sh anotherproject 5
Finaly, if you do not want to use Amazon and already have a list of valid hostnames to run the test over then you could simply call:
$ ./jmeter-ec2.sh myproject
and the test will be run over as many hosts were specified in the jmeter-ec2.properties file.
Project Source: https://github.com/oliverlloyd/jmeter-ec2