jmeter-ec2 | Run JMeter on Amazon’s ec2 Cloud

jmeter-ec2 is a simple, freely-available shell script that automates running Apache JMeter on Amazon EC2 or any other server. It does things like:

  • Using Amazon’s API to launch instances
  • Installing JAVA & JMeter
  • Copying test files to each server
  • Adjusting thread counts to ensure the load is evenly distributed over each host
  • Editing the jmx for file paths to external data files
  • Displaying real-time aggregated results from the test as it is running
  • Downloading and collating all jtl files
  • Terminating instances.

Jump to Example for an idiot-level step by step example.

UPDATE: The name ‘jmeter-ec2′ is actually something of a misnomer now as I recently added the ability to specify a list of already existing hosts that the the script will run jmeter on (via the REMOTE_HOSTS property). These hosts can be any number of linux based machines in any locations that you have access to, you are not limited to using Amazon EC2.

Even though ‘it’ has a ‘name’, it’s just a shell script. As a shell script, it is mainly designed to improve the efficiency of an otherwise repetitive task. It is not an application, it does not write tests scripts, identify bottlenecks or make you better at your job. About the most advanced thing it does do is aggregate the output from the Generate Summary Results listener to the screen, which is basic math. That said, it does save an awful lot of time.

[Source: https://github.com/oliverlloyd/jmeter-ec2 with usage instructions in the README file]

The Cloud and Load Testing
One of the things the Cloud is useful for is load testing; very large amounts of hardware can be used to generate load at minimal cost with the added benefit that, if your application you are testing is external to your corporate network, your tests will be run from a realistic location which prevents any problems with artificial bottlenecks occurring on your LAN. This type of testing, using externally located hosts, is increasingly common and JMeter is a superb open-source solution for facilitating this. There are also several excellent paid for options for running load tests from the Cloud. SOASTA are, in my opinion, developing a very interesting offering in this area – they even allow tests up to 100 threads/users for free using their Cloud Test Lite solution. But if you are comfortable using Apache JMeter, do not need advanced analytics or presentation and if you have a requirement for using multiple externally-hosted hosts for load testing then JMeter and EC2 will probably do the job.

There are still cases where a traditional test lab located internally can and should be used for load testing but there is currently a sea change happening in the industry where more and more functionality is being delivered via the SaaS or PaaS model and in my view our approach to Performance Testing needs to adapt to this. Fundamentally, if the application that is being tested is remotely hosted then the test rig should be also.

Using JMeter on Amazon EC2
Running JMeter in the Cloud is not as simple as you might think. Certainly, assuming you run JMeter from the command line then spinning up a single instance and running a test from there is not especially difficult, but running a test over multiple instances is time consuming.

Some problems with trying to run JMeter in the Cloud:

  1. If you want to use your local machine to control the test you have to navigate the joyous process of getting RMI to work over multiple subnets and firewalls to allow your local machine to control multiple remote (EC2) slaves – you can do this by tunnelling RMI communication and patching JMeter, it’s messy though. To workaround this issue you have to use a remote Master as well as remote Slaves.
  2. Even then, in master/slave mode when running high throughput tests JMeter will eventually reach an IO or network bottleneck that will affect the results (even using Batch mode). Too many processes trying to write to a single file at the same time inevitably start to queue. To be certain of avoiding this issue you have to not use Distributed mode and instead run multiple independent tests over n hosts that do not use the GUI and do not write results to a single master but instead run at the command line and keep results local. Then, after the test is complete, you have to collate everything. This approach has several annoying problems: a) excessive terminal windows, b) limited visibility on test progress as it happens, c) to preserve throughput the jmx file must be adjusted for the number of hosts in use and d) too much time is spent on repetitive tasks that could be automated.

jmeter-ec2 Shell Script – Details
To get around the problems listed above and improve my life/work balance I wrote a shell script and then GitHub made me invent a name for it.

jmeter-ec2 running:

Displaying real-time test results:

Completing the test:

Project Files:
Jmeter-ec2.properties
This file contains several properties required for running the script. Most of these relate to your Amazon account and should already have been setup as part of your Amazon AWS registration. This file should have executable permissions (it is ‘run’ to set each property).

In place of using Amazon to create the hardware it is also possible to specify a list of hosts to run the test over. If the REMOTE_HOSTS property is populated in this file then the script will use these machines instead.

jmeter-ec2.sh
This is the main shell script. It should have executable permissions.

install.sh
Used to install Java & JMeter on each slave instance (not to run locally!)

jmeter & jmeter.properties
These are edited versions of two of JMeter source files. They are included here to enable certain features of the script. After JMeter is installed, if these files are present in the root directory then they are uploaded in place of the default files.

The values changed from their defaults are:
[jmeter]:
HEAP="-Xms2048m -Xmx2048m"                 - OPTIONAL (specific to your test)
NEW="-XX:NewSize=256m -XX:MaxNewSize=256m" - OPTIONAL (specific to your test)

[jmeter.properties]:
jmeter.save.saveservice.output_format=csv  - REQUIRED, DO NOT CHANGE
jmeter.save.saveservice.hostname=true      - OPTIONAL (but useful in this context)
jmeter.save.saveservice.thread_counts=true - REQUIRED, DO NOT CHANGE
summariser.interval=15                     - OPTIONAL (should be tuned)

The main change is to set the output format to csv. Without this the script will not be able to collate the results and they will be corrupted, but the test will still run. The post processing step requires that thread counts be present as it uses a column index to adjust them for the number of hosts used in the test.


An Example: (Using Mac OSX)

Prerequisites:

  • That the testplan to be run has a Generate Summary Results listener*. See here.
  • That your JMeter testplan has a duration or loop count value set**. See here.

[*Without this no results will be displayed to the screen but the test will still run. No other listeners need to nor should be present.]

[**Without this the test will run forever or until you press CTRL-C. All testplans should also employ some form of pacing as best practice - load tests should not be run without some way to control the throughput. One way this can be achieved in JMeter is using the Constant Throughput Controller.]

Prerequisits specific to using Amazon:

  • That you have an Amazon AWS account. See here.
  • You have Amazon’s API tools installed on your machine. See here.

Step by Step Instructions:
STEP 1 - First create a  directory called something like /Users/oliver/jmeter-ec2. This is the script home and it is where you will place your testfiles, project by project.

STEP 2 – Clone / Fork the files from GitHub placing the files into the directory just created. Tip: For simple read-only snapshots use the ZIP button to download.

This gives:

STEP 3 – Now extract the contents of example-project.zip to give:

Note. myproject.jmx is simply a dummy testplan but it demonstrates the structure required for the script to work.

myproject.jmx:

STEP 4 – If you would like to execute a jmx file called foobar.jmx then you should copy this into the /myproject/jmx directory and rename the directory /myproject to /foobar.

Any jmx testplan used should have:

1. A Generate Summary Results listener.

2. Duration or Loop Count should be set to a fixed value. This is not essential as the test can be shutdown using CTRL-C, the script will capture this and process any results up to this point.

STEP 5 – If your testplan uses any external files then copy these into the /data directory under your project. For example. If your testplan foobar.jmx references a file mydatafile.csv then copy this into /foobar/data. During execution the script will automatically adjust any references in the jmx file to point to the remote version of this file.

STEP 6 – The final step is to update the values in the jmeter-ec2.properties file. If using Amazon then these need to correspond with your Amazon AWS account, the AMI you intend to use and the directory you created above. If not using Amazon then REMOTE_HOSTS needs to be set to a valid, comma-separated list of hostnames.

jmeter-ec2.properties file:
# This is a java stye properties file for the jmeter-ec2 shell script
#
# It is treated like a normal shell script and must have executable permissions
#
# See README.txt for more details about each property
#
LOCAL_HOME="/Users/oliver/jmeter-ec2"
REMOTE_HOME="/tmp"
AMI_ID="ami-a5e7dad1"
INSTANCE_TYPE="t1.micro"
INSTANCE_SECURITYGROUP="jmeter"
PEM_FILE="olloyd-eu"
PEM_PATH="/Users/oliver/.ec2"
INSTANCE_AVAILABILITYZONE="eu-west-1b"
USER="root"
RUNNINGTOTAL_INTERVAL="3"
REMOTE_HOSTS=""

Where:

  • LOCAL_HOME is the directory you created on your computer.
  • INSTANCE_SECURITYGROUPPEM_FILEPEM_PATH relate to your Amazon account.
  • AMI_ID, INSTANCE_TYPEINSTANCE_AVAILABILITYZONEUSER all relate to the AMI used.
  • REMOTE_HOSTS is an optional list of hosts to use in place of creating new ones.
  • REMOTE_HOME & RUNNINGTOTAL_INTERVAL can be left as shown.

The test can now be run; open a terminal and type:

$ cd /Users/oliver/jmeter-ec2
$ ./jmeter-ec2.sh myproject 1

If you copied your own foobar.jmx file into the /jmx directory in step 4 then use:

$ ./jmeter-ec2.sh foobar 1

1‘ is the count of hosts to request from Amazon, you can specify as many as your AWS account allows. If you try to launch more than you are allowed Amazon will only return the maximum possible according to your account limits, the script will adjust for this. If you are specifying your own hosts then this value is ignored.

If you have more than one project in the /jmeter-ec2 directory then in the command you would replace ‘myproject’ with ‘anotherproject’. For example, if you wanted to run anotherproject using 5 Amazon instances you would type:

$ ./jmeter-ec2.sh anotherproject 5

Finaly, if you do not want to use Amazon and already have a list of valid hostnames to run the test over then you could simply call:

$ ./jmeter-ec2.sh myproject

and the test will be run over as many hosts were specified in the jmeter-ec2.properties file.

 

Project Source: https://github.com/oliverlloyd/jmeter-ec2

61 thoughts on “jmeter-ec2 | Run JMeter on Amazon’s ec2 Cloud”

  1. Hi Oliver,
    Very well done, in the past I had problems with distributed runs and found that it is most reliable to run few separate jmeter instances over several ec2.
    I did all of the above detailed manual steps and wasted a lot of time and it was sucks.
    The good thing about it was that I knew what is the load I want to generate and it was constant, so I created 1 host for it, and then duplicated it over 20 instances to get to the required load.

    To be honest the results gathering was a bitch task to do manually and had no time to write scripts… at that time I took an assumption that statistically, all results should be very close, so I just use one of the 20 results and multiplied them accordingly.

    Anyway I guess that for my up coming projects I may use this script, probably fix it a bit to support JMeterPlugins Ultimate Thread Group, as this is the one I mainly use.

    Thanks for sharing this!

    1. I’ve added a step in install the JMeterPlugins.jar in the install.sh script. This will allow the Ultimate Thread Group functionality you mentioned.

      Your experience sounds very similar to mine. Past a certain point Distributed mode doesn’t work and you get forced into managing multiple separate instances which creates a tiresome overhead in the testing process.

      Scripting it this way is actually a hack, the real solution is to have JMeter write it’s output to a database. Databases are designed to handle very large numbers of insert requests from multiple sources and at the same time you can read the data that is being written in real time. This way you have access to 100% of the data (not just the output from the Generate Summary Results listener). Also, you have the bonus of having the data from each run stored in the same location which then allows you to report not just on one single test but to compare one run to another or even trend over time – very, very useful.

      But, for now, this hack saves some time.

  2. Hi

    Thanks for your work on this.

    Have been using your script and keep getting “no instances successfully initialised, exiting” but if I check using the AWS GUI I can see that a new instance has been created successfully. When your script exits, that instance remains so I need to go in and manually terminate it.

    Any thoughts on what might be happening here?

    Thanks, James

  3. …ok found problem. I was running a November 2011 version of EC2 API tools which has a much slimmer output for ec2-describe-instance-status. It does not include, for example, INSTANCESTATUS.

    I upgraded to 1.5.2.4 2011-12-15 and things seem to be working.

    Hope this helps someone.

    1. Hi James, Glad you got it sorted, and thanks for the update.

      As you figured out, the awk command run on the return result from the ec2-run-instances call is a bit sensitive and will fail if the format is different to what it expects. I could probably improve that but in any case, the script depends on the recently added STATUS CHECKS so that it can automate creating instances and connecting via ssh to them (before this check was available Amazon would report the instance as “running” and then the ssh connection would sometimes fail).

  4. Oliver, thanks for doing the work on this, it works well! One question I have though, is about analyzing the data afterward. I’ve been using http://loadosophia.org to generate results, but it doesn’t like the aggregated jtl file your script produces. The individual ones generated during the tests work fine, though. Any ideas? Any other tools I should look at to make pretty pictures?

    Thanks!

    1. I’m working on something right now on the pretty pictures front. Once that is ready I’ll post about it here.

    2. One more thing. I recently changed the aggregation routine in this script so it might work now. Before, I was tinkering with the allthreads column to get an accurate count of thread counts but I’ve found a better way of doing that now.

      If it still doesn’t work you could try:

      a) editing the steps in the script that deal with cat’ing the files together – it
      and/or b) changing the jmeter.properties file.

      It shouldn’t be too hard to make the results work with ap’s loadsophia site.

  5. Seems like the only thing that is missing in order to use loadsophia are the field names.
    Adding them at the CSV’s first line solves the problem and graphs are generated perfectly!

    timeStamp,elapsed,label,responseCode,responseMessage,threadName,dataType,success,bytes,grpThreads,allThreads,Latency,Hostname

  6. Yup, that would do it. I read this on the loadosophia site:

    “If you use CSV file format, make sure you have enabled field names saving in file header. ”

    So, yes, these are required.

    By the way, one reason I exclude them is because when I cat the files together I want to prevent multiple headers. But you could fairly easily change the script to add them in later – you just have to be *certain* that the headers you add match the rows of data in the file. Another reason is I recently added a step to upload the results to a database and this also requires no headers.

  7. Very nice script.
    After dealing with a lot of custom attempts to use standard Jmeter server approach, along with EC2 problems described here, I used this script successfully.

    Just to remarks/questions

    1) Why this permission denied?
    terminating instance(s)…
    INSTANCE i-dce91fa4 running shutting-down

    processing results…./jmeter-ec2.sh: line 730: 0: Permission denied

    ————————————————————————————-
    jmeter-ec2 Automation Script – COMPLETE

    2) I had to adjust install.sh like
    sudo apt-get update
    sudo DEBIAN_FRONTEND=noninteractive apt-get -qqy install default-jre
    in order to work correctly

  8. Point 1. That’s odd. The user running the script needs to have read/write/execute on the directory $LOCAL_HOME/$PROJECT, all I can think is that you are using a user that does not.

    Point 2. Thanks. The Amazon Standard AMIs change over time so sometimes things that worked before start to fail. I’ve added this line to the script though. This should make things more stable.

  9. Thanks a lot for your quick reply. Indeed the local user is a different one than the remote user. So current script considers only the remote user which is of course more important. This is for sure a disadvantage which could be fairly ignored.

    Meanwhile I was able also to run my own JMeter test plan so final target reached!

  10. Hi Oliver,

    thanks for the great script!

    Any idea why I might be getting this error while running it? I tried a few times and it says “lost connection” when copying install.sh to the instances.

    requesting 2 instance(s)…success

    waiting for instance status checks to pass (this can take several minutes)……all hosts ready

    assigning tags…
    TAG instance i-3c693744 ProjectName test
    TAG instance i-3e693746 ProjectName test
    TAG instance i-3c693744 Name jmeter-ec2-test
    TAG instance i-3e693746 Name jmeter-ec2-test
    complete

    copying install.sh to 2 server(s)…lost connection
    lost connection
    .grep: /home/ec2-user/jmeter-ec2/test/1342851775*scpinstall.out: No such file or directory
    .grep: /home/ec2-user/jmeter-ec2/test/1342851775*scpinstall.out: No such file or directory
    .grep: /home/ec2-user/jmeter-ec2/test/1342851775*scpinstall.out: No such file or directory
    .grep: /home/ec2-user/jmeter-ec2/test/1342851775*scpinstall.out: No such file or directory
    .grep: /home/ec2-user/jmeter-ec2/test/1342851775*scpinstall.out: No such file or directory
    .grep: /home/ec2-user/jmeter-ec2/test/1342851775*scpinstall.out: No such file or directory
    .grep: /home/ec2-user/jmeter-ec2/test/1342851775*scpinstall.out: No such file or directory
    .grep: /home/ec2-user/jmeter-ec2/test/1342851775*scpinstall.out: No such file or directory

    Thanks a lot,
    Michael

    1. You’re seeing this because you don’t have access on port 22 to these machines. You need to setup your ip in the security group you give the amazon boxes.

    2. I am getting a very similar error. It permission denied on terminating the

      compaq@compaq-Presario-V3000-RE162PA-ACJ:~/CloudTest/jmeter-ec2$ project=”myproject” count=”1″ ./jmeter-ec2.sh

      ————————————————————————————-
      jmeter-ec2 Automation Script – Running myproject.jmx over 1 AWS Instance(s)
      ————————————————————————————-

      requesting 1 instance(s)…success

      waiting for instance status checks to pass (this can take several minutes)…………all hosts ready

      assigning tags…
      TAG instance i-6b5af917 ProductKey myproject
      TAG instance i-6b5af917 Service -
      TAG instance i-6b5af917 Owner myproject
      TAG instance i-6b5af917 Name jmeter-ec2-myproject
      complete

      copying install.sh to 1 server(s)…lost connection
      .grep: /home/compaq/CloudTest/jmeter-ec2/myproject/1351075473*scpinstall.out: No such file or directory
      .grep: /home/compaq/CloudTest/jmeter-ec2/myproject/1351075473*scpinstall.out: No such file or directory
      .grep: /home/compaq/CloudTest/jmeter-ec2/myproject/1351075473*scpinstall.out: No such file or directory
      .grep: /home/compaq/CloudTest/jmeter-ec2/myproject/1351075473*scpinstall.out: No such file or directory
      .grep: /home/compaq/CloudTest/jmeter-ec2/myproject/1351075473*scpinstall.out: No such file or directory
      .grep: /home/compaq/CloudTest/jmeter-ec2/myproject/1351075473*scpinstall.out: No such file or directory
      .grep: /home/compaq/CloudTest/jmeter-ec2/myproject/1351075473*scpinstall.out: No such file or directory
      .grep: /home/compaq/CloudTest/jmeter-ec2/myproject/1351075473*scpinstall.out: No such file or directory
      .grep: /home/compaq/CloudTest/jmeter-ec2/myproject/1351075473*scpinstall.out: No such file or directory
      .grep: /home/compaq/CloudTest/jmeter-ec2/myproject/1351075473*scpinstall.out: No such file or directory

      I have tried to set the AWS security group associated with script for source IP.
      Any pointers will be of great help.

      1. What happens when you run ‘telnet [serverhostname] 22′? This is a simple to check to see if you do or do not have access.

        If you don’t, you either need to setup the AWS firewall OR you may also be blocked by your corporate (or home) firewall as you leave your network. It is common for outbound traffic to be blocked on port 22, esp. in corporate environments. In this case you’d have to raise a business case with your network admin to get this opened up. I would certainly not advise you to try and tunnel out.

      2. I had the same problem until I realized that the script was trying to copy install.sh to the instances before the SSH daemon was ready to accept connections. This happens a lot when using micro or small instances.

        My solution was to insert a ‘sleep 60s’ at line 274, right after:
        echo -n “copying install.sh …

        Haven’t seen the problem since.

        1. The script waits for the AWS status checks to come back OK and whilst technically the status checks are not a confirmation that the instance is ready to accept ssh connections (I’ve had dealings with Amazon on this very issue and they were to clear to stress this point) in my experience it is 99.8% of the time possible to ssh onto the box once they return.

          However, since this question was posted, I saw the exact same issue myself and in that case it was caused by a hardware failure at AWS so it may have been that…

          In any case, the best option would be to setup a while loop polling the instance on the ssh port. The trick is to use a short timeout to prevent long waits. I’ve actually already got this code elsewhere in the script; it looks like this:


          # Check if remote hosts are up
          for host in ${hosts[@]} ; do
          if [ ! "$(ssh -q -q \
          -o StrictHostKeyChecking=no \
          -o "BatchMode=yes" \
          -o "ConnectTimeout 15" \
          -i $PEM_PATH/$PEM_FILE \
          $USER@$host echo up 2>&1)" == "up" ] ; then
          echo "Host $host is not responding, script exiting..."
          echo
          exit
          fi
          done

          It’s used when existing hardware is passed in but I’ve not used it for new instances because I feel that the status checks are a better method.

          1. Hi Oliver – Nice work on the script. I am experiencing the same issues with lost connection when copying over the install.sh.

            copying install.sh to 1 server(s)…lost connection
            .grep: /Users/workspace/testing-ws/jmeter-ec2/sample-project/1364519238*scpinstall.out: No such file or directory

            I have verified my security group is configured correctly and can ssh into the box by doing a telnet ping on 22

            telnet ec2-23-23-69-1.compute-1.amazonaws.com 22 ⏎
            Trying 23.23.69.1…
            Connected to ec2-23-23-69-1.compute-1.amazonaws.com.
            Escape character is ‘^]’.
            SSH-2.0-OpenSSH_5.9p1 Debian-5ubuntu1
            ^C

            However, the scp of install.sh fails and I can’t get past that point. I have even put a sleep 60s as suggested by Zack, to no avail.

            Any ideas why this might be happening? Is there a work around?

            Thanks in advance.

  11. Hi,

    I have configured the properties file as per the above steps. However I am getting the below error. I am not able to figure out the issue. Please help.

    The directory Users/oliver/jmeter-ec2/foobar does not exist. what could be the possible reason?

  12. Hi ,

    I am interested in running the script from instances in different availability zones. Is this possible to add multiple availability zones here INSTANCE_AVAILABILITYZONE=”eu-west-1b” , “some other zone “.

    Veeraj

    1. It’s possible but it wouldn’t be a small change. You’d have to break up the request to Amazon into multiple calls, for the different availability zones.

      An alternative is to pre-create the hardware (you can do this using this script with the option `setup=TRUE` & `terminate=FALSE`) and then run the test using the `REMOTE_HOSTS` setting. Ie. You could create 3 machines in eu-west-1a, 3 in eu-west-1b and 3 in eu-west-1c and then combine all of their hostnames into the REMOTE_HOSTS property and then run your test over all 9, together. This would be fastest and easiest.

      But if you want to change the source then you’re very welcome to fork the code, make the change and push a merge request. The code where the change would need to start is here:

      # create the instance(s) and capture the instance id(s)
      echo -n “requesting $instance_count instance(s)…”
      attempted_instanceids=(`ec2-run-instances \
      –key $AMAZON_KEYPAIR_NAME \
      -t $INSTANCE_TYPE \
      -g $INSTANCE_SECURITYGROUP \
      -n 1-$instance_count \
      –region $REGION \
      –availability-zone \
      $INSTANCE_AVAILABILITYZONE $AMI_ID \
      | awk ‘/^INSTANCE/ {print $2}’`)

  13. Getting following report, while printing run summary at regular interval on console

    awk: warning: escape sequence `\=’ treated as plain `=’
    JMeter started at Wed Oct 24 18:02:59 IST 2012
    ===================================================================== START OF JMETER-EC2 TEST ================================================================================
    > [updates: every 15 seconds | running total: every 45 seconds]
    >
    > waiting for the test to start…to stop the test while it is running, press CTRL-C
    (standard_in) 2: syntax error
    (standard_in) 2: syntax error
    (standard_in) 2: syntax error
    (standard_in) 2: syntax error
    (standard_in) 2: syntax error
    (standard_in) 1: syntax error
    (standard_in) 1: syntax error
    (standard_in) 1: syntax error
    (standard_in) 1: syntax error
    (standard_in) 1: syntax error
    (standard_in) 1: syntax error
    (standard_in) 1: syntax error
    (standard_in) 1: syntax error
    (standard_in) 1: syntax error
    (standard_in) 1: syntax error
    (standard_in) 1: syntax error
    (standard_in) 1: syntax error
    (standard_in) 1: syntax error
    (standard_in) 1: syntax error
    (standard_in) 1: syntax error
    (standard_in) 1: syntax error
    >
    >

    1. I solved this problem by making sure the version in jmeter-ec2.properties was set to the same version of jmeter that the script was created in.

  14. Simply amazing! Thank you for sharing this!

    For those with the 1351075473*scpinstall.out error – I had the same thing. Oliver is right – ssh issue. I believe I fixed by:
    chmod 600 mypemfile.pem
    and starting a new instance.

    Oliver – I was wondering if a small enhancement might be possible? Could the script read the /data directory and assign a csv to a specific instance? For example, moredata_1.csv would go to EC2 instance 1, moredata_2.csv would got to EC2 instance 2, etc…

    Use Case: I have a long list of usernames and each user should only login once. Using the current script, each instance tries to login the same users.

    If I were better at scripting, I would have been happy to contribute to the github project.

    Thanks for considering…

    1. This was my issue too. For anyone else, this is the SSH error you cannot see:

      Permissions 0664 for ‘/home/ubuntu/.ec2/test.pem’ are too open.
      It is required that your private key files are NOT accessible by others.
      This private key will be ignored.

    2. I suppose what you really want is for the main data file to be read in and divided amongst each instance automatically so that different data goes to each jmeter process and you never get duplication . I’ve had this on a todo list for a while now…

      1. This feature would be amazingly useful. I am in the process of trying to hack this functionality into the script currently.

    3. Under time constraints I came up with a very quick hack to gain the functionality of what you want to do.

      In jmeter-ec2.sh find the following line:
      {sub(“filename\”>’”$filepath”””$endresult”””$filepath”””$endresult”‘${__P(filecounter,0)}<")}}1' \

      then find this line:

      -t $REMOTE_HOME/execute.jmx \

      and add a line underneath it so it looks like this:

      -t $REMOTE_HOME/execute.jmx \
      -Jfilecounter="$counter" \

      That will append the instance number to the end of the filename (strip your file extension for this to work) so in your data directory you should have filename0 filename1 filename2

      Unfortunately this is a quick and dirty hack with a myriad of data going to each instance but it gets the job done.

    4. Ooops my last reply was missing important info here it is again:

      Under time constraints I came up with a very quick hack to gain the functionality of what you want to do.

      In jmeter-ec2.sh find the following line:

      {sub(“filename\”>’”$filepath”””$endresult”‘’”$filepath”””$endresult”””$filepath”””$endresult”‘${__P(filecounter,0)}<")}}1' \

      then find this line:

      -t $REMOTE_HOME/execute.jmx \

      and add a line underneath it so it looks like this:

      -t $REMOTE_HOME/execute.jmx \
      -Jfilecounter="$counter" \

      That will append the instance number to the end of the filename (strip your file extension for this to work) so in your data directory you should have filename0 filename1 filename2. In your test plan just put filename no number and no file extension.

      Unfortunately this is a quick and dirty hack with a myriad of data going to each instance but it gets the job done.

    5. This is the line you want to find for some reason the comments fuck it up
      {sub(“filename\”>’”$filepath”””$endresult”‘<")}}1' \

  15. Hi,

    I am also getting the permission error. However, I have added the IP’s to the security group rules. When I try to connect, it asks for a password to open my .pem file. Is this to do with local and remote users having a different name?

    Thanks

    1. Hi Fred, You should be able to test this out by just ssh’ing using the same pem file manually – if it needs a password then that doesn’t sound like an AWS pem file.

  16. Thanks a lot for the script, it works like a charm!

    There is one report that doesn’t seem to take the consolidated results into account: JmeterPlugin’s “Active Threads over time” chart. I just ran a test distributed on 20 instances, and each instance was having 20-25 active threads at all time, giving a total of 400-500 total active threads across all instances. The “Active Threads” chart is only showing 20-25 active threads, as if it was only picking up threads from one of the instances.

    Looking at the consolidated JTL results file produced by jmeter-ec2, I see multiple occurences of each thread group (e.g. “Thread Group 1-1″). Could this be what is making the graph consider only a single occurence of “thread group 1-1″ and treat it as a single thread?

    If so, do you know if there is a way for each instance to use a different thread group number, so I would see “Thread Group x-y” in the log, where ‘x’ would be the instance number and ‘y’ the thread number on that instance?

  17. Yes, you’re right. The thread count listed is specific to each instance and not aggregated. In fact, this is by design, all data is specific but this is the one piece of data that the Graphs do not automatically aggregate.

    Initially, when I first wrote this script I put in a step to sum up the values here so that when I put the data into the graphs in JMeter it would display correctly. But these days I use a custom graphing layer and in this case it is better for the data to be kept separate. It wouldn’t be hard to add back in the step to sum things again though – you could create a branch with this in – but for the master branch I think it is better to leave it separated.

  18. I was wondering if anyone out there has published an AMI that was configured and setup with the latest code to help us bootstrap our load testing project?

    1. Sherif, if you just want an AMI with jmeter installed then it’s very simple, you only need to ensure java is present (most Ubuntu distribution have this) and then get the jmeter files from the apache site. That’s really all you need.

      A more complete process might involve putting the plugins in place but this obviously depends on your needs.

      Check out the install.sh script to see exactly what steps are run by the script talked about in this post but it’s basically the same as described above.

  19. Hey there!

    Thank you for the great guide!

    Maybe you can help me with a error, that freaks me out…

    > waiting for the test to start…to stop the test while it is running, press CTRL-C
    (standard_in) 2: parse error
    (standard_in) 2: parse error
    (standard_in) 2: parse error
    (standard_in) 2: parse error
    (standard_in) 2: parse error
    (standard_in) 1: parse error
    (standard_in) 1: parse error
    (standard_in) 1: parse error
    (standard_in) 1: parse error
    (standard_in) 1: parse error
    (standard_in) 1: parse error
    (standard_in) 1: parse error
    (standard_in) 1: parse error
    (standard_in) 1: parse error

    Any ideas what could be the problem?

  20. Thanks so much for this!

    My only issue is with the resulting .jtl files. loadosophia just plain gives me an error about jtl files, and I’ve not found anything else that will let me view the results. Once I have that figured out… I’m taken care of!

    Again, thanks! This is so very handy.

      1. I literally just got it to work. However, I had to add the headers in and save the file as a .csv before it would work. In any case, I’m managing to get “pretty pictures” for the people that prefer to not see the data in the form of numbers.

        I appreciate the follow up!

  21. Hi
    Connecting to remote host returns

    “Host 10.50.XX.YY is not responding, …” ??

    If I remove address from REMOTE_HOSTS than it creates new instance without any issues.
    Groups, regions, keys, zones looks to be set properly.

    Any ideas?
    Thanks

    1. Hi Piotr, that is actually by design. REMOTE_HOSTS is designed to allow the script to work with instances / hosts that are already pre-defined and configured. If you enter a value here then the script will not contact AWS and try to use this hardware instead; it obviously needs to be able to connect to that box to do this. If you leave it blank the script will try to create new hardware.

      1. Thanks

        One more question… Is it possible to use VPC security groups instead of EC2 groups?
        When I enter VPC sec group I get below message
        requesting 1 instance(s)…Client.InvalidParameterCombination: VPC security groups may not be used for a non-VPC launch
        How can I do this?

        1. VPC is a different ball game. Although the basic EC2 calls are the same there are differences in params and the script is not setup to handle them. Pull requests welcome.

          1. Thanks
            Will try to work on it as it would be very helpful

            on a side note… looks like connection issue that groovybayo had is something more common as I’m not able to create new instance with this script and all the time receive “copying install.sh to 1 server(s)…lost connection” error

          2. looks like it was quite easy to customize this script to create instances within vpc (added -s $SUBNET_ID to instance creation and entered vpc-group-id instead of name into INSTANCE_SECURITYGROUP) and it worked

            as for lost connection issue it looks like ec2-tools in version 1.6.7.1 returns privateipaddress a bit different than previous versions so I changed line 165 to

            hosts=(`ec2-describe-instances –region $REGION ${attempted_instanceids[@]} | awk ‘/PRIVATEIPADDRESS/ {print $2}’`)

            and looks like it connects now without any issues

          3. Thanks for those changes, would you like to put them in a Pull Request on GitHub? I can add them in anyway but sometimes people prefer sending pulls.

  22. Hi Oliver, this is a great tool! I recently modified it to use spot instances. I changed the script up a lot (and I didn’t need to adjust the threads, so I didn’t use that functionality). As a result it’s a bit changed. A pull request may happen in the future if I ever clean it up, but feel free to take what is useful from it. I forked it here: a href=”https://github.com/curiousjgeorge/jmeter-ec2/tree/jogeorge/spotinstances”>https://github.com/curiousjgeorge/jmeter-ec2/tree/jogeorge/spotinstances. I already submitted a pull request for some minor changes.

  23. I’ve been using this tool, and thanking you each and every time I do for a while now.

    Another parsing question. Do any of you have any experience working with very large results files? loadosophia is great when the files are a gig (or so) or less, as they will then compress down to <100mb. However, for example… I just completed a full 72 hour soak on a system, and the results file is a lovely 6gb. Now, I can split this and process each file individually, but that creates issues with presentation as the axis values can differ from test to test.

    Any suggestions? I've looked into matplotlib, but the installation process has not only been problematic, it's caused some lovely issues with my python installation as well.

    Thanks again for the tool!

  24. Hi Oliver,

    This EC2 setup works excellent.
    However I have one issue.
    I have a Java code to be tested using this setup. I am using the jar file of the code by importing the same in Jmeter, (the jar file has to be placed in lib/ext folder in Jmeter).

    How to make the above jmeter script in EC2? How to transfer the jar file to EC2?
    Could you please reply?

    Thanks and regards,
    Jack

Comments are closed.