How to run Jmeter tests in non-GUI mode, automated with a shell script

Apache Jmeter is a great versatile tool for performance testing of web servers, DBMS servers etc. It's a pure Java application where you can use even your own Java code to customize and randomize your tests. My intention of this post is not to tell you about all of those features, but one such cool feature of Jmeter which is running Jmeter tests in non-GUI mode.

If you have worked with Jmeter you surely should have experienced that sometimes it get stuck or crashes while you are doing high concurrency (> 300) load testing. One solution for this is to create your test using the GUI and save it as a '.jmx' file and later run that in non-GUI mode and save the test results to a '.jtl' file. In that way you will be able to carry out your load test without crashing or maybe with more concurrency.

OKEY..... This is how to do it.

1. First prepare your test plan with Jmeter and save it, let's say the file name is 'mytest.jmx'. (It gets saved as a xml configuration file with the .jmx extention).

2. Then go to Apache JMETER_HOME/bin folder using command shell. And then run Jmeter with the following parameters, to execute the test.

./jmeter -n -t \absolute_path\mytest.jmx -l \absolute_path\mytest_results.jtl

3. While the test is running the results will be saved in the 'mytest_results.jtl'. When your test is complete you can open this jtl file in Jmeter and summarize the results or create graphs using it.
To do that open Jmeter and add a Summary Report to the test plan (Add --> Listener --> Summary Report). And then go to the Summary Report UI and 'Browse' and open your mytest_results.jtl file.


OK.....next comes the real headache :-). Let's say you have to repeat the above test several times with small changes to the test. I actually had a requirement at my work to repeat the same Jmeter test with different concurrency levels and number of loops.
The hard way to do this is create separate test scripts for different concurrency levels using the GUI and run them separately as above.
A less hard way to do this is open the test file (.jmx) using a text editor and change the values every time you repeat the test, in the ThreadGroup element in the test configuration(as below), and run the same file.

<ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="Thread Group" enabled="true">
        <stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
        <elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller" enabled="true">
          <boolProp name="LoopController.continue_forever">false</boolProp>
          <stringProp name="LoopController.loops">1000</stringProp>
        </elementProp>
        <stringProp name="ThreadGroup.num_threads">300</stringProp>
        <stringProp name="ThreadGroup.ramp_time">1</stringProp>
        <longProp name="ThreadGroup.start_time">1339414812000</longProp>
        <longProp name="ThreadGroup.end_time">1339414812000</longProp>
        <boolProp name="ThreadGroup.scheduler">false</boolProp>
        <stringProp name="ThreadGroup.duration"></stringProp>
        <stringProp name="ThreadGroup.delay"></stringProp>
      </ThreadGroup>

OK.... But why not automate the above 'less hard way' and make your life easy..
If you are a shell scripting master this is a "few lines" work, but since I'm not, this is how I did it.

#!/bin/bash

i=0

conc[0]=50
conc[1]=100
conc[2]=250
conc[3]=500
conc[4]=750
conc[5]=1000
conc[6]=1250

loop[0]=5000
loop[1]=2500
loop[2]=1000
loop[3]=500
loop[4]=334
loop[5]=250
loop[6]=200

cd ./jmx
while test $i != 7
do
sed 's/_loop_/'${loop[$i]}'/g' mytest.jmx > ./jmtest.jmx
sed 's/_con_/'${conc[$i]}'/g' jmtest.jmx > jmtest_${conc[$i]}.jmx
rm jmtest.jmx
cd ../apache-jmeter-2.6/bin/
./jmeter -n -t ../../jmx/jmtest_${conc[$i]}.jmx -l ../../jtl/results_${conc[$i]}.jtl
i=`expr $i + 1`
cd ../../jmx
done

For newbies like me, I will explain the script briefly.

For this script to work first you should do a small change to your Jmeter test configuration. You have to give a variable string for num_threads and loop instead of actual values, as below.

<ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="Thread Group" enabled="true">
        <stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
        <elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller" enabled="true">
          <boolProp name="LoopController.continue_forever">false</boolProp>
          <stringProp name="LoopController.loops">_loop_</stringProp>
        </elementProp>
        <stringProp name="ThreadGroup.num_threads">_con_</stringProp>
        <stringProp name="ThreadGroup.ramp_time">1</stringProp>
        <longProp name="ThreadGroup.start_time">1339414812000</longProp>
        <longProp name="ThreadGroup.end_time">1339414812000</longProp>
        <boolProp name="ThreadGroup.scheduler">false</boolProp>
        <stringProp name="ThreadGroup.duration"></stringProp>
        <stringProp name="ThreadGroup.delay"></stringProp>
      </ThreadGroup>

In the shell script, first I put the concurrency values and loop values in to 2 different arrays.
And then using a while loop I edit the concurrency and loop values in Jmeter test script for each time I have to repeat the test. I have used sed command for that. It searches for the _loop_ and _con_ strings in the Jmeter script and will replace it with the corresponding values in above two arrays.
Then inside the loop, changed Jmeter script is executed (for the number of times the while loop is run) in non GUI mode and results are saved in different jtl files.