Thursday 4 July 2013

SoapUI: Data Driven Testing (without going pro)


One of the neat things which SoapUI Pro offers is called data driven testing http://www.soapui.org/Data-Driven-Testing/functional-tests.html. The idea is that you can run a test several times and pull data out of a data source (file, database etc) to use in your tests. Sadly not all of us have SoapUI Pro but it's entirely possible to do something similar with SoapUI free (as Theo Marinos demonstrates in his excellent blog on the subject).

The above blog chooses a random value from a file each time you run a test in order to test a service. This is great for load testing, or if all you want to prove is that a service works correctly with one of a given set of values, but what if you want to test that for a given input (or inputs) to the service returns a given output (or outputs). Well that's also quite easy to do to. I've put together a simple example which reads every value in a csv file, calls a web service using the input value and ensures the response contains the expected response from the same line of the file.


Creating a service to test

As I didn't have a real service to test, the first thing I did was to create a SoapUI mock test service based on a very simple WSDL. In the mock service I put some groovy to generate a response based on the input message
def holder = new com.eviware.soapui.support.XmlHolder( mockRequest.requestContent )
def name = holder["firstName"];
if (name.substring(0,2)=="Mr")
    requestContext.greeting="Hello "+name;
else
    requestContext.greeting="Howdy "+name;
This generates a "greeting" variable. If the "firstName" element starts with Mr then it responds "Hello <name>", otherwise it replies "Howdy <name>" - simple. Then in the mock service I put this greeting variable to use:
<soapenv:Envelope xmlns:xsi="http:www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:urn="urn:examples:helloservice">
<soapenv:Header/>
    <soapenv:Body>
       <urn:sayHelloResponse soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
          <greeting xsi:type="xsd:string">${greeting}</greeting>
       </urn:sayHelloResponse>
   </soapenv:Body>
</soapenv:Envelope>

Creating some test data

OK so now we have something to test lets generate some test data. A simple CSV:
req_firstName, resp_greeting
Bob, Howdy Bob
Fred, Howdy Fred
Mr Jones, Hello Mr Jones
Jill, Howdy Jill

The first line is the name of the properties these will be stored in, the other lines are the values of those variables. As will become apparent in a second the groovy is completely generic and doesn't need changing as it can read any number of values into variables based on the first line.

So now we have a service to test and some data we need to setup a test...

Configuring the test

I setup a project with a very simple Test Case. It only has 4 test steps:
  1. A groovy script to load the given values from the file
  2. A soap test step to call the mock web service
  3. A conditional goto step to return to the start
  4. A step called "End" (the name here is important as we'll see below but the step can be of any type - in my case I used a delay test step)
The other thing which is needed is some properties in the test case:

  • TEST_FILE: the name (with path) of the csv file created above
  • One property for each column in the CSV (with the same name as the header row i,e, req_firstName and resp_greeting)
The only property which needs to be named as above is the TEST_FILE as all the others are user defined.

1. Groovy

 tc = testRunner.testCase;

// First run through: initiation
if (context.fileReader == null){
    log.info ("##Starting Data run... File="+tc.getPropertyValue("TEST_FILE").toString());
    context.fileReader = new BufferedReader(new FileReader(tc.getPropertyValue("TEST_FILE")));
    line = (String)context.fileReader.readLine();

    // Get variable names
    context.variableNames = line.split(",");
}

// Process each line
context.valueLine = context.fileReader.readLine();

// Data to process: load values
if (context.valueLine!=null){
    // Assign the parts of the line to the properties
    for (int i=0;i<context.variableNames.length;i++){
        variable = context.variableNames[i].trim();
        value= context.valueLine.split(",")[i].trim();
        log.info "Assigning: $variable=$value";
        tc.setPropertyValue(variable, value);
    }
}
// No data to process: tidy up
else{
    context.fileReader.close();
    context.fileReader=null;
    context.variableNames=null;
    testRunner.gotoStepByName("End");
}
The above can be pasted into a groovy step unchanged for any number of properties read from any csv data file.

2. SOAP Test

My soap test step request contains a reference to the request property:
<soapenv:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:urn="urn:examples:helloservice">
   <soapenv:Header/>
   <soapenv:Body>
      <urn:sayHello soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
         <firstName xsi:type="xsd:string">${#TestCase#req_firstname}</firstName>
      </urn:sayHello>
   </soapenv:Body>
</soapenv:Envelope>
The assertion in the test script (of type xpath) checks the response has the right value in it (again based on the csv value.
Declare:
declare namespace urn='urn:examples:helloservice';
declare namespace soapenv='http://schemas.xmlsoap.org/soap/envelope/';
/soapenv:Envelope/soapenv:Body/urn:sayHelloResponse/greeting
Expected Result:
${#TestCase#resp_greeting}

3. Conditional goto 

The goto step should always go back to the groovy step. When all the rows in the file are completed, the groovy code above will jump to "End" and thus stop the loop - this is why the end test step has to be called "End". In order to make the conditional goto unconditional we just need to make the condition always true - by use of the xpath true() function.

4. End

Finally the End step is just needed so they groovy script can jump out of the loop. It doesn't matter what this step does so long as there is a step called "End" after the goto step.

Running the test

So as the above shows there were 14 test steps (the first 3 steps executed four times for the four people in the csv, then the first and last steps executed to end the cycle). The request and response variables contain the values of the last line in the file as I didn't clear them. From here I could add more variables to the file or add a different set of tests - each test case is self contained so each one can have a different TEST_FILE property and different variables. Simple.

If the service didn't give the expected response to one of the values then the soap test step would fail because of the assertion.

Licensing

All the above is open source under the GPLv3 license without any guarantees or promises, that said it's not very complicated and seems to work well enough - hope it's useful.

11 comments:

  1. This comment has been removed by the author.

    ReplyDelete
  2. It is very useful! Thanks

    ReplyDelete
  3. Wonderful post! I really appreciate your post. It just solved my soapui challenges of data-driven web service test.

    I appreciate it alot. Thanks.

    ReplyDelete
  4. The below link may help you

    http://lgsofttest.blogspot.no/2014/06/datadriven-testing-in-soapui-open.html

    ReplyDelete
  5. Thank you so much!! How should I write the responses to a log file. Let us say that I loop through a list of social security numbers and creating accounts for them, the service response returns the account number. How can I take my transferred property value (ssn) write that to a file together with the response from the service?

    ReplyDelete
    Replies
    1. Hi Martin, nice to hear it was useful. That's a good question and one I may put into a future blog, for a quick answer...

      First you need to open a second file (for write) where you open the data file for reading. Simply add a couple of lines to the "Start Data Test" step:
      context.fileWriter = new BufferedWriter(newFileWriter(tc.getPropertyValue("LOG_FILE")));

      you'll need to close it in the "tidy up" place, where you close the other file:
      context.fileWriter.close();
      context.fileWriter=null;
      The LOG_FILE property needs to be fully qualified just like the TEST_FILE property is.

      Once that's done, just add a groovy step after your SOAP call (before the goto step) with the following code:

      def groovyUtils = new com.eviware.soapui.support.GroovyUtils( context )
      def tc = testRunner.testCase;
      def responseHolder = groovyUtils.getXmlHolder( tc.getTestStepByName("Test Request").getPropertyValue("response") );

      //to output a single value...
      def value = responseHolder["//greeting"];
      context.fileWriter.writeLine(tc.getPropertyValue("req_firstname")+","+value);

      //OR to output the whole thing...
      context.fileWriter.writeLine(tc.getPropertyValue("req_firstname"));
      context.fileWriter.writeLine(responseHolder.getXml());

      That should write out a csv with the input value and a value from the response. Or the input value then the whole response XML.

      Hope that helps,

      J

      Delete
    2. This comment has been removed by the author.

      Delete
    3. --Is this how you mean that the first Groovy script should look like? In terms o lines of code to add to the first Groovy script?


      tc = testRunner.testCase;

      // First run through: initiation
      if (context.fileReader == null){
      log.info ("##Starting Data run... File="+tc.getPropertyValue("TEST_FILE").toString());
      context.fileReader = new BufferedReader(new FileReader(tc.getPropertyValue("TEST_FILE")));
      line = (String)context.fileReader.readLine();
      context.fileWriter = new BufferedWriter(new FileWriter(tc.getPropertyValue("LOG_FILE")));

      // Get variable names
      context.variableNames = line.split(",");
      }

      // Process each line
      context.valueLine = context.fileReader.readLine();

      // Data to process: load values
      if (context.valueLine!=null){
      // Assign the parts of the line to the properties
      for (int i=0;i<context.variableNames.length;i++){
      variable = context.variableNames[i].trim();
      value= context.valueLine.split(",")[i].trim();
      log.info "Assigning: $variable=$value";
      tc.setPropertyValue(variable, value);
      }
      }
      // No data to process: tidy up
      else{
      context.fileReader.close();
      context.fileReader=null;
      context.variableNames=null;
      context.fileWriter.close();
      context.fileWriter=null;
      testRunner.gotoStepByName("End");
      }

      --Is this correct so far?

      --My Test Step is called "New Application" and I would like to pickup on data from the response, so the beginning of the 2nd Groovy script looks like this:

      def groovyUtils = new com.eviware.soapui.support.GroovyUtils( context )
      def tc = testRunner.testCase;
      def responseHolder = groovyUtils.getXmlHolder( tc.getTestStepByName("New Application").getPropertyValue("Response") );


      --In order to reach to the data that is located in the above response how should I define the rest of the 2nd groovy? The field it is locatedin a hiearchy of different XML elements and namespaces. How should the rest look like, I cannot figure it out?

      I.e. this part:
      //to output a single value...
      def value = responseHolder["//greeting"];
      context.fileWriter.writeLine(tc.getPropertyValue("req_firstname")+","+value);

      --Do I need to create a property value for data that I would like to return to the output file before?

      Delete
  6. Thank you very much for this tutorial. This example worked fine for me. However, I am doing a proof of concept for testing a Microsoft soap web service which has lots of namespace declarations in the response and having trouble in writing xpath expressions. Could you please throw some light as to where to go for understanding xpath expressions properly ? Are there any tools that can generate such expressions without needing to write them ?

    ReplyDelete
    Replies
    1. https://msdn.microsoft.com/en-us/library/ms256086%28v=vs.110%29.aspx is quite a good cheat sheet for Xpaths. I don't tend to use tools

      Delete
  7. this groovy script fails when csv files contains the blank column value...kindly help me

    ReplyDelete