Dec 202011

The deepest impact on my side was studying the following books during the last year: Continuous Integration by Paul M. Duvall, Steve Matyas and Andrew Glover and Continuous Delivery by Jez Humble and David Farley. These books and presentations during the OOP 2011 in Munique pushed me into the direction that I am trying to describe below.

Before I go into details how we use Cucumber, I want to describe a little bit our environment: In our company we are developing a medical application that can run as a standalone application or in a client / server setup. Each configuration consists for the user of three UI processes and about fifteen background processes or services. The data persistence is realized with a PostgreSQL database, which is handled through one of the service processes. It is a radiological reviewing workstation that is highly optimized on high volume throughput. The image data for each patient can easily be about 2.5 GB and the technical challenge is to display any of next possible radiological images, which can have easily 26-50MB each in less than a second, because our customers are used to diagnose about 60-120 cases per hour. Depending of the current workflow step, the user has to work with a different UI process.

The application has about 2,000,000 lines of C++ code, currently about 10.000 single requirements and most of the functional tests are manual paper based or steered by our test management system SpiraTest. So it is obvious that we have plenty of work to do. Beside the fact that we have way too much requirements or be more precise that the granularity of the requirements is too fine, certain parts of the code are hardly testable in an automated way. Our transition from the V-model to SCRUM two years ago brought up dramatically the fact that test automation is one important step for us to stay productive and ensure quality.

We had so far only UnitTests, based on the GoogleTest framework, and UI tests, implemented with TestComplete. Because of the known problems with UI tests we wanted to take into account tests on business level or acceptance level as well. According to the literature and our own researches only two widely known acceptance test frameworks exists: Fitnesse and Cucumber. Fitnesse would have the advantage of a direct C++ support, but it was more important for us that the description of the system behavior could be defined in a ubiquitous language and that there is a way to combine the definition of requirements and tests within one artifact and so we chose Cucumber.

During the following I assume that the reader is familiar with the Cucumber terms like Scenario, Feature, Step Definition etc. If you don’t feel throng enough in this area I recommend having a look into the main web-page or one of the existing books.

An of the shelve approach with Cucumber was not possible, because our application is written in Qt and C++. At the beginning we tried to use the C++ binding Cukebins, which was still in a pre-alpha state at that time. This would not have stopped us of using and developing it further together with the maintainer. But then an internal discussion started, that we may want to change our main development language from C++ to C# and so it would not been wise to write all Step Definitions in C++ and rewrite them later in C#. As well we wanted to give “non C++ colleagues” the possibility to write Features and/or modify existing Step Definitions, so we came to the conclusion that it would be the best way in staying with the Step Definitions in the Cucumber natural language Ruby.

The next step was that we had to think about, how to connect our C++ environment and the Ruby world. Beside the circumstance, that we have multiple UI processes that needs to be addressed, that we even need the possibility to address different clients within a single scenario and Cucumber itself is currently only capable to address a single target, it was clear that we need a central process instance that receives all instructions from the Step Definitions and forwards them to the individual UI processes or even the background services. We called this instance CukeCenter.

The communication layer between all our application processes and services is handled via a proprietary socket based inter-process communication, we call it simple IPC. So this layer was intended to be used as communication layer between the CukeCenter and our processes.

Then we had the idea to create this CukeCenter as a native Ruby binding. This was no good idea, because this had required that we compile Ruby with Visual Studio 2008 and the problems had started to raise. With a recipe from the web we were even able to compile Ruby with this compiler but even creating a simple module that should be accessible from Ruby via SWIG caused major trouble. And we knew that we still had to compile Cucumber and all its dependencies with the Microsoft compiler. So moved finally this attempt to the lessons learned section and looked for a different approach.

What would be an appropriate interface between Ruby and our CukeCenter? It should be easy to handle, open to extensions and bindings on C++ and Ruby should be available. We chose XML-RPC. Implementing a lightweight interface was pretty easy, because we wanted to transmit only POD types and the rest should happen generically. We never had in mind to define for each call into our application an individual RPC call. With Ruby’s method_missing feature we could avoid defining all test functions explicitly. We created inside the C++ application an interface, where each test function/injection point gets registered with a command registrar. For reasons of simplicity we use on Ruby side the same names as on application side.

This was perhaps a little bit abstract. Lets get a bit more concrete, lets see a bit of code:
Inside a step definition we call something:

Given /the patient with its name (.*)/ do |name|
  patientlist.deletePatientByName name

Here is patientlist the name of the UI application which implements a patient browser. Inside the application we have to code something like:

// defining the test function
void deletePatientByName(const Source& source, Sink& sink); 

// registering the function and its name with the registrar 
CommandRegistrar(deletePatientByName, "deletePatientByName");

// implementation of the test function which is normally called by the business layer 

void deletePatientByName(const Source& source, Sink& sink) 
  std::string name;
  source >> name;

I omitted here the code inside Ruby which encapsulates the command name and the parameter into XML-RPC and the code inside the CukeCenter which is responsible for extracting the function name and the parameter and forwarding it to the application via IPC.

Now the basics were in place and we were able to start writing Feature files and the corresponding Step Definitions.

The next issue which appeared that each Cucumber scenario should be independent from each other. So we needed an elegant way to bring the system back into a defined state after each step was executed. For instance, a standard set of patient data is all the time on the system and should be used for as much steps as possible. But we have scenarios in which new patient data must be sent to the system, then gets modified during the step and then? Well one way could be, that we restore the database after the execution of each step. But this would take much too long. Holding the transaction open and performing a roll-back could be a way for others, but the architecture of our application does not allow this.

So we added a cleaning mechanism that gets executed after finishing each step. Our first attempt was to use hooks for this task, but this resulted in lot’s of noise inside the Feature files. So inside the steps the code looks like

@patient_container.sendPatientByName "Jane Doe"
@cleaner.push_undo_action { patientlist.deletePatientByName "Jane Doe"} 
patientlist.openPatientForReviewing "Jane Doe" 
@cleaner.push_undo_action { patientlist.closeReviewingModule }

So in the first line a new test patient is send to the system. In the next line we pass the cleaner instance a Ruby block that should be executed after the scenario is finished. In the 3rd line an other command is send to the system which is part of the scenario. And in the last line a block is passed to the cleaner instance for undoing the previous action. So when the complete scenario is finished, all stored block in the cleaner are executed in reverse order.

Our Cucumber tests are triggered automatically after each (clean/masterbuild) of our application by our Jenkins build system and get executed on a virtual machine which is is set back to a defined snapshot before each run.

So I hope this is useful for someone and I want to apologize for my English, as it is not my mother language.

  6 Responses to “Acceptance Test with Cucumber of a C++ Application”

  1. I was recommended this internet internet site by my cousin.

    • Dec08 I am very happy to read this. This is the kind of dleiats that needs to be given and not the accidental misinformation that is at the other blogs. Appreciate your sharing this best doc.

  2. I dugg some of you post as I thought they were quite beneficial invaluable 959523

  3. Why you stoped doing this blog? I found your four post very interesting as I’m in the same process as you were a yeear before, evaluating to change to SCRUM and what technologies we need .

    • Thanks for the feedback.
      I did not stop this blog. I am a little bit buzzy at the moment and so I do not have the time at the moment. I have already a new small project running and after success I intent to publish my experience here.
      Regards, Felix

  4. Thankyou for helping out, great information.

Sorry, the comment form is closed at this time.