Use the DMN Tester in Continuous Integration CI

5 min readFeb 22, 2021

A few weeks ago I described, how you can test your DMN Tables in no time. See Testing (Camunda)-DMN Tables automatically.

So the next thing we would like to have is that we can easily find out if the DMN-Tables were changed by the Business — and if this could have introduced a bug.

This blog explains, how you can achieve this with the DMN Tester.

Test Case Creation

In the last post I showed, how you can create your tests. This added the expected Results to the DMN Table Configuration.

Here is an example:

Which creates the following Test Cases in the configuration:

data {
    inputs=[
        {
            ...
        },
        {
            inputs {
                guestCount=9
                season=Summer
            }
            results=[
                {
                    outputs {
                        desiredDish="Light Salad and a nice Steak"
                    }
                    rowIndex=6
                }
            ]
        },
        {
            ...
        }
    ]
}

A Test Case consists of:

Test Inputs: guestCount=9, season=Summer
Test Results for each matched Row: rowIndex=6, all Outputs — desiredDish=”Light Salad and a nice Steak”

When running the DMN Tester the next time, we can check if for the inputs guestCount=9, season=Summer the row index is still 6 and the output desiredDish stays ”Light Salad and a nice Steak”.

That works fine — but that would mean that I have to start and run the DMN Tester whenever there is a change in one of the DMN Tables.

As you might guess that’s too much for me😀.

Regression Tests

How can we now use our DMN Tester to check if there were changes that introduced possible bugs?

Run the DMN Tester?

Of course we can run the DMN Tester against the DMN Configurations and its Test Cases.

But how can we interpret the results and report them?

What every CI-Tool understands are Test-Reports. To have them you need Unit tests — so let’s generate them!

Prepare your Tests

What tests should fail? To start easy I decided to pass a Test two things must be true:

The status of a Test Row must be INFO. The test fails with the statuses WARN or ERROR.
A Test Case must match its row index and all its outputs.

With this in mind you have to do again two things:

Resolve all your problems the DMN Tester shows you. Check all your Warnings and Errors with the Business. Usually they can be fixed with:
- Fixing Errors.
- Removing Test Inputs that must never happen.
- Changing the Hit Policy. If possible try always to use UNIQUE. If there are too many results — this is a sign to group some test inputs in a separate DMN table (refactoring through decomposing).
Mark all rows as Test Cases after validating them with the Business. As shown above.

Running the Tests

I created a Docker Image that does generate the tests and runs them:

Generate Unit Tests

One Test Class for each DMN Configuration:

class BeveragesSuite extends AnyFunSuite

We use here Scala Test as it lets you describe each test with a readable text.

Every Test is defined by its inputs. So we generate for each one a test method like:

test ("Test Inputs: desiredDish -> Roastbeef | guestsWithChildren -> true")

If the evaluation with dmn-scala is successful we just let the Test succeed and add the according information:

succeed
/*
DmnEvalRowResult
- Status: INFO
- DMN Table: beverages
- Test Inputs: 
   - desiredDish: Roastbeef
   - guestsWithChildren: false
- Matched Rule: row-506282952-9 (3)
   - Inputs:
      - desiredDish: "Roastbeef"
      - guestsWithChildren: 
   - Outputs:
      - beverages: TestSuccess(Bordeaux)
- Matched Rule: row-506282952-12 (6)
   - Inputs:
      - desiredDish: 
      - guestsWithChildren: 
   - Outputs:
      - beverages: TestSuccess(Water)
- Error: -

*/

(succeed is an Assertion of ScalaTest.)

In case of a failure we let the Test fail with the according information:

fail("""Dmn Row of 'dish' failed with Status ERROR:
        DmnEvalRowResult
        - Status: ERROR
        - DMN Table: dish
        - Test Inputs: 
           - season: Fall
           - guestCount: 10
        - Matched Rule: row-506282952-5 (5)
           - Inputs:
              - season: "Fall","Winter","Spring"
              - guestCount: > 8
           - Outputs:
              - desiredDish: TestFailure(Stew,The output 'desiredDish' did not succeed: 
                - expected: 'Stew2'
                  actual : 'Stew')
                - Error: -
        """)

The Docker Image

Ok but how do we know that something is wrong? Well we have to run the Tests and display the Test Reports.

With the following command you can run everything:

docker run \
  --name camunda-dmn-tester-ci \
   --rm \
   -it \
   -e TESTER_CONFIG_PATHS="/dmnConfigs" \
   -v $(pwd)/dmnConfigs:/opt/workspace/dmnConfigs \
   -v $(pwd)/dmns:/opt/workspace/dmns \
   -v $(pwd)/target:/opt/workspace/target \
   -v $HOME/.ivy2:/root/.ivy2 \
   pame/camunda-dmn-tester-ci

Ok, line by line:

--name camunda-dmn-tester-ci Gives a readable name for the container:
--rm removes the container after finishing the tests.
-it is optional — the interactive mode gives me nice colors on my mac.
-e TESTER_CONFIG_PATH="/dmnConfigs"a comma-separated list where you have the DMN configurations.
-v $(pwd)/dmnConfigs:/opt/workspace/dmnConfigsadds the DMN configurations to the container.
-v $(pwd)/dmns:/opt/workspace/dmnsadds the DMNs to the container.
-v $(pwd)/target:/opt/workspace/targetprovides the Test Reports so you can access them outside of the container.
-v $HOME/.ivy2:/root/.ivy2adds the local cache to the container to speed up things a bit.
pame/camunda-dmn-tester-ciis the image you need.