Use the DMN Tester in Continuous Integration CI

A few weeks ago I described, how you can test your DMN Tables in no time. See Testing (Camunda)-DMN Tables automatically.

So the next thing we would like to have is that we can easily find out if the DMN-Tables were changed by the Business — and if this could have introduced a bug.

This blog explains, how you can achieve this with the DMN Tester.

Test Case Creation

Here is an example:

Which creates the following Test Cases in the configuration:

data {
inputs=[
{
...
},
{
inputs {
guestCount=9
season=Summer
}
results=[
{
outputs {
desiredDish="Light Salad and a nice Steak"
}
rowIndex=6
}
]
},
{
...
}
]
}

A Test Case consists of:

  • Test Inputs: guestCount=9, season=Summer
  • Test Results for each matched Row: rowIndex=6, all Outputs — desiredDish=”Light Salad and a nice Steak”

When running the DMN Tester the next time, we can check if for the inputs guestCount=9, season=Summer the row index is still 6 and the output desiredDish stays ”Light Salad and a nice Steak”.

That works fine — but that would mean that I have to start and run the DMN Tester whenever there is a change in one of the DMN Tables.

As you might guess that’s too much for me😀.

Regression Tests

Run the DMN Tester?

But how can we interpret the results and report them?

What every CI-Tool understands are Test-Reports. To have them you need Unit tests — so let’s generate them!

Prepare your Tests

  1. The status of a Test Row must be INFO. The test fails with the statuses WARN or ERROR.
  2. A Test Case must match its row index and all its outputs.

With this in mind you have to do again two things:

  1. Resolve all your problems the DMN Tester shows you. Check all your Warnings and Errors with the Business. Usually they can be fixed with:
    - Fixing Errors.
    - Removing Test Inputs that must never happen.
    - Changing the Hit Policy. If possible try always to use UNIQUE. If there are too many results — this is a sign to group some test inputs in a separate DMN table (refactoring through decomposing).
  2. Mark all rows as Test Cases after validating them with the Business. As shown above.

Running the Tests

Generate Unit Tests

class BeveragesSuite extends AnyFunSuite

We use here Scala Test as it lets you describe each test with a readable text.

Every Test is defined by its inputs. So we generate for each one a test method like:

test ("Test Inputs: desiredDish -> Roastbeef | guestsWithChildren -> true")

If the evaluation with dmn-scala is successful we just let the Test succeed and add the according information:

succeed
/*
DmnEvalRowResult
- Status: INFO
- DMN Table: beverages
- Test Inputs:
- desiredDish: Roastbeef
- guestsWithChildren: false
- Matched Rule: row-506282952-9 (3)
- Inputs:
- desiredDish: "Roastbeef"
- guestsWithChildren:
- Outputs:
- beverages: TestSuccess(Bordeaux)
- Matched Rule: row-506282952-12 (6)
- Inputs:
- desiredDish:
- guestsWithChildren:
- Outputs:
- beverages: TestSuccess(Water)
- Error: -

*/

(succeed is an Assertion of ScalaTest.)

In case of a failure we let the Test fail with the according information:

fail("""Dmn Row of 'dish' failed with Status ERROR:
DmnEvalRowResult
- Status: ERROR
- DMN Table: dish
- Test Inputs:
- season: Fall
- guestCount: 10
- Matched Rule: row-506282952-5 (5)
- Inputs:
- season: "Fall","Winter","Spring"
- guestCount: > 8
- Outputs:
- desiredDish: TestFailure(Stew,The output 'desiredDish' did not succeed:
- expected: 'Stew2'
actual : 'Stew')
- Error: -
""")

The Docker Image

With the following command you can run everything:

docker run \
--name camunda-dmn-tester-ci \
--rm \
-it \
-e TESTER_CONFIG_PATHS="/dmnConfigs" \
-v $(pwd)/dmnConfigs:/opt/workspace/dmnConfigs \
-v $(pwd)/dmns:/opt/workspace/dmns \
-v $(pwd)/target:/opt/workspace/target \
-v $HOME/.ivy2:/root/.ivy2 \
pame/camunda-dmn-tester-ci

Ok, line by line:

  • --name camunda-dmn-tester-ci Gives a readable name for the container:
  • --rm removes the container after finishing the tests.
  • -it is optional — the interactive mode gives me nice colors on my mac.
  • -e TESTER_CONFIG_PATH="/dmnConfigs"a comma-separated list where you have the DMN configurations.
  • -v $(pwd)/dmnConfigs:/opt/workspace/dmnConfigsadds the DMN configurations to the container.
  • -v $(pwd)/dmns:/opt/workspace/dmnsadds the DMNs to the container.
  • -v $(pwd)/target:/opt/workspace/targetprovides the Test Reports so you can access them outside of the container.
  • -v $HOME/.ivy2:/root/.ivy2adds the local cache to the container to speed up things a bit.
  • pame/camunda-dmn-tester-ciis the image you need.

This will:

  • create a Docker Container.
  • generate the Unit Tests.
  • run the Unit Tests with the SBT-Build Tool.
  • create the Test Reports in the Unit Test HTML-format.
  • and then remove the container when finished.

The next screenshot displays the test command of SBT and the information where to find the Test Reports:

And as mentioned in the console, you find the Test Reports here:

target/test-reports/index.html

The Report looks like this:

With the following explanation, for a failed test:

Conclusion

This is a prove of concept — so let me know what you think. The next step is to include these tests in a CI-Pipeline like Jenkins.

Working for finnova.com in the Banking business. Prefer to work with Scala / ScalaJS.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store