Since Smallworld is now running on the JVM, lots of tools are now suddenly usable to do analysis on Smallworld code. Using JaCoCo it is possible to determine the code coverage of your Smallworld 5 Magik code and its tests. Code coverage lets you determine if your unit tests cover the parts certain parts of your code, usually the critical parts. This article shows how to use JaCoCo to determine the code coverage of a single test in sw_xsd_loader running in Smallworld 5.2.5.

JaCoCo is a tool which can instrument Java Bytecode on the fly when it is loaded. When the instrumented code runs, the executed source-lines are recorded. Using this data, a report can be created showing which lines were executed, for instance.

Preliminary

Before starting, download the latest JaCoCo release from the JaCoCo website. Furthermore, you’ll need a running Smallworld 5 installation. This guide uses Smallworld 5.2.5 and JaCoCo 0.8.6.

Steps to determine code coverage

To determine the code coverage in Smallworld 5 we need to do several steps:

1. Compile the magik sources into jars
2. Load the jars and instrument on the fly
3. Run the tests
4. Generate the report

1. Compile the magik sources into jars

Step 1 includes loading the product and calling the method compile_all_modules() on it. This saves the compiled magik to (a) jar(s) under the libs directory.

Magik> smallworld_product.add_product(".../munit")  # For module dependencies.
...
Magik> prd << smallworld_product.add_product(".../sw_xsd_loader")  # Load the actual product under test.
...
Magik> prd.compile_all_modules()  # Compile modules, save the libs/jars.
...
Magik> quit()

The quit is required as we need to load the libraries from disk, or JaCoCo won’t be able to instrument them.

2. Load the jars and instrument on the fly

Once the jars are saved to the libs directory, these can be loaded: Step 2. Before we do this, we need to start our Smallworld session with a JaCoCO java-agent. Start your session with the additional parameter to runalias: -j -javaagent:.../jacoco-0.8.6/lib/jacocoagent.jar.

Another option is to edit your environment file and add the following, before starting your session:

SET SW_LAUNCH_JAVA_ARGS=%SW_LAUNCH_JAVA_ARGS% -javaagent:...\jacoco-0.8.6\lib\jacocoagent.jar

Your session will start normally, but a file jacoco.exec will be created. Then, in this session load your product again, load the module containing your test(s) and run the test:

Magik> smallworld_product.add_product(".../munit")  # For module dependencies.
...
Magik> smallworld_product.add_product(".../sw_xsd_loader")   # Load the actual product under test.
...

3. Run the tests

Step 3 is running the tests themselves. First we load the tests-module, then we execute one test suite:

Magik> sw_module_manager.load_module(:xsd_loader_tests)  # Load the tests for the product.
...
Magik> test_runner.run_in_foreground(char16_vector_test.suite())   # Run a single test suite.
...
Magik> quit()

JaCoCo now has recorded the code that lines of Magik code that were executed and stored in the file jacoco.exec.

4. Generate the report

Now that we have the file jacoco.exec, we can generate a report from it. You can do this using the jacococli.jar file, like so:

java -jar .../jacoco-0.8.6/lib/jacococli.jar report jacoco.exec --classfiles .../sw_xsd_loader/libs --html .../coverage-report

This will create a new directory called coverage-report. In this directotry you’ll find a index.html file which you can open with your browser. Navigate to the source file called magik/sw_xsd_loader/xsd_loader/xsd_loader/source/char16_vector.magik and you’ll see the following:

Conclusion

Using readily available tools like JaCoCo we can determine the code coverage of tests. This gives us valuable information to determine what we test, and if test enough.