Recently I wrote about Smallworld/Magik Code Coverage using Jacoco. This worked ok’ish, although it looks a bit weird with manged exemplar/method names. Furthermore, it only gave a percentage of the code that was run, seeing which lines were actually ‘hit’ was impossible. The sw5-jacoco-reporter is a report generator which is ended towards Smallworld 5 generating a better report with individual line coverage and proper method names.

JaCoCo is a tool to determine which code was run, during your unit tests for example. Although JaCoCo targets Java (hence the name Java Code Coverage Library,) it actually can be used for other languages which run on the JVM as well. The JaCoCo API is readily available with examples to use and extend JaCoCo to your own needs. As such, JaCoCo makes it easy to build your own report generator.

Given the way Smallworld 5 generates the Java bytecode, it is already fairly similar to the resulting Java bytecode generated by Java. This makes it easy for JaCoCo to already generate a report, albeit with some unresolved details such as original method names.

To generate a better coverage report, JaCoCo needs to be provided some additional information such as the source files. Furthermore, a transformation has to be done to use the original method names. Basically, this is what sw5-jacoco-reporter does.

The result of using sw5-jacoco-reporter is a report which provides the original method names and branch- and line-coverage. See the end of this article for examples.

Preliminary

Before starting, download the latest JaCoCo release from the JaCoCo website. Furthermore, you’ll need a running Smallworld 5 installation. This guide uses Smallworld 5.2.5 and JaCoCo 0.8.6.

Download the sw5-jacoco-reporter binary, or find the latest release at the releases page in the Github repository.

Steps to determine code coverage

To determine the code coverage in Smallworld 5 we need to do several steps:

1. Compile the magik sources into jars
2. Load the jars and instrument on the fly
3. Run the tests
4. Generate the report using sw5-jacoco-reporter

Note that these steps are mostly the same as the original article, except for step 4 which uses the new reporter. Steps 1, 2, and 3 are included here for clarity. For more information see the article Smallworld/Magik Code Coverage using Jacoco.

1. Compile the magik sources into jars

Step 1 includes loading the product and calling the method compile_all_modules() on it. This saves the compiled magik to (a) jar(s) under the libs directory.

Magik> smallworld_product.add_product(".../munit")  # For module dependencies.
...
Magik> prd << smallworld_product.add_product(".../sw_xsd_loader")  # Load the actual product under test.
...
Magik> prd.compile_all_modules()  # Compile modules, save the libs/jars.
...
Magik> quit()

The quit is required as we need to load the libraries from disk, or JaCoCo won’t be able to instrument them.

2. Load the jars and instrument on the fly

Once the jars are saved to the libs directory, these can be loaded: Step 2. Before we do this, we need to start our Smallworld session with a JaCoCO java-agent. Start your session with the additional parameter to runalias: -j -javaagent:.../jacoco-0.8.6/lib/jacocoagent.jar.

Another option is to edit your environment file and add the following, before starting your session:

SET SW_LAUNCH_JAVA_ARGS=%SW_LAUNCH_JAVA_ARGS% -javaagent:...\jacoco-0.8.6\lib\jacocoagent.jar

Your session will start normally, but a file jacoco.exec will be created. Then, in this session load your product again, load the module containing your test(s) and run the test:

Magik> smallworld_product.add_product(".../munit")  # For module dependencies.
...
Magik> smallworld_product.add_product(".../sw_xsd_loader")   # Load the actual product under test.
...

3. Run the tests

Step 3 is running the tests themselves. First we load the tests-module, then we execute one test suite:

Magik> sw_module_manager.load_module(:xsd_loader_tests)  # Load the tests for the product.
...
Magik> test_runner.run_in_foreground(char16_vector_test.suite())   # Run a single test suite.
...
Magik> quit()

JaCoCo now has recorded the code that lines of Magik code that were executed and stored in the file jacoco.exec.

4. Generate the report using sw5-jacoco-reporter

Now that we have the file jacoco.exec, we can generate a report from it. You can do this using the jacococli.jar file, like so:

java -jar .../sw5-jacoco-reporter-1.0.0.jar --jacoco-file .../jacoco.exec --product-dir .../sw_xsd_loader --html .../coverage-report-sw5

This will create a new directory called coverage-report-sw5. In this directory you’ll find a index.html file which you can open with your browser. Navigate to the source file called magik.sw_xsd_loader.xsd_loader.char16_vector_31 and you’ll see the following:

Clicking on the method will show the individual branch- and line-coverage:

Conclusion

Earlier JaCoCo already provided some useful information about the code coverage our unit tests. Now, with the sw5-jacoco-reporter, better insights are provided given the individual branch- and line-coverage and proper method names.