November 22, 2020 code coverage
Since Smallworld is now running on the JVM, lots of tools are now suddenly usable to do analysis on Smallworld code. Using JaCoCo it is possible to determine the code coverage of your Smallworld 5 Magik code and its tests. Code coverage lets you determine if your unit tests cover the parts certain parts of your code, usually the critical parts. This article shows how to use JaCoCo to determine the code coverage of a single test in sw_xsd_loader running in Smallworld 5.2.5.
JaCoCo is a tool which can instrument Java Bytecode on the fly when it is loaded. When the instrumented code runs, the executed source-lines are recorded. Using this data, a report can be created showing which lines were executed, for instance.
Before starting, download the latest JaCoCo release from the JaCoCo website. Furthermore, you’ll need a running Smallworld 5 installation. This guide uses Smallworld 5.2.5 and JaCoCo 0.8.6.
Steps to determine code coverage
To determine the code coverage in Smallworld 5 we need to do several steps:
- Compile the magik sources into jars
- Load the jars and instrument on the fly
- Run the tests
- Generate the report
1. Compile the magik sources into jars
Step 1 includes loading the product and calling the method
compile_all_modules() on it. This saves the compiled magik to (a) jar(s) under the
Magik> smallworld_product.add_product(".../munit") # For module dependencies. ... Magik> prd << smallworld_product.add_product(".../sw_xsd_loader") # Load the actual product under test. ... Magik> prd.compile_all_modules() # Compile modules, save the libs/jars. ... Magik> quit()
The quit is required as we need to load the libraries from disk, or JaCoCo won’t be able to instrument them.
2. Load the jars and instrument on the fly
Once the jars are saved to the
libs directory, these can be loaded: Step 2. Before we do this, we need to start our Smallworld session with a JaCoCO java-agent. Start your session with the additional parameter to
Another option is to edit your
environment file and add the following, before starting your session:
SET SW_LAUNCH_JAVA_ARGS=%SW_LAUNCH_JAVA_ARGS% -javaagent:...\jacoco-0.8.6\lib\jacocoagent.jar
Your session will start normally, but a file
jacoco.exec will be created. Then, in this session load your product again, load the module containing your test(s) and run the test:
Magik> smallworld_product.add_product(".../munit") # For module dependencies. ... Magik> smallworld_product.add_product(".../sw_xsd_loader") # Load the actual product under test. ...
3. Run the tests
Step 3 is running the tests themselves. First we load the tests-module, then we execute one test suite:
Magik> sw_module_manager.load_module(:xsd_loader_tests) # Load the tests for the product. ... Magik> test_runner.run_in_foreground(char16_vector_test.suite()) # Run a single test suite. ... Magik> quit()
JaCoCo now has recorded the code that lines of Magik code that were executed and stored in the file
4. Generate the report
Now that we have the file
jacoco.exec, we can generate a report from it. You can do this using the
jacococli.jar file, like so:
java -jar .../jacoco-0.8.6/lib/jacococli.jar report jacoco.exec --classfiles .../sw_xsd_loader/libs --html .../coverage-report
This will create a new directory called
coverage-report. In this directotry you’ll find a
index.html file which you can open with your browser. Navigate to the source file called
magik/sw_xsd_loader/xsd_loader/xsd_loader/source/char16_vector.magik and you’ll see the following:
Using readily available tools like JaCoCo we can determine the code coverage of tests. This gives us valuable information to determine what we test, and if test enough.