Two years ago I wrote about the Clojure test runner of my dreams. Back then, I asked for a Clojure test runner that would support clojure.test and that would have the following features:
- Output catching to make noisy test suites quiet.
- Test tagging for selectively running tests.
- JUnit output for integration with CI tools such as Circle.
- Test slowness reporting to speed up slow test suites.
Let’s see where we are now.
After my post, Eftest soon got all the features. It’s a great library and while the included Leiningen plugin is quite rudimentary, bat-test offers a feature-rich wrapper for use with Leiningen and Boot.
When Arne Brasseur announced that he is developing Kaocha,
I didn’t see much point in a new test runner. However, I was working a new
project that used tools.deps as the dependency management tool. Since bat-test
does not have a clj -m -compatible version, I decided to give Kaocha a go.
After using Kaocha for a couple of months, I have to admit that Arne has created a something worthwhile. Kaocha offers test running with all the above-mentioned features and more (e.g. watcher, Cloverage integration) in one coherent, well-documented package. Furthermore, Kaocha’s design allows easy extensions. For example, it was super-easy to create a plugin that toggles on Orchestra instrumentation.
If you’re looking for a new test runner, definitely check out Kaocha.
Side-note: Arne’s work on Kaocha has been funded by Clojurists Together. Their funding has paid for a number of improvements in Clojure projects you’re most likely using. The money comes from individual and corporate sponsors. It seems like a great way for Clojure-using companies to fund work on the tools that their developers use. I’m happy that Metosin has been a sponsor since the beginning and I’m hoping to see more companies on the member list.
What’s next?
Test runners are good now. Does this mean that the Clojure testing landscape is perfect now? Well… I see at least two areas of improvement.
ClojureScript unit testing. Doo (which I co-maintain) is tricky to set up, I’ve had a number of problems with boot-cljs-test, and using Karma directly is a lot of work. Getting my list of dream features to work is possible but tedious.
Luckily there are two new developments: Figwheel Main has built-in unit testing support and Kaocha has kaocha-cljs. Both look promising. This is what I’m most excited about Kaocha – I’m hoping that Kaocha manages to bring the same turn-key experience to ClojureScript that it already has for Clojure.
Assertion libraries. In clojure.test the assertion are written with the is
macro, which isn’t very expressive. The main selling point of Midje was
its powerful syntax for writing assertions. Unfortunately that power came with
the cost of a complex implementation.
Apart from writing your own predicates, what are the current options? I’m aware of testit (my go-to choice),
iota, and
matcher-combinators. Each of
them is significant improvement over is macro, but I don’t love any of them.
I guess I need to come up with a list of features for the assertion library of
my dreams!