I discovered Kotlin recently and realized that, for many people, it was only a new syntax for Java. But the more I play with it, the more convinced I am that a combo of two of its features is vastly underrated:
- The ability to create domain-specific languages (DSL), thanks to some syntactic sugar;
- The ability to write scripts and create interpreters for them.
You can easily guess how nice these features can play together. We can create a DSL, tailor it to our needs, and use it in scripts. Why scripts? Let me introduce the example I will develop here to illustrate why.
Not long ago, I started keeping track of my personal finances. I wanted to keep track of the total value of everything I own (money in bank accounts, stocks, real estate), and I was disappointed in the third-party services I found. So I developed my own tool to fit my needs. (Want to know more? I wrote a book about it!)
My program’s input had to be the values of my accounts over time (I wanted to manually write each value instead of relying on an unreliable API to fetch them automatically). To provide this input, I had quite a few options:
- I could store them in a database and develop tools to populate it (a GUI, a CLI…)
- I could put them in a JSON file
- Why not store them in a Kotlin file?
Providing the input in a Kotlin file sounded like a weird thing to do. On the one hand, I could benefit from IntelliJ’s autocomplete features since I would use the functions and classes I wrote. On the other hand, it meant I would have to rebuild the program each time I updated the input?? How unpractical does it seem?
Here is where scripts come into play. With Kotlin, you can write a script interpreter embedding your domain logic, including a DSL, and write scripts providing the input to that logic.
In this article, we’ll build a “minimum viable example”, a net worth tracker to demonstrate how it can work. But if you already know Kotlin, you likely already use scripts and DSLs without even knowing it: in your build.gradle.kts file, you are using the Gradle DSL in a script!
The full example is available on GitHub. The main branch contains the final version, and you’ll find there two branches step-1 and step-2 with the code of the first two sections. I removed the imports from the code examples to make the post lighter, but you can follow the link attached to code blocks to see the full file content.
Step 1: Create the domain logic
We want to write a program that takes accounts and snapshots as inputs and displays a report as an output. A snapshot represents the states of several accounts at a given date, so the class contains a date attribute and a mapping between accounts and their respective value.
// https://tinyurl.com/yckwfds4
data class Account(val id: String, val name: String)
// https://tinyurl.com/57vr36mw
data class Snapshot(val date: LocalDate, val balances: Map<Account, Double>)
To create a report, we need to provide a list of accounts and snapshots. Our minimal report will have two features: displaying the account list and displaying each snapshot (with, for each of them, the value of each account).
// https://tinyurl.com/2whmpup6
class Report(val accounts: List<Account>, val snapshots: List<Snapshot>) {
fun displayAccountList() {
println("ACCOUNTS:")
accounts.forEach { println("- ${it.name}") }
}
fun displaySnapshots() {
println("SNAPSHOTS:")
snapshots.forEach { snapshot ->
println("* ${snapshot.date}:")
accounts.forEach { account ->
println(" - ${account.name}: ${snapshot.balances[account]?.toString() ?: "-"}")
}
}
}
}
As a first step, let’s use these classes in the main
function. We first create two accounts, then two snapshots defining a value for these accounts. Finally, we create a report and display the accounts’ and the snapshots’ list.
// https://tinyurl.com/s8564hy5
fun main(args: Array<String>) {
val checking = Account("checking", "Checking")
val savings = Account("savings", "Savings")
val snapshots = listOf(
Snapshot(
LocalDate.parse("2022-01-01"),
mapOf(checking to 1000.0, savings to 2000.0)
),
Snapshot(
LocalDate.parse("2022-02-01"),
mapOf(checking to 1200.0, savings to 2500.0)
)
)
val report = Report(listOf(checking, savings), snapshots)
report.displayAccountList()
report.displaySnapshots()
}
Ok, our program works, but we will improve it in two ways: first, we will make it possible to extract the content of main
into a script (so we don’t have to rebuild the application when we add new snapshots), then we will create some helpers (our DSL) to make it easier to edit the script.
Step 2: Create a script definition
Using Kotlin to write scripts is unfortunately not a very well-documented feature. It changed a lot in the past, so the tutorials you can find are not always still valid. But it doesn’t mean we cannot enjoy it!
The first step is to include some dependencies to our project:
// https://tinyurl.com/5ase8sdr
dependencies {
// ...
implementation("org.jetbrains.kotlin:kotlin-scripting-common:1.6.0")
implementation("org.jetbrains.kotlin:kotlin-scripting-jvm:1.6.0")
implementation("org.jetbrains.kotlin:kotlin-scripting-jvm-host:1.6.0")
}
To be able to write a script interpreter, we need to create two entities: a script definition and a script host. Let’s start with the script definition, NwtScript
(nwt
stands for net worth tracker). It can be as simple as an abstract class annotated with @KotlinScript
. To make writing script easier, we can also provide some classes that will automatically be imported, using a ScriptCompilationConfiguration
object:
// https://tinyurl.com/yc26svyp
@KotlinScript(
fileExtension = "nwt.kts",
compilationConfiguration = NwtScriptConfiguration::class
)
abstract class NwtScript
object NwtScriptConfiguration: ScriptCompilationConfiguration({
defaultImports(Account::class, Snapshot::class, Report::class, LocalDate::class)
})
We also provide an extension for our scripts; note that, so we enjoy IntelliJ’s features such as autocomplete, this extension has to be composed of a prefix of your choice, then .kts
.
We can now create the script host NwtScriptHost
to use this script definition. This class has nothing very complex: it has a public execFile
method that accepts a filename, evaluates it using a BasicJvmScriptingHost
, and displays possible log messages.
// https://tinyurl.com/2p88u9a5
class NwtScriptHost {
private fun evalFile(scriptFile: File): ResultWithDiagnostics<EvaluationResult> {
val compilationConfiguration = createJvmCompilationConfigurationFromTemplate<NwtScript> {
jvm {
// Script will have access to everything in the classpath
dependenciesFromCurrentContext(wholeClasspath = true)
}
}
return BasicJvmScriptingHost().eval(scriptFile.toScriptSource(), compilationConfiguration, null)
}
fun execFile(path: String) {
val scriptFile = File(path)
val res = evalFile(scriptFile)
res.reports.forEach {
if (it.severity > ScriptDiagnostic.Severity.DEBUG) {
println(" : ${it.message}" + if (it.exception == null) "" else ": ${it.exception}")
}
}
}
}
As the final step before we create our script, we need to tell IntelliJ about our script definition, so it knows what classes we can access and provide the autocomplete and code annotation features we like.
We need to create a folder src/main/resources/META-INF/kotlin/script/templates
and in it an empty file named nwt.script.NwtScript.classname
(the full class name + .classname
). Here is the part I’m least comfortable explaining because I found how to make it work on some forums and not in official documentation 😅.
After creating the file, rebuild the application, and you may need to restart IntelliJ, invalidate caches… To be honest, I’m not sure exactly what works, but in the end, when you create a .nwt.kts
file (let’s say my.nwt.kts
at the project’s root), IntelliJ should tell you there is a script definition available for it 🎉.
Put the content of main
in the new script file, and if everything went well, you should see that IntelliJ annotates the parameters of Account
, for instance:
// https://tinyurl.com/4enfpevm
val checking = Account("checking", "Checking")
val savings = Account("savings", "Savings")
val snapshotJanuary = Snapshot(
LocalDate.parse("2022-01-01"),
mapOf(checking to 1000.0, savings to 2000.0)
)
val snapshotFebruary = Snapshot(
LocalDate.parse("2022-02-01"),
mapOf(checking to 1200.0, savings to 2500.0)
)
val report = Report(
listOf(checking, savings),
listOf(snapshotJanuary, snapshotFebruary)
)
report.displayAccountList()
report.displaySnapshots()
To transform our program into an interpreter for our script, we can now update our main
function:
// https://tinyurl.com/mpzhf66e
fun main(vararg args: String) {
if (args.size != 1) {
println("usage: <app> <script file>")
} else {
NwtScriptHost().execFile(args[0])
}
}
To run it, we can update the project’s run configuration, defining “my.nwt.kts” in the Program arguments field. You will notice that the application won't be rebuilt if you update the script file and rerun it!
It is already lovely, but we can make our script better by providing high-level functions, so we don’t feel like writing code!
Step 3: Create builders and the DSL
We will use a well-known design pattern to implement our DSL: the Builder pattern. For instance, to create a Snapshot
, we can create a class SnapshotBuilder
. Its constructor takes two parameters: the snapshot’s date and a map of accounts (we will see later how we provide these accounts).
// https://tinyurl.com/4tuffu26
class SnapshotBuilder(val date: LocalDate, val accounts: Map<String, Account>) {
private val balances: MutableMap<Account, Double> = mutableMapOf()
fun balance(accountId: String, amount: Double) {
val account = accounts[accountId] ?: throw Exception("Invalid account ID")
balances[account] = amount
}
fun build(): Snapshot {
return Snapshot(date, balances)
}
}
To use the builder, we can call its balance
method to define the accounts’ balances; then, we call the build
method to get the snapshot.
val builder = SnapshotBuilder(LocalDate.parse("2022-01-01"), accounts)
builder.balance("checking", 1000.0)
builder.balance("savings", 2000.0)
val snapshot = builder.build()
But we can do better. We can define a snapshot
function taking two parameters:
-
date
: the snapshot’s date, -
initialize
: a function to initialize the builder. This function will receive the builder asthis
(thanks to the call toapply
), meaning that we will be able to call thebalance
method without any prefix.
fun snapshot(val date: String, initialize: SnapshotBuilder.() -> Unit): Snapshot {
// We’ll see soon where `accounts` comes from
return SnapshotBuilder(LocalDate.parse(date), accounts).apply(initialize).build()
}
val snapshot = snapshot("2022-02-01") {
balance("checking", 1000.0)
balance("savings", 2000.0)
}
To me, this is what makes Kotlin great for writing DSLs. When using the snapshot
function, you don’t realize you are creating and using a builder!
Let’s create another builder to build several snapshots as a next step. This new SnapshotsBuilder
will use the SnapshotBuilder
and embed the snapshot
we just saw (renaming it on
for a better DSL, you’ll see):
// https://tinyurl.com/52fbfhhn
class SnapshotsBuilder(val accounts: Map<String, Account>) {
private val snapshots: MutableList<Snapshot> = mutableListOf()
fun on(date: String, initialize: SnapshotBuilder.() -> Unit) {
val snapshotBuilder = SnapshotBuilder(LocalDate.parse(date), accounts).apply(initialize)
snapshots.add(snapshotBuilder.build())
}
fun build(): List<Snapshot> {
return snapshots
}
}
We will use this builder via another function, snapshots
:
fun snapshots(initialize: SnapshotsBuilder.() -> Unit) {
// We’ll see soon where `accounts` comes from
snapshots.addAll(SnapshotsBuilder(accounts).apply(initialize).build())
}
snapshots {
on("2022-01-01") {
balance("checking", 1000.0)
balance("savings", 2000.0)
}
on("2022-02-01") {
balance("checking", 1200.0)
balance("savings", 2500.0)
}
}
Let’s continue by creating a builder for reports. The ReportBuilder
class technically doesn’t implement the builder pattern since it doesn’t build anything (and doesn’t have a build
method), but it will still be helpful to us:
// https://tinyurl.com/2v35d9fu
class ReportBuilder(val accounts: List<Account>, val snapshots: List<Snapshot>) {
private val report = Report(accounts, snapshots)
fun accounts() {
report.displayAccountList()
}
fun snapshots() {
report.displaySnapshots()
}
}
Same as we did for snapshots, we can create a report
function.
fun report(initialize: ReportBuilder.() -> Unit) {
// We’ll see soon where `accounts` and `snapshots` come from
ReportBuilder(accounts.values.toList(), snapshots).apply(initialize)
}
report {
accounts()
snapshots()
}
Last but not least, let’s create the last builder to unify them all! Our NwtBuilder
embeds the snapshots
and report
functions as methods and is responsible for keeping the state of our program: the accounts
and the snapshots
:
// https://tinyurl.com/2p8n8av6
class NwtBuilder {
private val accounts: MutableMap<String, Account> = mutableMapOf()
private val snapshots: MutableList<Snapshot> = mutableListOf()
fun account(id: String, name: String) {
accounts[id] = Account(id, name)
}
fun snapshots(initialize: SnapshotsBuilder.() -> Unit) {
snapshots.addAll(SnapshotsBuilder(accounts).apply(initialize).build())
}
fun report(initialize: ReportBuilder.() -> Unit) {
ReportBuilder(accounts.values.toList(), snapshots).apply(initialize)
}
}
// https://tinyurl.com/3cue36re
fun nwt(initialize: NwtBuilder.() -> Unit) {
NwtBuilder().apply(initialize)
}
With the new nwt
function, we won’t need access to the classes Account
, Snapshot
, and Report
in the script. So we can update the script definition:
// https://tinyurl.com/mhecyc6n
@KotlinScript(fileExtension = "nwt.kts", compilationConfiguration = NwtScriptConfiguration::class)
abstract class NwtScript
object NwtScriptConfiguration: ScriptCompilationConfiguration({
// give access to our `nwt` function without importing it
defaultImports("nwt.dsl.nwt")
})
Now we have everything we need to update our script. No more val
, no more explicit object creation. Only the bare minimum to provide to the interpreter the inputs (the accounts and their balances over time) and what we want as output (the report definition).
// https://tinyurl.com/3xh24mny
nwt {
account("checking", "Checking")
account("savings", "Savings")
snapshots {
on("2022-01-01") {
balance("checking", 1000.0)
balance("savings", 2000.0)
}
on("2022-02-01") {
balance("checking", 1200.0)
balance("savings", 2500.0)
}
}
report {
accounts()
snapshots()
}
}
Again, since we updated the script definition, you may need to rebuild the project, invalidate caches, restart IntelliJ, etc., to make it effective. But look at how nice it is that IntelliJ recognizes our DSL:
Kotlin is already widely used to write domain-specific languages. You can use it to write Gradle configuration files, to create HTML documents, and I hope I convided you that creating your own DSL is worth it too, especially used in combination with the ability to write script interpreters.
Top comments (1)
I agree with you that programming language’s feature for defining DSL is mostly underrated but, as you well explained, could be a real game changer in developing sw