At Contact-Stack, we've recently added Elm to one of our Phoenix projects. Elm is a fantastic language with a clear syntax, functional design, immutable data and a helpful compiler.

For reasons of mild personal preference, we opted to not use elm-webpack-loader when integrating the Elm code with the current Javascript setup that we already have from Phoenix. Though ideally, we would still like the experience we have when we edit the Javascript, ie. it is rebuilt and Phoenix reloads the current browser with the newly built assets.

Unfortunately for us the Elm compiler doesn't have a 'watch' mode so we can't rely on that. We need a separate process to run the Elm compiler whenever there is a change. I cannot find it now but I've seen comments from Richard Feldman, a prominent member of the Elm community, suggesting that the Chokidar project can be used to set up a simple watcher which runs the Elm compiler. Chokidar is a node project that does a great job of wrapping some of the node standard library functionality to provide a robust watcher. It is used by a number of high profile node projects, including Webpack, to provide file watching functionality.

For reference, the exact build command that I would like run is:

elm make src/Main.elm --output=../priv/static/js/elm.js

From within the assets directory in the standard Phoenix project layout.

Now to start, we might consider adding the chokidar-cli which allows you to set up watchers with a simple command. We can add chokidar-cli with yarn add -D chokidar and then run:

chokidar "**/*.elm" -i node_modules -c "elm make src/Main.elm --output=../priv/static/js/elm.js"

If we run that in the assets directory it works great so maybe that's a good start. We add -i node_modules as for some reason chokidar starts tracking some files in node_modules too and we don't need it to.

How do we go about adding it to Phoenix? If we look in the config/dev.exs we see a block that looks like this:

config :contact_stack, ContactStackWeb.Endpoint,                                                                                                           
  http: [port: 4000],                                                                                                                                      
  debug_errors: true,                                                                                                                                      
  code_reloader: true,                                                                                                                                     
  check_origin: false,                                                                                                                                     
  watchers: [                                                                                                                                              
    node: [                                                                                                                                                
      "node_modules/webpack/bin/webpack.js",                                                                                                               
      "--mode",                                                                                                                                            
      "development",                                                                                                                                       
      "--watch-stdin",                                                                                                                                     
      cd: Path.expand("../assets", __DIR__)                                                                                                                
    ]                                                                                                                                                      
  ]                                                                                                                                                        

The relevant entry, as you might guess, is the watchers list. This is a list of key-value pairs that each provide a program and a set of arguments for Phoenix to run as part of its watcher functionality. So in this case, it is going to run node with that list of arguments which will result in it running webpack in development mode. The last part is to ask Phoenix to run it in the assets directory.

So we could try to extend this way:

   watchers: [
     node: [
       "node_modules/webpack/bin/webpack.js",
       "--mode",
       "development",
       "--watch-stdin",
       cd: Path.expand("../assets", __DIR__)
+    ],
+    node: [
+      "node_modules/.bin/chokidar",
+      "**/*.elm",
+      "-i",
+      "node_modules",
+      "-c",
+      "elm make src/Main.elm --output=../priv/static/js/elm.js",
+      cd: Path.expand("../assets", __DIR__)
     ]
   ]

And actually, this seems to work great. We run this and, every time we save an Elm file, Phoenix runs the Elm compiler with the command we've provided.

Unfortunately, if we kill the Phoenix server and check the running processes on our machine:

^C
BREAK: (a)bort (A)bort with dump (c)ontinue (p)roc info (i)nfo
       (l)oaded (v)ersion (k)ill (D)b-tables (d)istribution
$ ps -ef | grep chokidar
michael  17499     1  0 16:16 ?        00:00:00 /home/michael/.nave/installed/12.14.0/bin/node node_modules/.bin/chokidar "**/*.elm" -c "elm make src/Main.elm --output=../priv/static/js/elm.js"

We can see that the Chokidar process is still running. This is not great. We want to be able to freely restart our Phoenix dev server as often as we like without building up a back log of Chokidar processes which are all watching our Elm files and each running the Elm compiler on every change.

So why is this happening? Well, I'm guessing that Phoenix uses Elixir's Port functionality to run the watcher subprocesses and Elixir's Ports functionality as a big warning about zombie processes that are left over when the main Elixir system process has stopped. It seems that Elixir doesn't, or perhaps can't, proactivity kill subprocesses that it has started when closing down. Rather it relies on those subprocesses noticing that their standard input has been closed and exiting themselves. I haven't come across this mechanism outside of Elixir but it might well be common. And if we glance up at the webpack watcher config again we'll see that they're using a --watch-stdin flag. Coincidence? Probably not.

Unfortunately, chokidar-cli doesn't have a --watch-stdin flag nor any search results for stdin in the code so it seems like we can't rely on that.

But webpack is written in Javascript running on node so it must be possible and the main chokidar package is a library that allows you to easily access the file-watching functionality.

If we go splunking through the webpack code looking for references to standard input then we come across these lines in the webpack-cli project. Now I don't perfectly understand what is going on here but it seems that it is listening for a 'end' event on the standard input stream and using process.exit() to close the whole program if it happens. That seems to fit with what Elixir's Ports expect.

If we combine that with some relatively basic chokidar library usage as they outline in the README then we get something like this:

const chokidar = require("chokidar");
const { execSync } = require("child_process");

// Exit the process when standard input closes due to:
//   https://hexdocs.pm/elixir/1.10.2/Port.html#module-zombie-operating-system-processes
//
process.stdin.on("end", function() {
    console.log("standard input end");
    process.exit();
});

process.stdin.resume();

// Set up chokidar to watch all elm files and rebuild the elm app ignoring process errors
chokidar.watch("**/*.elm", { ignored: "node_modules" }).on("all", (event, path) => {
    console.log(event, path);
    try {
        execSync("./node_modules/.bin/elm make src/Main.elm --output=../priv/static/js/elm.js");
    } catch (error) {}
});

And if we save it in a file called assets/watch-elm.js. And then we change our config/dev.exs config to read:

    node: [
      "./watch-elm.js",
       cd: Path.expand("../assets", __DIR__)
    ]

Then we can run mix phx.server and see that not only does the Elm compiler get run correctly on changes but when we kill our dev server the watcher process dies too. Success!

Conclusion

Adding new watcher processes to Phoenix is relatively easy in some ways but this matter of watching standard-input is a bit confusing and is probably handled quite differently in different languages.

Notes

The Phoenix documentation does provide a helper bash script that you can use to wrap an executable and which does the 'listening for standard input to close' for you. I have used that successfully when having to run a subprocess during Elixir tests but I wasn't unable to get it working on first try in this situation. Possibly the complexities of the extra escaping of the command line arguments got the better of me. I'm not sure.