Consuming a modular OCaml project structure

IN MY first post on this blog, I proposed a module-oriented project layout for OCaml/ReasonML. In short, a modular project structure. One of the key points there was that each project should expose a single toplevel module, with a name that's as unambiguous as possible, so that it can be unique across all possible projects in the ecosystem.

Of course in the general case this is impossible, but it should be possible to approximate by using a GitHubUsername__Project naming structure, e.g. Yawaramin__SomeProject. (As I explained previously, because of the way modules work, different projects can't put different submodules inside the same parent module.) So suppose I publish some projects with the toplevel modules:

• Yawaramin__Project1
• Yawaramin__Project2

Suppose 'Bob' publishes:

• Bob__Project1

Then, you set up a new project that consumes all three of the above projects. One option is to use these modules as-is throughout your project:

let x = Yawaramin__Project1.Something.x;

// OR

module Something = Yawaramin__Project1.Something;
...
let x = Something.x;

But for the same reasons that I recommend actually setting up nested modules for your own project in the first place, I also recommend setting up nested modules for third-party projects. Specifically, to set up toplevel modules based on a Scope.Project format. It would work like this in your project layout:

src/
  Bob.re
  Yawaramin.re
  ... other files ...

And the files would contain the aliased modules:

// Bob.re
module Project1 = Bob__Project1;

// Yawaramin.re
module Project1 = Yawaramin__Project1;
module Project2 = Yawaramin__Project2;

Now, you get the toplevel-aliased modules as 'namespaces' throughout your project:

let x = Yawaramin.Project1.Something.x;

With the accompanying convenient auto-completion, and the immediate visibility of what namespaces are available in the toplevel.

Namespaces

The above layout results in a modular structure that is, effectively, namespacing (scope -> project -> modules), and this is very much intentional. As the Zen of Python states, in its final aphorism:

Namespaces are one honking great idea -- let's do more of those!

I've said this before but I think we can learn a lot in the ReasonML/OCaml world from the Zen of Python, and one of the biggest lessons is that we need better namespacing.

You may rightly point out that namespaces should be provided by the compiler or at least the build system, not manually by the users. This is true! There have been various discussions and suggestions about namespacing. But at the moment we don't have a concrete roadmap for a full-fledged namespacing solution:

1. Toplevel namespace module (BuckleScript, dune)
2. Namespace module for each directory (dune, but only with dune file in each directory to tell dune it's a 'library')
3. 'Scope' namespace (neither)

Only a manual approach can cover all three aspects of what I consider to be the full solution.

The 'where did this module come from?' problem

One of the problems in the OCaml/ReasonML ecosystem is that various projects expose various modules at the toplevel, and users need to keep track of what they are and where they came from.

BuckleScript and dune both mitigate this by default, by wrapping every project in a toplevel namespacing module. But they only do this for one level of wrapping–i.e., only the project name–not the 'scope' or 'user name'. So if you want to namespace by scope as well (which I think is a good idea, because the same user/org can publish many packages), you still have to do that manually, like above.

Once you do that, though, you end up with a set of toplevel modules in your project that exactly capture where all third-party modules are coming from. So suppose you see a module used in your project: ReasonML.React. You open up your project src/ directory, see a file ReasonML.re there–and unless there's some trickiness with global module opens earlier in the file or in the project, you immediately know that ReasonML.React can only be from there.

Another scenario, of course, is that a module Foo.Bar is an alias of some other module, e.g.:

module Foo = Yawaramin.Foo;
...
let x = Foo.Bar.x;

Again you can follow the process of looking for the toplevel module Yawaramin–if you have src/Yawaramin.re in your project then you've found it.

Reasoning

So why would you actually need all this? Careful namespace management is the concern of projects that need to scale up, and in the OCaml/Reason world people right now often tend to put all their modules in a single (global) namespace because, well, they just don't have that many modules.

And that's OK too. You don't need this immediately. But as the ecosystem grows larger, and people maybe want to publish projects with the same name, say @person1/coolproject and @person2/coolproject, we'd be able to have their modules be neatly namespaced in Person1.Coolproject and Person2.Coolproject, instead of clashing with the same toplevel module. That would be a honking good idea.