Strategies for organizing large Model-View-Update projects that have worked for us.

We have been working on functional UI projects using the MVU pattern for 4 years now. We first started using this pattern with Elm. Then we switched to full stack F#, using the Elmish library for MVU. We evolved our approach to MVU several times along the way. This post details some practices we learned.

This post assumes some experience with MVU. I do not mention the view function very much since we do not usually find it a challenge to organize. A view function for a particular page usually resides near that page's update function.

Modified TEA for UI state

The Elm Architecture (TEA) is one of the early ideas behind Elm. It defined a nested hierarchy of Msg, Model, Update, and View. But it was disavowed by Elm because it has a fatal flaw. TEA required a lot of laborious boilerplate to convert Msg between parents and children. It also made simultaneous active pages very difficult. We went away from it. But as our programs got larger, we found ourselves with gigantic Msg and update files, requiring us to use Ctrl-F to find things inside them. Basically everything was in a couple of huge piles of code instead of being organized for quick human access.

Quick stats
Latest project's UI code (F#)

• Largest app - 35k lines of code
• All apps - 65k lines of code

It became clear that we needed some organization. TEA is really just grouping all the things for a single page together in a module. This idea makes a lot of sense, so we went back to that for everything except Msg. All Msgs are declared first in their entirety, centrally and separately from the page parts. This means that every page module can directly send any messages they want, even to other pages. This got rid of all that Msg mapping and OutMsg patterns plaguing TEA.

Update 6 Jan 2021: I just checked out Elm's current documentation and it now mentions only Model, init, view, and update in page modules. So they must have come to a similar conclusion. When we were in the Elm community TEA included Msg but became disavowed because of its shortcomings, and the documentation about it was removed. Then there was a propaganda blitz for flat Msg. We switched away from Elm before this latest iteration on Elm app structure.

In our modified TEA, Msg can still have a hierarchical organization pattern as needed. So we can place a particular set of messages in their own file.

Remember that organization works for you, not the other way around. Every time you add a level of organization to your project, you also add more overhead (structure). It is better to keep things flat as much as possible. And only increase the degree of organization when you discover it is needed.

for UI state

An important subtlety in the section title is "Modified TEA for UI state". Often an app will have data that it is tracking independent of whatever the UI elements are doing. This kind of data should be placed outside the UI state. Everything in the UI state should be viewed as just what is currently being displayed, available to throw away at any time.

It is necessary to point this out because our industry is so accustomed to object-oriented UIs. There the ownership of state is distributed among many different objects. This can lead MVU users to try to make UI pages "own" specific kinds of data on behalf of other pages. But with MVU, pages only represent what is displayed on the screen. This is the way.

This has a couple of major advantages. It eliminates a common source of bugs: two different pages using shared state inconsistently. It also eliminates the need to manage the UI state's lifecycle, since it is part of the page lifecycle.

// changing pages
let pageData, cmds = Home.init ()
{ model with Page = Home pageData }, cmds

In the above example, the important thing here is what is missing. There is no code to check the previous page. Or clean up shared state. It is not needed. Any data that the page needs it will request or create on init.

Independent data

What if pages need to display or update that shared, independent data? The best answer really depends on the situation, but here are some approaches. If the independent data is read-only reference data, you can pass it as an argument to the page's init.

For a more robust approach, treat it as an external integration. This works for cases where the state isn't static or pages can request changes to it. For this, provide Cmds to make these requests. Also Msg cases to process changes in an MVU style. This is a headless MVU service pattern, where the independent data is the service state.

What does "independent data" look like?

I keep using this term independent data like it is something special. It is not. To demonstrate how "not a big thing" it is, here is an example Model.

type Model =
  { ApiKey: string
    Page: Page }

Here, ApiKey is what I am calling independent data. And Page is a piece of UI state. The app model could contain multiples of either kind. The Msg for this app might contain a set of messages to manipulate either or both parts of Model. It depends entirely on what the app needs.

update refinements

UI states are just data that represents what is on the screen. So in them we make extensive use of both records and (discriminated) unions. Records are what most people are most accustomed to, similar to structs or DTOs. Unions like you will see below are used when the state can only be one of multiple possibilities. Here is a typical example of a UI state:

type Page =
  | Home of Home.Model
  | CourseSearch of Course.Search.Model
  | CourseDetail of Course.Detail.Model
  // and so forth

This is basically spelling out that Page can be in one of these states. And then each page also has its own inner state (for example Home.Model), which is also some combination of records and unions.

That is well and good, but there are a couple of practical issues when we go to use this in update. With this structure we have to dig the page data out of the union type. That is Problem #1. Separately, since Msg is an app-wide type -- where any page can send any message -- now it is possible to send a Msg to a page which is not active. That is Problem #2.

Technically #2 was always possible. A delayed API operation and an impatient user can lead to receiving a response for page which is no longer active.

To resolve both of these problems, we went to an update style that looks like this:

let update msg model =
  match model.Page, msg with
  | Home pageData, HomeMsg pageMsg ->
    let pageData, cmds = Pages.Home.update pageMsg pageData
    { model with Page = Home pageData }, cmds

This serves to unwrap the page and message data from their respective union types. While also verifying we are handling a message appropriate for the UI state.

You might also notice this has a strong similarity to something else -- a state machine. At its core, a state machine is just a set of states and the valid transitions between them. This match is spelling out the valid transitions (messages) for each UI state.

We also do something that many of you FPers will not like. We use a catch-all match for any unmatched state transitions. This allows us to specify only the scenarios that we want to handle and ignore the rest.

let update msg model =
  match model.Page, msg with
  ...
  | _ ->
    // no changes
    model, []

This is a classic expression problem trade-off of extensibility vs completeness. With the underscore match we are choosing extensibility. By choosing this we lose the advantage of the compiler telling us of intended matches that we forgot. But we also do not get the multitude more unintended matches that we want to ignore. We believe our choice makes sense under the circumstances. Logging the catch-all case provides enough of a hint when we forget to add an intended case.

Routing and Navigation

A "route" is simply enough data to load a specific page. In MVU, pages are constructed by their init function. Therefore a route is just the arguments to the page's init function, along with a tag to indicate the page. It is so much simpler to think about it this way, that I eventually started naming this InitArgs in new projects instead of Route. And this can be used practically to package up all the arguments to the init function. Here is an example.

// PageTypes\Course\Detail.fs
module Course.Detail =
  type InitArgs =
    { CourseId: int }

// AppTypes.fs
type InitArgs =
  | HomeInit of ()
  | CourseDetailInit of Course.Detail.InitArgs

// PageFns\Course\Detail.fs
module Course.Detail
  let init initArgs =
    { CourseId = initArgs.CourseId
      Data = Loading }
    , [ LoadCourseDetail initArgs.CourseId ]

I'll cover the folder organization of the project later.

Switching pages then is a matter of adding a Msg case.

// AppTypes.fs
type Msg =
  | SwitchPage of InitArgs

And handling page changes looks like this.

let switchPage initArgs =
  match initArgs with
  | HomeInit pageArgs ->
    let pageData, cmds = Home.init pageArgs
    { model with Page = Home pageData }, cmds
  | CourseDetailInit pageArgs ->
    let pageData, cmds = Course.Detail.init pageArgs
    { model with Page = CourseDetail pageData }, cmds

let update msg model =
  match model.Page, msg with
  | _, SwitchPage initArgs ->
    switchPage initArgs

Like Msg, InitArgs must be centralized and declared ahead of time so that any page has the ability switch to any other page. Or to calculate a URL so the user can switch pages with a link. Speaking of...

adding URL navigation

The main code you have to fill in for this involves converting InitArgs toUrl and fromUrl. Here is an example.

module InitArgs =

  let toUrl initArgs =
    match initArgs with
    | HomeInit () ->
      "#/home"
    | CourseDetailInit { CourseId = courseId } ->
      sprintf "#/course/%i" courseId

  // using Elmish.UrlParser
  let parseInitArgs state =
    oneOf [
      map HomeInit (s "home" </> unitArg)
      map (fun courseId ->
               CourseDetailInit { CourseId = courseId })
          (s "course" </> int)
    ] state

  let fromUrl location =
    parseHash parseInitArgs location

toUrl is mainly for your pages to generate navigable links to other pages. It is optimal to change pages with links when possible, so that the browser's back/forward buttons work as the user expects. Sometimes it is also desirable to manually update the URL from your app when the page options change significantly so that this URL gets added to history.

fromUrl is used in navigation events to pick the page to display. The function will also get used on app init, in case the user came to your app from a bookmark at a specific page. All that remains is to wire up those cases. MVU libraries have built-in integration for this. It can be done manually as well. Pass the location into app init to handle it there. And add an event listener for location changes, tag it in a Msg case like LocationChanged and dispatch it. Then handle it as another case of update where you call fromUrl and switchPage.

Side effects

A main emphasis of functional programming is pure functions. The complement of this emphasis is that side effects become an explicit concept. In MVU, explicit side effects are represented by Cmd.

However Cmd implementations have some less-than-ideal characteristics. In Elm they were very painful to extend with your own functionality (ports). In F# Elmish, Cmd simply defines a function signature which enables you to do whatever you want and dispatch Msgs to report the results back. This is about the most extensible you can get.

However I saw some opportunities for improvement. First it seemed like a mix of concerns to have update potentially creating a side effect function. Secondly it left update less testable than it could be. You can test the model easily, but it is quite invasive to test whether the correct side effects were requested. The ideal would be for Cmd to be just data representing the side effect and its necessary arguments.

Effect

I called the type Effect. It is entirely user-defined. I did not want to call it Cmd since Elmish already uses this name.

type Effect =
  | GetCurrentDateTime
  | SendApiRequest of ApiRequest

The update function returns an Effect list instead of Cmd.

let update msg model : Model * Effect list =
  match model.Page, msg with
  | Home pageData, HomeMsg pageMsg ->
    let pageData, effects = Home.update pageMsg pageData
    { model with Page = Home pageData }, effects

This completely decouples side effects from the update function. Effect is just data and is not tied to any implementation. It also makes it possible for Effect to be value equatable. Which means you can simply setup some expected data and use = to test that actual output matches, just like with Model. No mocks, stubs, or anything invasive at all. So now when you test update you can not only test model changes, but also whether the correct Effects are triggered.

It is sometimes necessary to work with unpredictable reference data like JS Files inside Effect. A match statement can be used to specially check those cases and use value equality for the rest.

The last piece is to code up the side effect implementation. I call the function for this perform.

let perform config effect dispatch =
  match effect with
  | GetCurrentDateTime ->
    dispatch (CurrentDateTimeIs System.DateTime.Now)
  | SendApiRequest request ->
    async {
      // http request details elided
      dispatch (ApiResponseIs result)
    } |> Async.StartImmediate

For Elmish, this new Effect/perform pattern is pretty easy to integrate. It just requires a couple of helper functions.

let init arg =
  let (model, effects) = App.init arg
  (model, effects |> List.map (App.perform model.EffectConfig))

let update msg model =
  let (model, effects) = App.update msg model
  (model, effects |> List.map (App.perform model.EffectConfig))

Program.mkProgram init update App.view
|> Program.withReactBatched "elmish-app"
#if DEBUG
|> Program.withConsoleTrace
#endif
|> Program.run

Here, EffectConfig is where I keep data needed by side effects. For example: API URLs, local storage keys, auth endpoints, etc. These are usually settings found in a JS file deployed with the app. They are grabbed just before the program starts and passed in as the init argument.

Practical issues

My web apps so far have a small number of highly reusable effects, so I tend to make Effect just a flat universal type that any page can use. In larger apps, it can be annoying to get the return Msg back to the specific page that requested it. What I presented above has perform sending responses back with non-page-specific Msgs. So you'd have to add a match for this message to each page that might receive one of these responses.

An alternative approach that I later found my devs using is to provide a return tag with the Effect. This tag is a function which takes the return value and wraps it in a Msg. The Msg would target the specific page that requested it. This hinders the testability of Effects, since you now have to specially treat most of the Effect cases to ignore the tag function and only test the equatable data. But it works since we only have a dozen or less effects, and a handful that use return tags.

You could also avoid this by customizing the available Effects per page. But we haven't found this worth the extra code.

To be perfectly honest, we do not test update currently. In our years of working with MVU there has not been a strong need. Yes we have broken things, but with the other mentioned principles they have been easy to spot and fix quickly. Yet we still wanted the ability to easily test update in case we need to introduce dynamically generated UIs.

The way we handle side effects is my favorite adaptation to the original MVU pattern. Because over the years I have observed that Separation of Concerns is the most important principle to maintainable software. And effect-as-data provides this via loose coupling between logic and side effect implementations.

Project organization tricks

Note on Model

Originally TEA had Msg, Model, init, and update inside the page module. We had to move Msg and the newly invented InitArgs out of the page module and fully define these types for all the pages before the page function definitions.

We can technically keep a page's Model with its functions. But then Model would be the only type that we defined there. So we ended up just moving it up as well. In essence, we split the page's related parts into PageTypes (defined first) and PageFns.

A typical project file structure would look like this.

App project

• PageTypes
  • Home.fs - Home's Msg, InitArgs, Model
  • Course
    • Search.fs
    • Detail.fs
• AppTypes.fs - App's Msg, InitArgs, Model
• InitArgs.fs - has toUrl, fromUrl
• PageFns
  • Home.fs - Home's init, update, view
  • Course
    • Search.fs
    • Detail.fs
• App.fs - App's init, update, view, perform
• Main.fs - Elmish Program wireup

F# has a single-pass compiler. And file order is compile order. So this organization structure is almost like we are creating our own two-pass compiler where types are compiled first, then the functions that use them. If F# had a 2-pass compiler -- types then functions -- it might be possible to keep all the page parts organized in a single module.

VS 2019 bug

The project structure would look like above. Except Visual Studio has a long-standing F#-specific bug. When you have two file paths that have the same subfolder as part of the path, intellisense glitches out. In the example above PageTypes and PageFns both contain a Course subfolder. Any file in the PageFns\Course subfolder will have completely broken intellisense as well as any other file defined after it. It also stops displaying the correct file order in the Solution Explorer window.

We simply name the PageFns subfolders with an underscore on the end (PageFns\Course_\Search.fs) to make the path different. The module names are kept the same (no underscore).

Another way around this is to not use folders but instead use multi-dotted files. Example: Course.Search.fs. Of course, this sacrifices the ability to roll-up/hide all the Course files when I am not using them.

I hope this gets priority to fix soon, because it is embarrassing to have to mention it in a post like this one.

Trick: Merging modules

One really nice property of F# is that opening two different namespaces will effectively merge all the types and functions across same-named modules within them. For example.

open PageTypes
open PageFns

type MyMsg = Home.Msg
let myInit = Home.init

// "Home" is a combination of all the stuff from:
// PageTypes\Home.fs
// PageFns\Home.fs

This still allows you to use all the page parts as though they were under one module. This is also how I extend existing types with new functions.

Pain point

The one major pain point with all the tactics I describe above is structural duplication. What I mean by that is Msg, InitArgs, and Model all have the same basic tree structure, but with different types as leaf nodes. To avoid conflicts, I must name them slightly differently.

// AppTypes.fs
type Msg =
  | HomeMsg of Home.Msg
  | CourseSearchMsg of Course.Search.Msg
  | CourseDetailMsg of Course.Detail.Msg

type InitArgs =
  | HomeInit of Home.InitArgs
  | CourseSearchInit of Course.Search.InitArgs
  | CourseDetailInit of Course.Detail.InitArgs

type Page =
  | Home of Home.Model
  | CourseSearch of Course.Search.Model
  | CourseDetail of Course.Detail.Model

Anyone familiar with Haskell is probably jumping up and down, screaming "Higher-Kinded Types". F# doesn't have those.

However I can think of 3 ways to reduce these 3 tree structures to 1 in F#. I will list them (quite subjectively) from least desirable to most.

These are thought experiments. I have not tried them in a real app yet.

3. Generics as fake HKTs

The setup for this is worse than the original solution.

type Area<'home, 'courseSearch, 'courseDetail> =
  | Home of 'home
  | CourseSearch of 'courseSearch
  | CourseDetail of 'courseDetail

type Msg =      Area<Home.Msg,
                     Course.Search.Msg,
                     Course.Detail.Msg>

type InitArgs = Area<Home.InitArgs,
                     Course.Search.InitArgs,
                     Course.Detail.InitArgs>

type Page =     Area<Home.Model,
                     Course.Search.Model,
                     Course.Detail.Model>

Adding a new page is pretty egregious. You have to touch Area in 2 places, then add a line to each of Msg, InitArgs, and Page.

The only improvement with this approach is that the update function is marginally nicer than the original solution.

let update msg model =
  match model.Page, msg with
  | Home pageData, Home msgData ->
    ...
  | CourseSearch pageData, CourseSearch msgData ->
    ...

Some people like this approach, but overall I think this has more losses than gains.

2. Union leaf data

This one is also more work to setup initially.

type Leaf<'msg, 'init, 'page> =
  | Msg of 'msg
  | Init of 'init
  | Page of 'page

type Area =
  | Home of         Leaf<Home.Msg,
                         Home.InitArgs,
                         Home.Model>
  | CourseSearch of Leaf<Course.Search.Msg,
                         Course.Search.InitArgs,
                         Course.Search.Model>
  | CourseDetail of Leaf<Course.Detail.Msg,
                         Course.Detail.InitArgs,
                         Course.Detail.Model>

Adding a new page is a slight improvement over the original solution. It is typing a few lines in one place instead of a new line in a few places.

The update function is slightly more verbose than the original solution.

let update msg model =
  match model.Page, msg with
  | Home (Page pageData), Home (Msg pageMsg) ->
    ...
  | CourseDetail (Page pageData), CourseDetail (Msg pageMsg) ->
    ...

The weird part of this approach is that model.Page and msg have exactly the same type. That means you can accidentally swap them and not get a compiler error. Then the UI won't work and there is no obvious reason why. Although, it should be possible to add some match cases to detect this and log a warning at runtime.

Overall I'd prefer this over #3, but it is still not ideal.

1. Only Msg

The last way I can think of to get one tree structure involves just eliminating InitArgs and Page, leaving us with only Msg.

Bye InitArgs

Getting rid of InitArgs is pretty far-reaching, but also has other benefits. Essentially the init function is removed and its behavior is put in a new case of update function. Then the InitArgs for that page becomes a Msg case. We will have to add one thing, a "zero" or empty model.

// PageTypes\Course\Detail.fs
namespace PageTypes.Course

module Detail =

  type Msg =
    | Init of courseId: Guid
    | CourseLoaded of Result<Course, QueryError>

  type Model =
    { CourseId: Guid
      Data: Remote<Course> }

  let empty =
    { CourseId = Guid.empty
      Data = Loading }

// PageFns\Course\Detail.fs
namespace PageFns.Course

module Detail =

  let update msg model =
    match msg with
    | Init courseId ->
      { model with CourseId = courseId }
      , [ ApiRequest (GetCourse courseId) ]
    | CourseLoaded (Ok course) ->
      { model with Data = Loaded course }, []
    // and so on

  let view model dispatch =
    // stuff

We have to do special handling of the Init message in the main update. We are using this kind of code instead of the switchPage function from before.

// App.fs
let update msg model =
  match model.Page, msg with
  | _, CourseDetail ((Course.Detail.Init _) as pageMsg) ->
    let pageData, effects = Course.Detail.update pageMsg Course.Detail.empty
    { model with Page = CourseDetail pageData }, effects

I used a similar approach when I built a Clojure MVU library.

Moving Page

UI state like Page is a bit different from Msg or InitArgs. The details of it are only used by the page functions. Outside parties do not need to know the page Model's contents. Previously we set it up as a public tree type for consistency with how we are handling other types. But there is another way. We can use a marker interface.

A marker interface is just an empty interface. Any type can just say it implements a marker interface, since there is nothing to implement. Marker interfaces can be looked at as a slightly different union type. You do not have to define all cases up front... a type locally chooses to opt in to the marker interface. But you also do not get compiler guarantees of complete matches. It is another expression problem trade-off toward extensibility rather than completeness.

// Types.fs
type IPage = interface end

// PageTypes\Course\Detail.fs
namespace PageTypes.Course

module Detail =

  type Model =
    { CourseId: Guid
      Data: Remote<Course> }
    interface IPage

// AppTypes.fs
type Model =
  { EffectConfig: EffectConfig
    Page: IPage }

// App.fs
let update msg model =
  match model.Page, msg with
  | :? Home.Model as pageData, Home pageMsg ->
    ...

We can essentially "tag" each page model as IPage. And any IPage can be stored in the Model.Page property. Unwrapping it to a specific page's model is a few more keystrokes. But there is no central Page union type to maintain anymore.

I have not tried this beyond checking that it would compile in a Fable Elmish project.

End result

At this point, we have eliminated all the duplicate tree structured types and are left with only Msg. We have also eliminated the init function. So the page logic is centralized in update.

Conclusion

Over the last 4 years I have made MVU my playground and learned quite a bit. It is a surprisingly resilient and flexible pattern for organizing UI applications. Because of its state-machine-like qualities, we even use it server-side. Hopefully some of the adaptations I discovered have been interesting to you.

This post has a lot of code snippets, but it could use a companion repo with more complete examples. With my ADHD, an effort like that will never go anywhere if left up to me. So if something like that is of interest to you, let me know. I would be more than happy to contribute.

cover image from unDraw

Addendum

I tried the #1 Only Msg approach and faced some challenges.

Moving InitArgs into Msg and removing init

Moving InitArgs into Msg

One thing I didn't cover was how this affects navigation. Instead of converting InitArgs to/from URLs, this now means converting Msg to/from URLs. The data to create a URL needs to be dug out of the specific page messages. And there will need to be a catch-all match because only a small amount of messages correspond to URLs.

match msg with
| CourseDetail (Course.Detail.Init initArgs) ->
  "#/course/" + string initArgs.CourseId
| _ ->
  defaultUrl

Alternative InitArgs tactic

It has become our standard practice to create an InitArgs record for each page. This record is a container for all the necessary parameters to the init function. And these page InitArgs need to be app-wide like page Msgs. So any page may use them to construct links to other pages.

We could instead use tupled values or anonymous records for a page's init args.

type InitArgs =
  //| CourseSearchInit of Course.Search.InitArgs
  | CourseSearchInit of search: string * page: int * pageSize: int

We then avoid having to define and organize those page InitArgs records. But some duplication of the parameter definitions may be necessary. For example in init if the type would be ambiguous in its usage.

let init (search, page, pageSize) =
  // ambiguous type error, requires annotating search as string
  let normalizedSearch = search.Trim()
  ...

And of course there are the same tradeoffs of using tuples vs records. When creating a new value, records are more verbose but easier to understand because the values are labeled.

Using tuples versus organizing all the page's InitArgs types to be app-wide, I do not think there is a clear winner. So you just have to weigh the tradeoffs on a case by case basis. It is probably easier to start with tuples if unsure.

Removing init

When removing init, a disadvantage is sometimes empty values can be a pain to construct. Like arbitrarily nested records. (I mainly encountered this when trying to remove the overall app init.) So using an explicit init function can feel more natural in those cases.

Overall

This does not feel like a worthwhile change. InitArgs is an additional app-wide type, but it has its own specific usefulness -- converting to/from URLs. And although init usually feels like a special case of update, it can sometimes be useful.

Moving Page

I mentioned using the IPage marker interface to represent pages instead of a central DU. But because F# does not auto-upcast, the main update has to explicitly upcast every page model to IPage. That's arguably worse than just tagging the page-specific model to go in a central DU.

One of the big benefits of using a marker interface is keeping the page Model with its init and update functions. And this is still possible when using a DU. Simply define the central Page DU in the main app file with the app's init and update (versus declaring it app-wide like Msgs). We used to organize it this way, but when I moved the other types (Msg, InitArgs) out of the pages I dragged Model with them. So I over-organized that.

Closing thoughts

Turns out that some of the annoying duplication I mentioned in the main article adds more value than it costs. And there are still minor tweaks available to tone down the annoyance. Without fundamentally deviating from the standard MVU structures.