Architecture for Non-Trivial R Shiny Applications

Introduction

Idea for battery came from three major libraries and frameworks in JavaScript ecosystem. Mainly Angular, React and Vue. This ecosystem is probably the biggest in the world,
where most software is Open Source.

Those three major libraries and frameworks are all created with components and events architecture. Also browsers are adding their own system of Web Components and events are built-in. This can teach us that this is very good software pattern. Especially for Web Applications.

Battery borrowed ideas mostly from AngularJS (previous now defunct version of Angular). But similar system
exists in React and Vue.

Our First Application

I you want to create non trivial Shiny App. You need to create some kind of architecture. In our first application when I worker and Roche and Genentech, we created event based architecture but the code don't have much structure.
The code was splitted into component files but the content of those files were just global functions. In server.R we had .events environment object that had reactive values
and in whole code base there were observeEvent calls and the events were invoked by adding values to properties of .events environment. To force update we sometimes used boolean value for the event or list with a timestamp.

In our application, we were trying to use shiny modules, but they were not working for us.

Making a Proper Architecture

After a while, when we had workshop, we decided that we need to do something with this mess. We had a task to create better solution. I was the only one that came with the solution. I created basic PoC, I've called my library battery. Because it was created using R6 class library.
R6 is also symbol for batteries named also AA. The main idea for the library came from the front-end frameworks like AngularJS and ReactJS. Where you can components and events that propagate through the tree of components. First version had two events systems one taken from shiny and one internal
that allow to send messages with meaningful information and share state.

First version of the Battery framework, was used to rewrite part of the application. Our application had main tabs and we rewrite single tab. It was working very nice. Around that time we were starting new application. This time we used Battery to create architecture of our app. And it was looking
much more clean. After you learn how the framework works, you knew better where things in source code were located, by intuition and adding features and fixing bugs were much easier.

The Battery Framework

The base thing in Battery architecture framework is a component. You create components using similar code to R6Class.

Button <- battery::component(
    classname = 'Button',
    public = list(
        label = NULL,
        count = NULL,
        constructor = function(label) {
            self$label <- label
            self$connect('click', self$ns('button'))
            self$count <- 0
            self$on('click', function() {
                self$count <- self$count + 1
            })
            self$output[[self$ns('button')]] <- shiny::renderUI({
                self$events$click
                shiny::tags$span(paste("count", self$count))
            })
        },
        render = function() {
            tags$div(
                class = 'button-component',
                tags$p(class = 'buton-label', self$label),
                shiny::uiOutput(self$ns('button'))
            )
        }
    )
)

Above code is basic example of component, you usually don't want this type of detail, component should be something bigger then single widget. If you have panel with few buttons, drop downs and labels, this is usually best candidate for single component. Adding component for single button is probably overkill but of course you can use Battery this way.

You may notice that battery constructor is named differently then R6 class initialize, the reason for this was that there were that R6 initialize have lot of code and it will break in very cryptic errors when you forget to call super$initialize(...).

Creating Proper Tree

If you want you application to work properly, and that events propagation function as it should. You should create proper tree of components. You have two options for this.

Panel <- battery::component(
  classname = 'Panel',
  public = list(
    title = NULL,
    constructor = function(title) {
      self$title <- title

      ## first way to create proper tree
      Button$new(label = 'click Me', component.name = 'btn1', parent = self)

      ## second way using appendChild directly
      btn <- Button$new(label = 'click Me Too', parent = self)
      self$appendChild("btn2", btn)

      self$output[[self$ns('a')]] <- shiny::renderUI({
          self$children$btn1$render()
      })
      self$output[[self$ns('b')]] <- shiny::renderUI({
          btn$render()
      })
    },
    render = function() {
      tags$div(
        tags$h2(self$title),
        tags$div(shiny::uiOutput(self$ns('a'))),
        tags$div(shiny::uiOutput(self$ns('b')))
      )
    }
  )
)

To create tree of components that don't render when it don't have to. It's good to use this pattern. Use single output for renderUI for every child component if components use reactiveness to rerender itself.

Root Component

In your application you should have root component. For instance App that will have all your other components.
When you're adding this component to shiny you should use this code in server.R:

server <- function(input, output, session) {
    app <- App$new(input = input, output = output, session = session)
    output$app <- renderUI({
      app$render()
    })
}

Only root component need to be called with input, output and session arguments. All child components only need parent parameter. You will be able to access those objects using self$.

Namespaces

In Battery to make component system works we need to some how namespace shiny widgets. There method $ns(...) that accept single string and create namespaced name, the output will have class name and counter so each instance of the component will have different ID and there will be no conflicts but in you code you can use pretty names like 'button', you only need to care about single component, to not have two shiny widgets with same name.

Events listeners

To add event listener you use method $on(...) with name of the event. There are two types of events. Internal events that you can trigger from R code. Using $trigger(...) method of using $emit(...) and $broadcast(...). trigger will invoke event on same component, emit will propagate event to its parents and broadcast will propagate the events to its children.

To create internal event, you have three options:

1. Using self$connect('click', self$ns("button")) this will make connection between shiny input events and internal events. If you make connection like this you can use $on(...) to listen to that event.
2. Using self$on(...), with on you're adding event listener for particular event.
3. Using self$createEvent('name'), if you call about two functions createEvent will be called for you.

After you've added internal event you can also use it in reactive context to trigger for instance renderUI.
All internal events created for the component are accessible from self$events$<NAME>.

To add shiny event you have only one option.

self$on(self$ns("button"), function() {

}, input = TRUE)

self$on(self$ns("input"), function(value) {
  print(value)
}, input = TRUE)

What nice about self$on is that it accept two arguments, but they both are optional, first is value and second is target. Target is more important when you use $emit(...) or $broadcast(...) and want to know, which component triggered the event.

Components inheritance

Because components are in fact R6 classes you can use inheritance to create new components based on different ones.

You have two options of inheritance:

1. ExistingComponent$extend

HelloButton <- Button$extend(
  classname = 'HelloButton',
  public = list(
    constructor = function() {
      super$constructor('hello')
      n <- 0
      self$on("click", function(target, value) {
        n <<- n + 1
        value <- paste(self$id, "click", n)
        self$emit("message", value)
      })
    }
  )
)

1. inherit argument to battery::component:

HelloButton <- battery::component(
  classname = 'HelloButton',
  inherit = Button,
  public = list(
    constructor = function() {
      super$constructor('hello')
      n <- 0
      self$on("click", function(target, value) {
        n <<- n + 1
        value <- paste(self$id, "click", n)
        self$emit("message", value)
      })
    }
  )
)

Both API are exactly the same.

Services

They were added, when we realized that, using $broadcast(...) and $emit(...) was not efficient when you
just want to send message to the sibling in component tree.

Services can be any objects (you can use R6 classes) that need to be shared between objects in the tree. Because the can share state it's good idea to use reference object like R6Class.

There is one service object that can be used as example, it's battery::EventEmitter that share similar API as internal Battery events.

You can use it like this:

e <- EventEmitter$new()

e$on("sessionCreated", function(value) {
  print(value$name)
})

e$emit("sessionCreated", list(name = "My Session"))

to add service you can use $addService(...) method:

self$addService("emitter", e)

or when creating instances of the components.

a <- A$new(
    input = input,
    output = output,
    session = session,
    services = list(
        emitter = e
    )
)

Because services are shared by whole component tree, you can only add one service with given name. You can access the service using:

self$service$emitter$emit("hello", list(100))

in any component. This can be used to listen in on component and emit in other in order to send event to siblings or any different object directly, without the need to emit/broadcast the events.

Static values

Static values work similar to self and private in R6 class but it's shared between instances of the component. There is one static per component. You can add initial static value when creating component:

A <- battery::component(
    classname = 'A',
    static = list(
        number = 10
    ),
    public = list(
        inc = function() {
            static$number = static$number + 1
        }
    )
)

a <- A$new()
b <- A$new()

a$inc()

a$static$number === b$static$number

Unit testing

Battery have built in unit testing framework. It use some meta programming tricks that are in R language. So you can easier test your components and reactive value based events. It mocks shiny reactive listeners used by batter so you will not have errors that checking input can only happen in reactive context.

When creating unit tests all you need to do is call

battery::useMocks()

Then you can use:

session <- list()
input <- battery::activeInput()
output <- battery::activeOutput()

to mock input output and session. In most cases session can be list unless you're testing. If you look at the code there is also mock for session (that you can create using battery::Session$new()), but it's used mostly internally to test separation of the battery apps state, when multiple users access shiny app.

At the end of you test you should call

battery::clearMocks()

With input and output mocks you can trigger shiny input change e.g.: using input$foo <- 10. And you can check the output$button and compare it to shiny tags:

expect_equal(
  output$foo,
  tags$div(
    tags$p('foo bar'),
    tags$div(
      id = x$ns('xx'),
      class = 'shiny-html-output',
      tags$p('hello')
    )
  )
)

To see more examples how to test your application check the Battery unit tests.

Debugging Battery Events Using Logger

There is one instance of EventEmitter that is used in battery to add logs to application.

You can use it in your app.

self$log(level, message, type = "battery", ...)

and you can listen to the events using:

self$logger('battery', function(data) {
   if (data$type == "broadcast") {
      print(paste(data$message, data$args$name, data$path))
   }
})

level can be any string, e.g. 'log', args in listener is list of any additional arguments added when calling $log(...).

By default battery is adding its own events using battery and info levels that you can listen to to see how events are invoked in your application. The type parameter is always name of the method. So if you want to check how events are executed you can use this code:

self$logger(c('battery', 'info'), function(data) {
  if (data$type == 'on') {
    print(paste(data$message, data$id, data$args$event))
  }
})

This should be executed as one of the first lines of your root component (usually app).

Spies

If you create your component with spy option set to TRUE it will spy on all the methods. Each time a method s called it will be in component$.calls named list, were each function will have list of argument lists

e.g.

t <- TestingComponent$new(input = input, output = output, session = session, spy = TRUE)

t$foo(10)
t$foo(x = 20)
expect_that(t$.calls$foo, list(list(10), list(x = 20)))

constructor is also on the list of .calls, everything except of functions that are in base component R6 class.

Conclusion

Making proper architecture in environment where there are no architecture framework can be a challenge. But with Battery this may be easier that you think. It may take a while before your team learn the framework but after that initial effort your life will be much simpler, since you will follow the rules and not need to think how to make your architecture work. This is especially important for big shiny apps like we had at Roche and Genentech.

I'm happy that as of August 2022 Battery framework is now Open Source and is available at GitHub Genentech organization.