Chapter 14 The reactive graph

14.1 Introduction

To understand reactive computation you must first understand the reactive graph. In this chapter, we’ll dive in to the details of the graph, paying much more attention to precise order in which things happen. In particular, you’ll learn about the importance of invalidation, which helps ensure that Shiny does the minimum amount of work needed. You’ll also learn about the reactlog package which can automatically draw the reactive graph for you.

Before you read this chapter, I highly recommend that you re-read Chapter 4 if it’s been a while since you looked at it. That chapter lays the ground work for the concepts that we’ll explore in more detail here.

14.2 A step-by-step tour of reactive execution

We’ll begin with a thorough investigation of the reactive graph. You learned the basics in Section 4.3, but there are a few important details I skipped so that you could get the basic idea before learning all the complexities. To explain the process of reactive execution, we’ll use the graphic shown in Figure 14.1. It contains three reactive inputs, three reactive expressions, and three outputs37. Recall that reactive inputs and expressions are reactive producers; and reactive expressions and outputs are reactive consumers.

Complete reactive graph of an imaginary app containing three inputs, three reactive expressions, and three outputs.

Figure 14.1: Complete reactive graph of an imaginary app containing three inputs, three reactive expressions, and three outputs.

The connections between the components are directional, with the arrows indicating the direction of reactivity. The direction might surprise you, as it’s easy to think of a consumer taking a dependencies on one or more producers, but we’ll see shortly that the flow of reactivity is more accurately modelled as flowing in the opposite direction.

The underlying app is not important, but if you want something concrete, you could pretend that it was derived from this not-very-useful app.

ui <- fluidPage(
  numericInput("a", "a", value = 1),
  numericInput("b", "b", value = 1),
  numericInput("c", "c", value = 1),
  textOutput("x"),
  textOutput("y"),
  textOutput("z")
)

server <- function(input, output, session) {
  two_a <- reactive(input$a * 2)
  two_a_b <- reactive(two_a() + input$b)
  b_c <- reactive(input$b * input$c)
  
  output$x <- renderText(two_a_b())
  output$y <- renderText(two_a_b())
  output$z <- renderText(b_c())
}

14.2.1 A session begins

Figure 14.1 shows the final state of the app — all reactive relationships have been discovered, and all the computation has complete (illustrated using green). Now we’ll work step by step to see how we get to this point from loading a Shiny app. We begin with Figure 14.2, right after the app has started and the server function has been executed for the first time. There are no connections because Shiny has no a priori knowledge of the relationships, and all the reactive consumers are in their starting state, invalidated. An invalidated reactive expression or output has yet to be run, and is coloured grey. The reactive inputs are green, indicating that their values are available.

Initial state after app load. There are no connections between objects and all reactive expressions are invalidated (grey).

Figure 14.2: Initial state after app load. There are no connections between objects and all reactive expressions are invalidated (grey).

14.2.2 Execution begins

Now we start the execution phase, as shown in Figure 14.3. In this phase, Shiny picks an invalidated output and starts executing it.

Next Shiny starts executing an arbitrary observer/output, coloured orange.

Figure 14.3: Next Shiny starts executing an arbitrary observer/output, coloured orange.

You might wonder how Shiny decides which of the invalidated outputs to execute. In short, you should act as if it’s random: your observers and outputs shouldn’t care what order they execute in, because they’ve been designed to function independently38.

14.2.3 Reading a reactive expression

During an output’s execution, it may read from one or more reactive producers, as in Figure 14.4.

The output needs the value of a reactive expression, so it starts executing.

Figure 14.4: The output needs the value of a reactive expression, so it starts executing.

Two things happen here:

  • The reactive expression needs to start computing its value, i.e. it turns orange. Note that the output is still orange: just because the reactive expression is now running, it doesn’t mean that the output has finished. The output is waiting on the reactive expression to return its value so its own execution can continue, just like a regular function call in R.

  • Shiny records a relationship between the output and reactive expression, i.e. we draw an arrow. The direction of the arrow is important: the expression that records that it is used by the output. We’ll come back to the details of this in Section 14.5.

14.2.4 Reading an input

This particular reactive expression happens to read a reactive input. Again, a dependency/dependent relationship is established, so in Figure 14.5 we add another arrow. Unlike reactive expressions and outputs, reactive inputs have nothing to execute, since they just represent a value, so they can return immediately.

The reactive expression also reads from a reactive value, so we add another arrow.

Figure 14.5: The reactive expression also reads from a reactive value, so we add another arrow.

14.2.5 Reactive expression completes

In our example, the reactive expression reads another reactive expression, which in turn reads another input. We’ll skip over the blow-by-blow description of those steps, since they’re just a repeat of what we’ve already described, and jump directly to Figure 14.6.

The reactive expression has finished computing so turns green.

Figure 14.6: The reactive expression has finished computing so turns green.

Now that the reactive expression has finished executing it turns green to indicate that it’s up-to-date. It caches its result before returning it to the output that requested it; this minimises the amount of future work it needs to do if its dependencies are unchanged.

14.2.6 Output completes

Now that the reactive expression has returned its value, the output can finish executing, and change colour to green, as in Figure 14.7.

The output has finished computation and turns green.

Figure 14.7: The output has finished computation and turns green.

14.2.7 The next output executes

Now that Shiny has executed the first output, it chooses another one to execute. This output turns turns orange, Figure 14.8, and starts reading values from reactive producers. Complete reactives can return their values immediately; invalidated reactives will kick of their own execution graph. This cycle will repeat until every invalidated output enters the complete (green) state.

The next output starts computing, turning orange.

Figure 14.8: The next output starts computing, turning orange.

14.2.8 Execution completes, outputs flushed

Now in Figure 14.9, all of the outputs have finished execution and are now idle. This round of reactive execution is complete, and no more work will occur until some external force acts on the system (e.g. the user of the Shiny app moving a slider in the user interface). In reactive terms, this session is now at rest.

All output and reactive expressions have finished and turned green.

Figure 14.9: All output and reactive expressions have finished and turned green.

Let’s stop here for just a moment and think about what we’ve done. We’ve read some inputs, calculated some values, and generated some outputs. But more importantly, in the course of doing that work, we also discovered the relationships between the reactive objects. Now when a reactive input changes we know to invalidate reactive expressions and re-run outputs.

Just as importantly we also know which nodes are not dependent on each other: if no path exists from a reactive input to a output, then changing the input can’t affect the output. That allows Shiny to do the minimum amount of re-computation when an input changes.

14.2.9 An input changes

The previous step left off with our Shiny session in a fully idle state. Now imagine that the user of the application changes the value of a slider. This causes the browser to send a message to the server function, instructing Shiny to update the corresponding reactive input.

When a reactive input or value is modified, it kicks off an invalidation phase. The invalidation phase starts at the changed input/value, which in Figure 14.10 we’ll fill with grey, our usual colour for invalidation.

The user interacts with the app, invalidating an input.

Figure 14.10: The user interacts with the app, invalidating an input.

14.2.10 Notifying dependencies

Now, we follow the arrows that we drew earlier. Each reactive consumer that we find is put into invalidated state, and we keep following the arrows until there’s nothing left. The results of this process are shown in Figure 14.11, with the arrows that Shiny has followed in lighter grey.

Invalidation flows out from the input, following every arrow from left to right.

Figure 14.11: Invalidation flows out from the input, following every arrow from left to right.

14.2.11 Removing relationships

Next, each invalidated reactive expression and output “erases” all of the arrows coming in to or out of it, yielding Figure 14.12. You can think of each arrow as a one-shot notification that will fire the next time a value changes.

Invalidated nodes forget all their previous relationships so they can be discovered afresh

Figure 14.12: Invalidated nodes forget all their previous relationships so they can be discovered afresh

It’s less obvious is why we erase the arrows coming in to an invalidated node, even if the node they’re coming from isn’t invalidated. While those arrows represent notifications that haven’t yet fired, the invalidated node no longer cares about them. Reactive consumers only care notifications in order to invalidate themselves, that here that has already happened.

It may seem perverse that we put so much value on those relationships, and now we’ve thrown them away! But this is an important part of Shiny’s reactive programming model: though these particular arrows were important, they are now themselves out of date. The only way to ensure that our graph stays accurate is to erase arrows when they become stale, and let Shiny rediscover the relationships around these nodes as they re-execute.

This marks the end of the invalidation phase.

14.2.12 Re-execution

Now we’re in a pretty similar situation to when we executed the second output, with a mix of valid and invalid reactives. It’s time to do exactly what we did then: execute the invalidated outputs, one at a time, starting off in Figure 14.13.

Now re-execution proceeds in the same way as execution, but since we're not starting from scratch we don't have as much work to do.

Figure 14.13: Now re-execution proceeds in the same way as execution, but since we’re not starting from scratch we don’t have as much work to do.

Again, I won’t show you the details, but the end result will be a reactive graph at rest, with all nodes marked in green. The neat thing about this process is that Shiny has done the minimum amount of work — we’ve only done the work needed to update the outputs that are actually affected by the changed inputs.

14.2.13 Exercises

  • Draw the reactive graph for the following server function and then explain why the reactives are not run.

    server <- function(input, output, session) {
      sum <- reactive(input$x + input$y + input$z)
      prod <- reactive(input$x * input$y * input$z)
      division <- reactive(prod() / sum())
    }

14.3 The reactlog package

Drawing the react graph by hand is a powerful technique to help you understand simple apps and build up an accurate mental model of reactive programming. But would it be great if we could automatically drawn the graph using the state that Shiny maintains internally? It turns out we can, generating the so called reactlog which shows how the reactive graph evolves over time for an app. Here I’ll give you the basics, and then you can learn more on https://rstudio.github.io/reactlog/.

To use the reactlog you’ll need to first install the reactlog package, then turn it by running reactlog::reactlog_enable(), and then start your app. You then have two options to see the reactlog:

  • While the app is running, press Cmd + F3 (Ctrl + F3 on Windows), to show the reactlog generated up to the that point.

  • After the app has closed, run shiny::reactlogShow() to see the log for the complete session.

reactlog uses the same graphical conventions as this chapter, so it should feel instantly familiar. The biggest difference is that reactlog draw every dependency, even if it’s not currently used, in order to keep the automated layout stable. Connections that are not active currently (but were in the past or will be in the future) are drawn as thin dotted lines.

Tips and tricks:

  • If you’re using the react log a lot, learn the keyboard shortcuts: use the left and right arrows to step through one item at a time. Use option

  • Reactive inputs and outputs have names, but reactive()s and observe()rs do not, so they’re labelled with their contents. To make things easier to understand you may want to supply the label argument, which will then appear on the reactive graph. You may want to label particularly important reactives with emoji so that they stand out.

14.4 Dynamism

Consider the following simple app:

ui <- fluidPage(
  selectInput("choice", "A or B?", c("a", "b")),
  numericInput("a", "a", 0),
  numericInput("b", "b", 10),
  textOutput("out")
)

server <- function(input, output, session) {
  output$out <- renderText({
    if (input$choice == "a") {
      input$a
    } else {
      input$b
    }
  }) 
}

You might expect the reactive to look like Figure 14.14.

If Shiny analysed reactivity statically, the reactive graph would always connect `choice`, `a`, and `b` to `out`.

Figure 14.14: If Shiny analysed reactivity statically, the reactive graph would always connect choice, a, and b to out.

But remember that Shiny dynamically reconstructs the graph after the output has been invalidated so it actually looks either of the graphs in Figure 14.15, depending on the value of input$choice. This ensure that Shiny does the minimum amount of work when an input is invalidated. In this, if input$choice is set to “b”, then the value of input$a doesn’t affect the output$out and there’s no need to recompute it.

But Shiny's reactive graph is dynamic, so the graph either connects `out` to `choice` and `a` (left) or `choice` and `b` (right).

Figure 14.15: But Shiny’s reactive graph is dynamic, so the graph either connects out to choice and a (left) or choice and b (right).

14.5 How does it work?

Before we carry on, I wanted to give you a quick sense of how things work behind the scenes. This knowledge isn’t likely to help you build better apps, but it might resolve some lingering questions. Take this simple example:

output$plot <- renderPlot({
  plot(head(cars, input$rows))
})

How does Shiny know that output$plot reads input$rows? Your first guess might be that renderPlot() parses its code looking for looking for references to input. This is a natural guess because it’s how you build up a reactive graph when reading code. Unfortunately, however, this technique is very fragile because even simple change would break things:

output$plot <- renderPlot({
  x <- input
  plot(head(cars, x$rows))
})

In computer science this approach is called static analysis, because it looks at the code without running it (i.e. it’s not moving, it’s static). Shiny instead uses dynamic analysis, where collects additional information about what’s going on as the code is run.

The basic process is something like this. renderPlot() starts by creating a reactive context. A reactive context is a internal object that’s used coordinate reactive consumers and producers. You’ll never see once of these objects as an app author, but they’re a crucial piece of infrastructure behind the scenes. The reactive context is then stored in a special place that’s accessible to other Shiny functions. Then once renderPlot() is done, it restores the previous context, using something like this imaginary code:

# renderPlot() creates and activate new context 
context <- ReactiveContext$new()
old <- setReactiveContext(context)

# Then we run the user code
plot(head(cars, input$rows))

# And finally restore the previous context
setReactiveContext(old)

Now, while a reactive consumer is running, reactive producers can grab the active context with something like getReactiveContext(). Then producers can return the requested value and register the current reactive context. When the reactive producer is later invalidated, it can consult its registry of contexts, and tell the to also invalidate.

So Shiny “magically” establishes the connections between producer and consumer with these two simple mechanisms:

  • Each reactive consumer creates a context object and “activates” it during execution.

  • Each reactive producer augments every read operation by saving the context object so it can be later invalidated.

This process ensures that there’s no way that Shiny can either accidentally overlook a reactive dependency relationship or erroneously establish a dependency that doesn’t exist.


  1. Anywhere you see output, you can also think observer. The only difference is that certain outputs that aren’t visible never be computed.↩︎

  2. If have observers whose side effects must happen in a certain order, you’re generally better off re-designing your system. Failing that, you can control the relative order of observers with the the priority argument to observe().↩︎