15 Reactive building blocks

Now that you’ve learned how the reactive graph really works, it’s useful to come back to some of the details of the underlying objects. By this point, you’ve used these objects many times in your apps, so now is a good time to discuss some of the finer details.

There are three fundamental building blocks of reactive programming: reactive values, reactive expressions, and observers. You’ve already seen most of the important parts of reactive values and expressions, so this chapter will spend most time on observers and the closely related outputs. We’ll also discuss a final piece of the invalidation puzzle: timed invalidation.

This chapter will again use the reactive console so that we can experiment with reactivity directly in the console without having to launch a Shiny app each time.

15.1 Reactive values

There are two types of reactive values:

They have a slightly different syntax for getting and setting values:

x <- reactiveVal(10)
x()       # get
#> [1] 10
x(20)     # set

y <- reactiveValues(a = 1, b = 2)
y$a       # get
#> [1] 1
y$b <- 20 # set

It’s unfortunate that there are two different syntaxes, but there’s no way to standardise them given the constraints imposed by R as a programming language. Fortunately there are no major differences in their behaviour, so you can choose which syntax you prefer. In this book, I use reactiveVal() because I like that the syntax for setting values makes it more obvious that something unusual is going on.

It’s important to note that both types of reactive values have so called reference semantics. Most R objects have copy-on-modify41 semantics which means that if you assign the same value to two names, the connection is broken as soon as you modify one:

a1 <- a2 <- 10
a2 <- 20
a1 # unchanged
#> [1] 10

This is not the case with reactive values — they always keep a reference back to the same value so that modifying any copy modifies all values:

b1 <- b2 <- reactiveVal(10)
b2(20)
b1()
#> [1] 20

The most common source of reactive values is the input argument passed to the server function. This is a special read-only reactiveValues(): you’re not allowed to modify the values because Shiny automatically updates them based on the users actions in the browser.

ui <- fluidPage(
  textInput("name", "name")
)
server <- function(input, output, session) {
  input$name <- "Hadley"
}
shinyApp(ui, server)
#> Error: Can't modify read-only reactive value 'Hadley'.

15.1.1 Exercises

  1. What are the differences between these two lists of reactive values?

    l1 <- reactiveValues(a = 1, b = 2)
    l2 <- list(a = reactiveVal(1), b = reactiveVal(2))
  2. Design and perform a small experiment to verify that reactiveValues() have reference semantics.

15.2 Reactive expressions

Recall that a reactive has two important properties: it is lazy and cached. This means that it does as little work as possible by only doing any work if it’s actually needed, and when it’s called twice in a row, returning the previous value. There are two important details that we haven’t yet covered: what reactive expressions do with errors, and that you can use on.exit() inside a reactive.

15.2.1 Errors

Reactive expressions cache errors in exactly the same way that they cache values. For example, take this reactive:

r <- reactive(stop("Error occured at ", Sys.time(), call. = FALSE))
r()
#> Error: Error occured at 2020-11-27 23:15:16

If we wait a second or two, we can see that we get the same error as before:

Sys.sleep(2)
r()
#> Error: Error occured at 2020-11-27 23:15:16

Errors are also treated the same way as values when it comes to the reactive graph: errors propagate through the reactive graph exactly the same way as regular values. The only difference is what happens when an error hits an output or observer:

  • An error in an output will be displayed in the app42.
  • An error in an observer will cause the current session to close. (If you don’t want this to happen you can wrap the code in try() or tryCatch().)

This same system powers req(), which emits a special type of error43. The error emitted by req() causes observers and outputs to stop what they’re doing but not otherwise fail. By default, it will cause outputs to reset to their initial blank state, but if you use req(..., cancelOutput = TRUE) they’ll preserve their current display.

15.2.2 on.exit()

You can think of reactive(x()) as a shortcut for function() x(), automatically adding laziness and caching. This means that you can use functions that only work inside functions. The most useful of these is on.exit() which allows you to run code when a reactive expression finishes, regardless of whether the reactive successfully returns an error or fails with an error. You can learn more about on.exit() at https://withr.r-lib.org/articles/changing-and-restoring-state.html.

15.3 Observers

An observer sets up a block of code that is run every time one of the reactive values or expressions it uses is updated:

y <- reactiveVal(10)
observe({
  message("`y` is ", y())
})
#> `y` is 10

y(5)
#> `y` is 5
y(4)
#> `y` is 4

Note that the observer runs immediately when you create it — it must do this in order to determine it’s reactive dependencies.

Observers differ from expressions in two important ways:

  • The value returned by an observer is ignored because they are designed to work with functions called for their side-effects, like cat() or write.csv().

  • They are eager and forgetful — they run as soon as they possibly can and they don’t remember their previous action. This eagerness is “infectious” because if they use a reactive expression, that reactive expression will also be evaluated.

15.3.1 Nesting observers

It’s important to think of observer() not as doing something, but as creating something (which then takes action as needed). In other words, you might want to think of observe() as newObserver(). That mindset helps you to understand what’s going on in this example:

x <- reactiveVal(1)
y <- observe({
  x()
  observe(print(x()))
})
#> [1] 1
x(2)
#> [1] 2
#> [1] 2
x(3)
#> [1] 3
#> [1] 3
#> [1] 3

Every time the observer is triggered, it creates another observer, so each time x changes, it’s value is printed one more time. As a general rule, you should only ever create observers or outputs at the top-level of your server function. If you find yourself needing to nest them, sit down and sketch out the reactive graph that you’re trying to create.

It can be harder to spot this mistake directly in a more complex app, but you can always use the reactlog: just look for unexpected churn in observers (or outputs), then track back to what is creating them.

15.3.2 isolate()

Observers are often coupled with reactive values in order to track state changes over time. But if you’re not careful, you quickly hit a problem. For example, take this code which tracks how many times x changes:

count <- reactiveVal(0)
x <- reactiveVal(1)
observe({
  x()
  count(count() + 1)
})

If you were to run it, you’d immediately get stuck in an infinite loop because the observer will take a reactive dependency on x and count; and since it modifies count, it will immediately re-run.

Fortunately, Shiny provides a function to resolve this problem: isolate(). This function allows you to access the current value of a reactive value or expression without taking a dependency on it:

count <- reactiveVal(0)
x <- reactiveVal(1)
observe({
  x()
  isolate(count(count() + 1))
})

x(1)
x(2)
count()
#> [1] 2
x(3)
count()
#> [1] 3

(isolate() also works inside of reactive(), but there are very few situations in which this is actually useful.)

15.3.3 observeEvent()

Another way to avoid this problem is to use the observeEvent() function that you learned about in Section 4.6. It decouples listening for an event from the action to handle that event, so you could rewrite the above code as:

observeEvent(x(), {
  count(count() + 1))
})

In Chapter 16 we’ll come back to the problem of managing state over time in much more detail.

observeEvent() has an number of additional arguments that allow you to control its details:

  • By default, observeEvent() will ignore any event that yields NULL (or in the special case of action buttons, 0). Use ignoreNULL = FALSE to also handle NULL values.

  • By default, observeEvent() runs once when you create it. Use ignoreInit = TRUE to skip this run.

  • Use once = TRUE to run the handler only once.

15.3.4 Output

If you’ve paid close attention you might have noticed that I’ve explained input, reactive expressions, and observers, but I haven’t said anything about output. How do reactive outputs fit into the picture? Take this simple output:

output$text <- renderText(paste("Hi ", input$name, "!"))

It looks like a reactive expression because we’re assigning the result of renderText(), but it feels like a observer, because it has a side-effect: updating the contents of a text box in the browser. It turns out that outputs are a special type of observer. They have two special properties:

  • They are defined when you assign them into output, i.e. output$text <- ... creates the observer.

  • They have some limited ability to detect when they’re not visible (i.e. they’re in non-active tab) they don’t recompute44.

Outputs are intimately connected to the browser, so there’s no way to experiment with them on the console.

15.4 Timed invalidation

There’s one final building blocking that we need to discuss. It’s a little different to the others we’ve discussed in this chapter because it’s not a type of object, but it’s instead a special behaviour: timed invalidation. You saw an example of this in Section 4.5.1 with reactiveTimer(), but the time has come to discuss the underlying tool that powers it: invalidateLater().

invalidateLater() causes any reactive consumer to be invalidated in the future, after a set number of milliseconds specified by the first argument. It is useful for creating animations and connecting to data sources outside of Shiny’s reactive framework that may be changing over time. For example, this reactive will automatically generate 10 fresh random normals every half a second45:

x <- reactive({
  invalidateLater(500)
  rnorm(10)
})

And this observer will increment a cumulative sum with a random number:

sum <- reactiveVal(0)
observe({
  invalidateLater(300)
  sum(isolate(sum()) + runif(1))
})

15.4.1 Polling

This reactive will re-read a csv file every second:

data <- reactive({
  on.exit(invalidateLater(1000))
  read.csv("data.csv")
})

But it has a serious downside: when you invalidate the reactive, you’re also invalidating all downstream reactives, so even if the data is the same, all downstream work has to be redone. To avoid this problem Shiny provides reactivePoll() which allows you to specify two functions: one that performs a relatively cheap check to see if the data has changed and another more expensive function that actually does the computation.

We can use reactivePoll() to rewrite the previous reactive as follows. We use file.mtime(), which returns the last time the file was modified, as cheap check to see if we need to reload the file.

server <- function(input, output, session) {
  data <- reactivePoll(1000, session, 
    function() file.mtime("data.csv"),
    function() read.csv("data.csv")
  )
}

Reading a file when it changes is a common task, so Shiny provides an even more specific helper that just needs a file name and a reading function:

server <- function(input, output, session) {
  data <- reactiveFileReader(1000, session, "data.csv", read.csv)
}

15.4.2 Long running reactives

If you’re performing a long running computation, there’s an important question you need to be aware of: when should you execute invalidateLater()? For example, take this reactive:

x <- reactive({
  invalidateLater(500)
  Sys.sleep(1)
  10
})

Assume Shiny starts it running at time zero, so it requests invalidation at time 500. The reactive takes 1000ms to run, so it’s now time 1000, and it gets invalidated. If an observer or output needs this value, we’ll need to immediately recompute, which again invalidates midway through the computation, and we get stuck in an infinite loop.

On other hand, if you run invalidateLater() at the end, it will invalidate 500ms after completion, so the reactive will be re-run every 1500 ms.

x <- reactive({
  on.exit(invalidateLater(500), add = TRUE)
  Sys.sleep(1)
  10
})

This is the main reason to prefer invalidateLater() to the simpler reactiveTimer() that we used earlier: it gives you greater control over exactly when the invalidation occurs.

15.4.3 Timer accuracy

The number of milliseconds specified in invalidateLater() is polite request, not a demand. R may be doing other things when you asked for invalidation to occur, so it has to wait. This effectively means that the number is a minimum — Shiny will do its best to run when you asked, but it might have other things going on so your code will run later than you expect.

In most cases, this doesn’t matter because small delays here and there are unlikely to affect the user’s perception of your app. However, in situations where many small errors will accumulate, you should record the exact elapsed time so you can adjust your calculations. For example, the following code computes distance based on velocity and elapsed time. Rather than assuming invalidateLater(100) always delays by exactly 100 ms, I compute the elapsed time and use it in my calculation of position.

velocity <- 3
distance <- reactiveVal(1)

last <- proc.time()[[3]]
observe({
  cur <- proc.time()[[3]]
  time <- last - cur
  last <<- cur
  
  distance(isolate(distance()) + velocity * time)
  invalidateLater(100)
})

15.4.4 Exercises

  1. Why will this reactive never be executed? Your explanation should talk about the reactive graph and invalidation.

    server <- function(input, output, session) {
      x <- reactive({
        invalidateLater(500)
        rnorm(10)
      })  
    }

15.5 The reactive graph

Before we carry on, I wanted to give you a quick sense of how things work behind the scenes. This knowledge isn’t likely to help you build better apps, but it might resolve some lingering questions. Take this simple example:

output$plot <- renderPlot({
  plot(head(cars, input$rows))
})

How does Shiny know that output$plot reads input$rows? Your first guess might be that renderPlot() parses its code looking for references to input. This is a natural guess because it’s how you build up a reactive graph when reading code. Unfortunately, however, this technique is very fragile because even simple change would break things:

output$plot <- renderPlot({
  x <- input
  plot(head(cars, x$rows))
})

In computer science this approach is called static analysis, because it looks at the code without running it (i.e. it’s not moving, it’s static). Shiny instead uses dynamic analysis, where it collects additional information about what’s going on as the code is run.

The basic process is something like this. renderPlot() starts by creating a reactive context. A reactive context is an internal object that’s used to coordinate reactive consumers and producers. You’ll never see one of these objects as an app author, but they’re a crucial piece of infrastructure behind the scenes. The reactive context is then stored in a special place that’s accessible to other Shiny functions. Then once renderPlot() is done, it restores the previous context, using something like this imaginary code:

# renderPlot() creates and activate new context 
context <- ReactiveContext$new()
old <- setReactiveContext(context)

# Then we run the user code
plot(head(cars, input$rows))

# And finally restore the previous context
setReactiveContext(old)

Now, while a reactive consumer is running, reactive producers can grab the active context with something like getReactiveContext(). Then producers can return the requested value and register the current reactive context. When the reactive producer is later invalidated, it can consult its registry of contexts, and tell each of them to also invalidate.

So Shiny “magically” establishes the connections between producer and consumer with these two simple mechanisms:

  • Each reactive consumer creates a context object and “activates” it during execution.

  • Each reactive producer augments every read operation by saving the context object so it can be later invalidated.

This process ensures that there’s no way that Shiny can either accidentally overlook a reactive dependency relationship or erroneously establish a dependency that doesn’t exist.