Chapter 4 Basic reactivity

4.1 Introduction

In the the previous chapter, we talked about creating the user interfaces. Now we’ll move on to discuss the server side of Shiny, where you use R code at runtime to make your user interface come to life!

In Shiny, you express your server logic using reactive programming. Reactive programming is an elegant and powerful programming paradigm, but it can be disorienting at first because it’s a very different paradigm to writing a script. The key idea of reactive programming is to specify a graph of dependencies so that when an input changes, all outputs are automatically updated. This makes the flow of an app considerably simpler, but it takes a while to get your head around how it all fits together.

This chapter will provide a gentle walkthrough of reactive programming, introducing some of the most basic reactive constructs you’ll use in your Shiny apps. We’ll start with a survey of the server function, discussing in more detail how the input and output arguments work. Next we’ll review the simplest form of reactivity (where inputs are directly connected to outputs), and then discuss how reactive expressions allow you to eliminate duplicated work. We’ll finish by reviewing some common roadblocks encountered by newer shiny users.

4.2 The server function

As you’ve seen, the guts of every Shiny app looks like this:

The previous chapter covered the basics of the front-end, the ui object that contains the HTML presented to every user of your app. In this chapter, we’ll focus on the backend, which is more complex, because every user gets an independent version of app. So instead of a single static object like ui, the backend is a function, server(). You’ll never call that function yourself; instead, Shiny invokes it whenever a new session begins.

Before we can continue, we need to precisely define a session, which captures the state of one live instance of a Shiny app. A session begins each time the Shiny app is loaded in a browser, either by different people, or by the same person opening multiple tabs. Each session needs to be completely independent so that when user A moves a slider, outputs update only for user A, not unrelated user B. The server function is called once for each session, creating a private scope that holds the unique state for that user, and every variable created inside the server function is only accessible to that session. This is why almost all of the reactive programming you’ll do in Shiny will be inside the server function5.

Server functions take three parameters: input, output, and session6. Because you never call the server function yourself, you’ll never create these objects yourself. Instead, they’re created by Shiny when the session begins, connecting back to a specific session. For the moment, we’ll focus on the input and output arguments, and leave session for later chapters.

4.2.1 Input

The input argument is a list-like object that contains all the input data sent from the browser, named according to the input ID. For example, if your UI contains a numeric input control with an input ID of count, like so:

Then you can access the value of that input with input$count. It will initially contain the value 100, and it will be automatically updated as the user changes the value in the browser.

Unlike a typical list, input objects are read-only. If you attempt to modify an input inside the server function, you’ll get an error:

This error occurs because input reflects what’s happening in the browser, and the browser is Shiny’s “single source of truth”. If you could modify the value in R, you could introduce inconsistencies where the input slider said one thing in the browser, and input$count said something different in R. That would make programming challenging! Later, in Chapter XYZ, you’ll see how to use functions like updateNumericInput() to modify the value in the browser, and then input$count will update accordingly.

One more important thing about input: it’s selective about who is allowed to read it. To read from an input, you must be in a reactive context created by a function like renderText() or reactive(). We’ll come back to that idea very shortly, but it’s a fundamentally important constraint that allows outputs to automatically update when an input changes. This code illustrates the error you’ll see if you make this mistake:

4.2.2 Output

output is very similar to input: it’s also a list-like object named according to the output ID. The main difference (surprise!) is that you use it for sending output, not for receiving input. You always use the output object in concert with a render function, as in the following simple example:

(Note that the ID is quoted in the UI, but not in the server.)

The render function does two things:

  • It sets up a reactive context that automatically tracks what inputs the output uses.

  • It converts the output of your R code into HTML suitable for display on a web page.

Like the input, the output is picky about how you use it. If you forget the render function you’ll get an error:

Or if you attempt to read from an output you’ll get an error:

4.3 Reactive programming

An app is going to be pretty boring if it only has inputs or only has outputs. The real magic of Shiny happens when you have an app with both. Let’s look at a simple example:

It’s hard to show exactly how this works in a book, but if you run the app, and type in the name box, you’ll notice that the greeting updates automatically as you type7:

This is the big idea in Shiny: you don’t need to specify when the output code is run because Shiny automatically figures it out for you. How does it work? What exactly is going on in the body of the function? Let’s look at the code inside the server function in more detail:

It’s easy to read this as “paste together ‘hello’ and the user’s name, then send it to output$greeting”. But this mental model is wrong in a subtle, but important, way. Think about it: with this model, you only issue the instruction once. But Shiny perform the action every time we update input$name, so there must be something more going on.

The app works because the code doesn’t tell Shiny to create the string and send it to the browser, but instead, it informs Shiny how it could create the string if it needs to. It’s up to Shiny when (and even if!) the code should be run. It might be run as soon as the app launches, it might be quite a bit later; it might be run many times, or it might never be run. This isn’t to imply that Shiny is capricious, only that it’s Shiny’s responsibility to decide when code is executed, not yours. A better mental model is to think of giving Shiny recipes, not commands.

4.3.1 Imperative vs declarative programming

This difference between commands and recipes is one of key differences between two important styles of programming:

  • In imperative programming, you issue a specific command and it’s carried out immediately. This is the style of programming you’re used to in your analysis scripts: you command R to load your data, transform it, visualise it, and save the results to disk.

  • In declarative programming, you express higher-level goals or describe important constraints, and rely on someone else to decide how and/or when to translate that into action. This is the style of programming you use in Shiny.

With imperative code you say “Make me a sandwich”. With declarative code you say “Ensure there is a sandwich in the refrigerator whenever I look inside of it”. Imperative code is assertive; declarative code is passive-aggressive.

Most of the time, declarative programming is tremendously freeing: you describe your overall goals, and the software figures out how to achieve them without further intervention. The downside is the occasional time where you know exactly what you want, but you can’t figure out how to frame them in a way that the declarative system understands8. The goal of this book is to help you develop your understanding of the theory that underlies Shiny so that happens as infrequently as possible.

4.3.2 Lazyness

One of the strengths of declarative programming in Shiny is it that it allows apps to be extremely lazy. A Shiny app will only ever do the minimal amount of work needed to update the output controls that you can currently see9. This lazyness, however, comes with an important downside that you should be aware of. Can you spot what’s wrong with the server function below?

If you look closely, you might notice that I’ve written greetnig instead of greeting. This won’t generate an error in Shiny, but it won’t do what you want. Because the greetnig output doesn’t exist, the code inside renderText() will never be run.

If you’re working on a Shiny app and you just can’t figure out why your code never gets run, make sure that your UI and server functions are using the same identifiers.

4.3.3 The reactive graph

Shiny’s lazyness has another important property. In most R code, you can understand the order of execution by reading the code from top to bottom. That doesn’t work in Shiny, because code is only run when needed. To understand the order of execution you need to instead look at the reactive graph, which describes how inputs and outputs are connected. The reactive graph for the app above is very simple:

The reactive graph contains one symbol for every input and output, and we connect an input to an output whenever the output accesses the input. This graph tells you that greeting will need to be recomputed whenever name is changed. We’ll often describe this relationship as greeting has a reactive dependency on name.

Note the graphical conventions we used for the inputs and outputs: the name input naturally fits into the greeting output. We could draw them closely packed togther, as below, to emphasise the way that they fit together. However, we won’t normally do that because it only works for the simplest of apps.

The reactive graph is a powerful tool for understanding how your app works. As as your app gets more complicated, it’s often useful to make a quick high-level sketch of the reactive graph to remind you how all the pieces fit together. Throughout this book we’ll show you the reactive graph to help understand how the examples work, and later on, in Chapter XYZ, you’ll learn how to use reactlog which will draw the graph for you.

4.3.4 Reactive expressions

There’s one more important component that you’ll see in the reactive graph: the reactive expression. We’ll come back to reactive expressions in detail very shortly; for now think of them as a tool that reduces duplication in your reactive code by introducing additional nodes into the reactive graph.

We don’t need a reactive expression in our very simple app, but I’ll add one anyway so you can see how it affects the graph:

Reactive expressions take inputs and produce outputs so they have a shape that combines features of both inputs and outputs. The shapes should help you remember how they fit together.

4.3.5 Execution order

It’s important to understand the order in which your code is run is determined solely by the reactive graph. This is different from most R code where the execution order is determined by the order of lines. For example, we could flip the order of the two lines in our simple server function:

You might think that this would yield an error because output$greeting refers to a reactive expression, text, that hasn’t been created yet. But remember Shiny is lazy, so that code is only run when the session starts, after text has been created.

Instead, this code yields the same reactive graph as above, so the order in which the code is run is exactly the same. Note, however, that organising your code like this is confusing for humans, and best avoided. Instead, make sure that reactive expressions and outputs only refer to things defined above, not below. This will make your code easier to understand.

This concept is very important and different to most other R code, so I’ll say it again: the order in which reactive code is run is determined only by the reactive graph, not by its layout in the server function.

4.4 Reactive expressions

We’ve quickly skimmed over reactive expressions a couple of times, so you’re hopefully getting a sense for what they might do. Now we’re going to dive into more of the details, and show why they are so important when constructing real apps. Reactive expressions are important for two reasons:

  • They help create efficient apps by giving Shiny more information so that it can do less recomputation when inputs change.

  • They make it easier for humans to understand the app by simplifying the reactive graph.

Reactive expressions have a flavour of both inputs and outputs:

  • Like inputs, you can use the results of a reactive expression in an output.

  • Like outputs, reactive expressions depend on inputs and automatically know when they need updating.

Because of this duality, some functions work with either reactive inputs or expressions, and some functions work with either reactive expressions or reactive outputs. We’ll use producers to refer either reactive inputs or expressions, and consumers to refer to either reactive expressions or outputs.

We’re going to need a more complex app to see the benefits of using reactive expression. First, we’ll set the stage by defining some regular R functions that we’ll use to power our app.

4.4.1 The motivation

Imagine I want to compare two simulated datasets with a plot and a hypothesis test. I’ve done a little experimentation and come up with the functions below: histogram() visualises the two distributions with a histogram, and t_test() compares their means with with a t-test:

If I have some simulated data, I can use these functions to compare two variables:

In a real analysis, you probably would’ve done a bunch of exploration before you ended up with these functions. I’ve skipped that exploration here so we can get to the app as quickly as possible. But extracting imperative code out into regular functions is an important technique for all Shiny apps: the more code you can extract out of your app, the easier it will be to understand. This is good software engineering because it helps isolate concerns: the functions outside of the app focus on the computation so that the code inside of the app can focus on responding to user actions.

4.4.2 The app

I’d like to use these two tools to quickly explore a bunch of simulations. A Shiny app is a great way to do because it lets you avoid tediously modifying and re-running R code. Below I wrap the pieces into a Shiny app where I can interactively tweak the inputs.

Let’s start with the UI. The first row has three columns for input controls (distribution 1, distribution 2, and plot controls). The second row has a wide column for the plot, and a narrow column for the hypothesis test.

And the server function combines calls out histogram() and t_test() functions after drawing from the specified distributions:

Before you continue reading, I recommend opening the app and having a quick play to make sure you understand its basic operation:

4.4.3 The reactive graph

Let’s start by drawing the reactive graph of this app. Shiny is smart enough to run code to produce an output only when the inputs it refers to change; it’s not smart enough to only selectively run pieces of code inside an output. In other words, outputs are atomic: they’re either executed or not as a whole.

For example, take this snippet from the server:

As a human reading this code you can tell that we only need to update x1 when n1, mean1, or sd1 changes, and we only need to update x2 when n2, mean2, or sd2 changes. Shiny, however, only looks at the output as a whole, so it will update both x1 and x2 every time one of n1, mean1, sd1, n2, mean2, or sd2 changes. This leads to the following reactive graph:

You’ll notice that the graph is very dense: almost every input is connected directly to every output. This creates two problems:

  • The app is hard to understand because there are so many connections. There are no pieces of the app that you can pull out and analyse in isolation.

  • The app is inefficient because it does more work than necessary. For example, if you change the breaks of the plot, the data is recalculated; if you change the value of n1, x2 is updated (in two places!).

There’s one other major flaw in the app: the histogram and t-test use separate random draws. This is rather misleading, as you’d expect them to be working on the same underlying data. Fortunately, we can fix all these problems by using reactive expressions to pull out repeated computation.

4.4.4 Simplifying the graph

In the server function below we refactor the existing code to pull out the repeated code into two new reactive expressions, x1 and x2, which simulate the data from the two distributions. To create a reactive expression, we call reactive() and assign the results to a variable. To later use the expression, we call the variable like its a function.

This transformation yields a substantially simpler graph:

This simpler graph makes it easier to understand the app because you can understand connected components in isolation; the values of the distribution parameters only affect the output via x1 and x2. This rewrite also makes the app much more efficient since it does much less computation. Now when you change the binwidth or range, only the plot changes, not the underlying data.

To emphasise this modularity the following diagram draws boxes around the independent components. We’ll come back to this idea in Chapter XYZ, when we discuss modules. Modules allow you to extract out repeated code for reuse, while guaranteeing that it’s isolated from everything else in the app. Modules are an extremely useful and powerful technique for more complex apps.

You might be familiar with the “rule of three” of programming: whenever you copy and paste something three times, you should figure out how to reduce the duplication (typically by writing a function). This is important because it reduces the amount of duplication of in your code, which makes it easier to understand, and easier to update as your requirements change.

In Shiny, however, I think you should adopt the rule of one: whenever you copy and paste something once, you should consider extracting the repeated code out into a reactive expression. The rule is stricter for Shiny, because reactive expressions don’t just make it easier for humans to understand the code, they also improve Shiny’s ability to efficiently rerun code.

4.4.5 Why do we need reactive expressions?

When you first start working with reactive code, you might wonder why we need reactive expression. Why can’t you you use your existing tools for reducing duplication in code: creating new variables and writing functions? Unfortunately neither of these techniques works in a reactive environment, for reasons that we’ll explain below.

If you try and use a variable to reduce duplication, you might write something like this:

If you try this code, you’ll get an error because you’re attempting to access input values outside of a reactive context. Even if you didn’t get that error, you’d still have a probelm: x1 and x2 would only be computed once, when the session begins, not every time one of the inputs was updated.

If you try and use a function, the app will work:

But it has the same problem as the original code: any input will cause all outputs to be recomputed, and the t-test and the histogram will be run on separate samples. Reactive expressions automatically cache their results, and only update when their inputs change10.

While variables calculate the value only once (the porridge is too cold), and functions calculate the value every time they’re called (the porridge is too hot), reactive expressions calculate the value only when it might have changed (the porridge is just right!).

4.5 Controlling timing of evaluation

Now that you’re familiar with the basic ideas of reactivity, we’ll discuss two more advanced techniques that allow you to either increase or decrease how often a reactive expression is executed. Here I’ll show how to use the basic techiques; in Chapter XYZ, we’ll come back to their underlying implementations.

To explore the basic ideas, I’m going to simplify my simulation app. I’ll use a distribution with only one parameter, and force both samples to share the same n. I’ll also remove the plot controls. This yields yields a smaller UI object and server function:

This app has following reactive graph:

4.5.1 Timed invalidation

Imagine you wanted to reinforce the fact that this is for simulated data by constantly resimulating the data, so that you see an animation rather than a static plot11. We can increase the frequencly of updates with a new function: reactiveTimer().

reactiveTimer() is a reactive expression that has a dependency on a hidden input: the current time. You can use a reactiveTimer() when you want a reactive expression to invalidate itself more often than it otherwise would. For example, the following code uses an interval of 500 ms so that the plot will update twice a second. This is fast enough to remind you that you’re looking at a simulation, without dizzying you with rapid changes.

Note how we use timer() in the reactive expressions that compute x1() and x2(): we call it, but don’t use the value. This lets x1 and x2 take a reactive dependency on timer, without worrying about exactly what value it returns.

4.5.2 On click

In the above scenario, think about what would happen if the simulation code took 1 second to run. Shiny would have more and more to do, and would never be able to catch up. The same problem can happen if someone is rapidly clicking buttons in your app and the computation you are doing is relatively expensive. It’s possible to create a big backlog of work for shiny, and while it’s working on the backlog, it can’t respond to any new events.

If this situation arises in your app, you might want to require the user to opt-in to performing the expensive calculation by requiring them to click a button. This is a great use case for an actionButton():

To use the action button we need to learn a new tool. To see why, lets first tackle the problem using the same approach as above. As above, we refer to simulate without using its value to take a reactive dependency on it.

This doesn’t achieve our goal because it just introduces an new dependency: x1() and x2() will update when we click the simulate button, but they’ll also continue to update when lambda1, lambda2, or n change. We want to replace the existing dependencies, not supplement them.

To solve this problem we need a new tool: a way to use input values input without taking a reactive dependency on them. We need eventReactive(), which has two arguments: the first argument specifies what to take a dependency on, and the second argument specifies what to compute. That allows this app to only compute x1() and x2() when simulate is clicked:

Note that, as desired, x1 and x2 no longer have a reactive dependency on lambda1, lambda2, and n: changing their values does will not trigger computation. I left the arrows in very pale grey just to remind you that x1 and x2 continue to use the values, but no longer take a reactive dependency on them.

4.6 Observers

So far, we’ve focussed on what’s happening inside the app. But sometimes you need to reach outside of the app and cause side-effects to happen elsewhere in the world. This might be saving a file to a shared network drive, sending data to a web API, updating a database, or (most commonly) printing a debugging message to the console. These actions don’t affect how your app looks, so you can’t use an output and a render function. Instead you need to use an observer.

There are multiple ways to create an observer, and we’ll come back to them later in Chapter XYZ. For now, I wanted to show you how to use observeEvent(), because it gives you an important debugging tool when you’re first learning Shiny.

observeEvent() is very similar to eventReactive(). It has two important arguments: eventExpr and handlerExpr. The first argument is the input or expression to take a dependency on; the second argument is the code that will be run. For example, the following modification to server() means that everytime that name is updated, a message will be sent to the console:

There are two important differences between observeEvent() and eventReactive():

  • You don’t assign the result of observeEvent() to a variable, so
  • You can’t refer to it from other reactive consumers.

Observers and outputs are closely related. You can think of outputs as having a special side-effect: updating the HTML in the users browser. To emphasise this closeness, we’ll draw them the same way in the reactive graph. This yields the following reactive graph for the app above:

This concludes our whirlwind tour of reactivity. The next chapter will help you practice all the material you’ve seen so far, as we practice turn an existing script into a Shiny app.

  1. The primary exception is where there’s some work that can shared across multiple users. For example, all users might be looking at the same large csv file, so you might as well load it once and share it between users. We’ll come back to that idea in Chapter XYZ.

  2. For legacy reasons, session is optional, but you should always include it.

  3. If you’re running the live app, notice that you have to type fairly slowly for the output to update one letter at a time. That’s because shiny uses a technique called debouncing, which means that it waits for XYZ ms before sending an update. That considerably reduces the amount of work that Shiny needs to do, without appreciably reducing the response time of the app.

  4. If you’ve ever struggled to get a ggplot2 legend to look exactly the way you want, you’ve encountered this problem!

  5. Yes, shiny doesn’t update the output if you can’t see it in your browser! Shiny is so lazy that it doesn’t do the work unless you can actually see the results.

  6. If you’re familiar with memoisation, this is a similar idea.

  7. I think the New York Times used this technique particularly effectively in their article discussing how to interpret the jobs report: