Chapter 3 Data

In this chapter, you will learn how to add data to your Shiny applications, and allow your users to upload and download data.

3.1 Adding data to your app

3.1.1 The easy part

You can load data into a Shiny app using any programmatic method you use in normal R code (i.e. point and click use of the interface can’t be replicated in Shiny). This includes:

  • CSVs and other flat files
  • R data files
  • Excel files
  • Web scraping
  • APIs
  • Data from BigQuery or SQL (Our BigQuery to R training course provides information to get your data from BigQuery into R)
  • Data stored in cloud buckets

Any call which allows you to read in data can be replicated in Shiny.

3.1.2 The tricky part

Your Shiny app’s working directory may be different from the directories you usually work with in other projects. Some file types may also be slower to load in Shiny, so consider which data formats to use. Where you read data in the app affects its loading speed, how the data is used, and which parts of the app can access it. Additionally, if your app needs to access data directly from an online source, it may require special permissions (though this is beyond the scope of this course).

3.1.3 The Shiny working directory

In Shiny apps, relative file paths must use the correct working directory, which by default is the app’s root folder (the folder containing server.R and ui.R). This isn’t necessarily the same as your project or R’s current working directory, so it’s important to specify paths carefully.

For example, if you have a Data folder in your project:

Project Root
├── Data/
│   └── file_name_here
├── test_shiny/
    ├── server.R
    ├── ui.R
  • from a standard R code in the same project, you’d access files in Data like this: "Data/file_name_here"
  • if your Shiny app (e.g. test_shiny) is in a subfolder of the project, you’ll need to go up one level to access the Data folder: "../Data/file_name_here"

3.1.4 Choosing sensible data storage

Your Shiny app won’t load completely until it’s finished doing all the data reading and processing. If this is slow, it will be frustrating for users!

Therefore, to improve loading speed:

  • choose file types which have fast reading options. The ideal is flat files such as CSVs (using data.table::fread()), Rdata files and parquet files
  • data tidying and processing steps are slow, especially if they need to be done every time to app opens. Use a pre-processing script where possible and save your data in a clean, tidy data format
  • if you reliant on external data sources, run pre-app QA check there are breaking changes introduced into the data

3.1.5 Where to load data

Data is almost always loaded and used inside the server. You can’t do any data processing within inputs or create outputs in the UI.

There are three different ways you can load data into the server, depending on what you want to do with it:

  1. Before the server call
# Load data before the server call
data <- readr::read_csv("Data/mydata.csv")

server <- function(input, output) {
  output$table <- renderTable({
    # Data is available directly without reloading
    data  
  })
}

Useful if the data does not change based on user input and is needed across multiple functions.

  1. Inside the server call, in a reactive function
server <- function(input, output) {
  # Reactive function to load data or filter it based on user input
  filtered_data <- reactive({
    data <- readr::read_csv("Data/mydata.csv")
    data[data$category == input$category, ]
  })
  
  output$table <- renderTable({
    filtered_data()
  })
}

Useful when data loading or processing depends on user selections, as it recalculates only when needed.

  1. Inside the server call, in a render function
server <- function(input, output) {
  output$table <- renderTable({
    data <- readr::read_csv("Data/mydata.csv")
    data[data$category == input$category, ]
  })
}

Typically used for simpler data processing or one-off calculations related to specific outputs.

Each method fits specific scenarios:

Ways to load in data Type of loading Used for
Before the server call Global loading Static data - only happens once (when the app is first loaded), so it’s fast and less processing power is required
Inside the server call, in a reactive function Reactive loading Dynamic data - happens many times (every time the data is called or an input is updated from the user), therefore is slower and more processing power is required.
Inside the server call, in a render function Render-based loading Producing a visual object (e.g. text, chart or table)

A summary of the two types of reactive environments in a Shiny app:

  • reactive(): Used to create intermediate data objects for the server, such as a vector or data frame, that can be used across multiple outputs.
    • ideal for processing data that might feed into multiple outputs.
    • cannot be used to create output objects directly for the UI
  • render(): Used to create visual objects for the UI, like charts or tables, which will display data to the user.
    • suitable for final data processing, focused on producing a single output.
    • contains the code to actually generate the output (e.g., a plot or table) and is used to create UI-facing objects only.

A reminder that the reactive() and render() functions need to be wrap in both round and curly brackets: reactive({}) and render({})

3.1.6 Exercise

10:00

Open Chapter 3 > Exercise-3.1.6 in the training repository and complete the following tasks:

  1. In your Shiny app, replace the gapminder data with eurovision data. This data can be read in directly from ‘https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2022/2022-05-17/eurovision.csv’.
  2. Read the data in inside the renderDataTable({}).
  3. Adjust the year filter so you can still filter the data on the year input widget, and have a go at filtering the dataset a few times.
3.1.6 Solution


UI

library(shiny)
library(lubridate)
library(gapminder)
library(dplyr)


shinyUI(fluidPage(
  
  titlePanel("My Shiny App"),
  
  sidebarLayout(
    sidebarPanel(
      selectInput("year",
                  label = "Select Year(s):",
                  ### Question 3: Adjust the year filter so you can still filter the data
                  ## on the year input widget
                  ##START HERE ----
                  choices = c(1956:2022),
                  selected = 2007,
                  multiple = TRUE
      )
    ),
    
    mainPanel(
      
      textOutput("welcomeText"),
      
      #Optional: 
      ## - You may want to change the input `name` to something more appropriate
      ## - If you do, make sure it matches the output$`name` in the renderDataTable in the server
      dataTableOutput("eurovisionTable")
    )
  )
))

Server

library(shiny)
library(lubridate)
library(gapminder)
library(dplyr)

shinyServer(function(input, output) {
  
  output$welcomeText <- renderText({
    paste("Welcome to the app. Showing data for", paste(input$year, collapse = ", "))
  })
  
  ### Question 1: Replace the gapminder data with eurovision.
  ### Question 2: Ensure to read in the data inside the renderDataTable({}) function
  ## Optional: 
  # - You may want to change the output$`name` to something more appropriate
  # - If you do, make sure it matches the input `name` in the dataTableOutput() in the UI
  output$eurovisionTable <- renderDataTable({
    ##START HERE ----
    eurovision_data <- 
      read.csv("https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2022/2022-05-17/eurovision.csv") %>%
      dplyr::filter(year %in% input$year)
  })
  
})

Training repository: Go to Chapter 3 > Exercise-3.1.6 > Solution

Run either the ui.R or server.R file to see the app in action.


3.2 Creating reactives

Reactive environments are created using the reactive({}) function, and MUST be assigned to an R object. As mentioned before, you will want to wrap the code inside the reactive in both round () and curly {} brackets.

eurovision_filtered <- reactive({
  eurovision_data %>%
    dplyr::filter(year %in% as.numeric(input$year))
})

You can then write any normal R code inside the reactive call, similar to how you write code inside a function.

3.2.1 Calling reactives

When calling and using reactives, you can treat them as if they were a special case of function with no arguments. You can refer to them by their assigned name followed by empty brackets e.g. my_data().

output$eurovision <- renderDataTable({
  eurovision_filtered()
})

If you do not include the brackets, you will get an error!

When you use a reactive, it behaves like a function and gives you back the most recent result it calculated. You can use one reactive inside another reactive or in functions that create outputs (like plots or tables) in Shiny. However, you can’t use a reactive outside of Shiny’s reactive system (like in code that runs only once when the app starts) because it won’t update properly.

3.2.2 Naming in reactive environments

Naming objects works slightly differently for reactive environments

  • objects used inside the reactive are created and destroyed inside there
  • named objects can’t be used outside the reactive environment they are created in
eurovision_filtered <- reactive({
  data <- eurovision_data %>%
    dplyr::filter(year %in% as.numeric(input$year))
})

As a result, names can be reused in different reactive environments for different objects.

3.3 Reactive efficiency

Like a function, reactives are not updated until they are called. This happens:

  • when they are used for the first time in another reactive({}) or render({}) call
  • when an input that feeds into them changes

This means that every time a user changes an input variable, the data (and outputs) will automatically update. This means the whole reactive will run every time.

Structuring your reactives can therefore improve the efficiency and speed of your dashboard:

eurovision_filtered <- reactive({
  readr::read_csv("eurovision.csv") %>%
    dplyr::filter(year %in% as.numeric(input$year))
})

3.3.1 Exercise

10:00

Open Chapter 3 > Exercise-3.3.1 in the training repository and complete the following tasks:

  1. Create a new reactive environment inside your app, and give it a sensible name
  2. Move the reading for the Eurovision data into this reactive environment
  3. Call the reactive environment inside your renderDataTable() so that the table still works
3.3.1 Solution


UI

library(shiny)
library(lubridate)
library(gapminder)
library(dplyr)


shinyUI(fluidPage(
  
  titlePanel("My Shiny App"),
  
  sidebarLayout(
    sidebarPanel(
      selectInput("year",
                  label = "Select Year(s):",
                  choices = c(1956:2022),
                  selected = 2007,
                  multiple = TRUE
      )
    ),
    
    mainPanel(
      
      textOutput("welcomeText"),
      
      dataTableOutput("eurovisionTable")
    )
  )
))

Server

library(shiny)
library(lubridate)
library(gapminder)
library(dplyr)

shinyServer(function(input, output) {
  
  output$welcomeText <- renderText({
    paste("Welcome to the app. Showing data for", paste(input$year, collapse = ", "))
  })
  
  ### Question 1: Create a new reactive environment, using an appropriate `name` 
  ## Hint: use the reactive({})
  ### Question 2: Move the reading for the Eurovision data into the reactive function
  ## Hint: remember to add the filter() so users can continue to filter by their chosen years
  
  ## START HERE ----
  eurovision_filtered <- reactive({
    eurovision_data <-
      read.csv("https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2022/2022-05-17/eurovision.csv") %>%
      dplyr::filter(year %in% input$year)
  })
  
  output$eurovisionTable <- renderDataTable({
    ### Question 3: Call the reactive environment inside your renderDataTable
    ## Hint: do not forget the brackets reactive_name() when calling the reactive environment
    
    ## START HERE ----
    eurovision_filtered()
    
  })
  
})

Training repository: Go to Chapter 3 > Exercise-3.3.1 > Solution

Run either the ui.R or server.R file to see the app in action.


3.4 Data downloads and uploads

3.4.1 Download buttons

Often you will want to give your users the option to download the data they have filtered. Shiny offers this functionality through the download button.

This works similarly to other outputs, being named as the first argument in the function call in the UI.

In the server, the filename of the resulting file and the content of the download are specified as functions.

Example in the UI:

downloadButton("download_data", "Download")

Example in the Server:

output$download_data <- downloadHandler(
  filename = function() {
    paste('data-', Sys.Date(), '.csv', sep = "")
  }, 
  content = function(con) {
    write.csv(eurovision_filtered(), con)
  }
)
  • downnloadHandler: sets up the logic for downloading data from the app. It takes two main parts: filename and content
  • filename:
    • defines what the downloaded file will be named
    • filename is a function that returns a name for the file. Here, it uses paste() to create a file name that combines "data-", the current date "Sys.Date()", and the .csv extension
    • example: if today’s date is 2024-11-07, the file name would be "data-2024-11-07.csv"
  • content:
    • specifies the data that will go into the file
    • content is a function with one argument, con, which stands for “connection.” con is a temporary file path where the data will be saved before downloading
    • Inside content, write.csv() saves the eurovision_filtered() data as a CSV file in the location provided by con. The eurovision_filtered() is a reactive expression (likely a filtered dataset) that contains the data to be downloaded

3.4.2 Allowing data upload

As well as making use of data you choose, you can also produce dashboards which allow users to upload their own data.

This can be done using the fileInput() function in the UI. This works in the same way as any other input; the first argument is a name and the parameters of the input can be set using the other arguments.

To access and use the contents of the file that was uploaded, you can call input$name in the server using the name you provided.

3.4.3 Exercise

10:00

Open Chapter 3 > Exercise-3.4.3 in the training repository and complete the following tasks:

  1. Add a download button to your app to allow users to download the filtered Eurovision data.
  2. Set the name of the downloaded file to include "Eurovision_data" and the date that the data was downloaded.
3.4.3 Solution


UI

library(shiny)
library(lubridate)
library(gapminder)
library(dplyr)


shinyUI(fluidPage(
  
  titlePanel("My Shiny App"),
  
  sidebarLayout(
    sidebarPanel(
      selectInput("year",
                  label = "Select Year(s):",
                  choices = c(1956:2022),
                  selected = 2007,
                  multiple = TRUE
      ),
      
      ### Question 1: Add a download button to the app 
      ## START HERE ----
      
      downloadButton("downloadData", "Download the data")
      
    ),
    
    mainPanel(
      
      textOutput("welcomeText"),
      
      dataTableOutput("eurovisionTable")
    )
  )
))

Server

library(shiny)
library(lubridate)
library(gapminder)
library(dplyr)

shinyServer(function(input, output) {
  
  output$welcomeText <- renderText({
    paste("Welcome to the app. Showing data for", paste(input$year, collapse = ", "))
  })
  
  eurovision_filtered <- reactive({
    eurovision_data <-
      read.csv("https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2022/2022-05-17/eurovision.csv") %>%
      dplyr::filter(year %in% input$year)
  })
  
  output$eurovisionTable <- renderDataTable({
    eurovision_filtered()
  })
  
  ### Question 2: Set the name of the downloaded file to include "Eurovision_data"
  ### and the date that the data was downloaded
  ## Hint: 
  # - remember the output$`name` needs to be the same as the input `name` in the UI 
  # - you can use the Sys.Date() or lubridate::today() to get today's date
  
  ## START HERE ----
  
  output$downloadData <- downloadHandler(
    filename = function() {
      paste("Eurovision_data-", lubridate::today(), ".csv", sep = "")
    }, 
    content = function(con) {
      write.csv(eurovision_filtered(), con)
    }
  )
  
})

Training repository: Go to Chapter 3 > Exercise-3.4.3 > Solution

Run either the ui.R or server.R file to see the app in action.


This is a good opportunity to take a 10-minute break away from the computer to refresh your mind, stretch, and reset before continuing onto Chapter 4.