GCP to R Playbook
1
Introduction
1.1
What does this playbook cover?
1.2
Presumed knowledge
1.3
Latest updates
1.4
GCP Project Transition
1.4.1
What is changing?
1.4.2
Do I need to learn Terraform?
1.4.3
Resources
1.5
We want to hear from you!
1.6
Working with the Data Engineering team
1.7
Key terminology
1.8
Why move data into GCP?
2
Google Cloud Platform
2.1
Definition
2.2
How to: Access GCP
2.2.1
An existing project
2.2.2
A new project
2.3
GCP Permissions
2.3.1
Common permissions
2.3.2
Less common permissions
2.3.3
How to: Check your permissions
2.3.4
How to: Amend your permissions
2.4
How to: Connect GCP to R
2.4.1
Outdated method
2.4.2
Errors in connecting GCP to R
3
Google Cloud Storage
3.1
Definition
3.1.1
Why use GCS with R?
3.2
How to: Access GCS
3.3
Buckets
3.3.1
Naming buckets
3.3.2
Regions
3.3.3
Storage types
3.4
How to: Use GCS
3.4.1
Create a bucket
3.4.2
Connect to GCS from R
3.4.3
Add data
3.4.4
Share data
3.4.5
Read data
3.4.6
Delete a bucket
3.5
Making best use of GCS
3.5.1
When to use GCS
3.5.2
When to not use GCS
3.6
Data Lake
3.7
Data Management
3.7.1
Retaining outdated versions of objects
3.7.2
Lifecycle
3.7.3
Retention
3.7.4
Monitoring
3.8
Metadata
3.8.1
Bucket metadata
3.8.2
Object metadata
4
BigQuery
4.1
Definition
4.2
How to: Access BigQuery
4.2.1
BigQuery interface
4.2.2
BigQuery editor
4.2.3
Terminology
4.3
How to: Use BigQuery
4.3.1
Create a Dataset and Table
4.3.2
Naming BigQuery Objects
4.3.3
Querying using SQL
4.3.4
Best practices for writing SQL in BigQuery
4.3.5
Optimise your SQL Queries
4.3.6
Saving your queries
4.4
Effective Table Design
4.4.1
OLTP: Online Transaction Processing
4.4.2
OLAP: Online Analytical Processing
4.4.3
The Shift towards Denormalisation and One Big Table (OBT)
4.4.4
Adopting Tidy Data principles
4.4.5
Best practices
4.5
Advanced Table Design
4.5.1
Partitioning in BigQuery
4.5.2
Nested or Repeated Columns
4.5.3
Clustering
4.6
How to: Use BQ in R
4.6.1
Connect to BQ from R
4.6.2
Read data: In (Cloud) R
4.6.3
Add data: From (Cloud) R
4.7
Migrating legacy SQL code
4.7.1
T-SQL vs. BigQuery
4.7.2
Creating stored procedures
4.7.3
How to: Write a stored procedure
4.8
Shiny apps
4.8.1
Bringing data into R Shiny
4.8.2
Top tips
4.8.3
Troubleshooting tips with Shiny and BigQuery
4.8.4
How to: Publish Shiny apps on RSConnect
5
Contacts
Published with bookdown
GCP to R Playbook
Chapter 5
Contacts
To contact the Data Engineering (DE) team:
email at
Data.Engineering@dft.gov.uk
check
People Finder
read the
DE handbook