Meet the toolkit

Lecture 1

John Zito

Duke University
STA 199 Spring 2025

2025-01-14

Reminders

• If you have not yet completed the Getting to Know You survey, please do so ASAP!

• If you have not yet accepted the invite to join the course GitHub Organization, please do so pronto!

• Make your appointments in the Testing Center now!

• Any questions about the syllabus?

Course toolkit

Course toolkit

Course operation

• Materials: sta199-f24.github.io
• Submission: Gradescope
• Discussion: Ed Discussion
• Gradebook: Canvas

Doing data science

• Computing:
  • R
  • RStudio
  • tidyverse
  • Quarto
• Version control and collaboration:
  • Git
  • GitHub

Toolkit: Computing

Learning goals

By the end of the course, you will be able to…

• gain insight from data
• gain insight from data, reproducibly
• gain insight from data, reproducibly, using modern programming tools and techniques
• gain insight from data, reproducibly and collaboratively, using modern programming tools and techniques
• gain insight from data, reproducibly (with literate programming and version control) and collaboratively, using modern programming tools and techniques

Reproducible data analysis

Reproducibility checklist

What does it mean for a data analysis to be “reproducible”?

Short-term goals:

• Are the tables and figures reproducible from the code and data?
• Does the code actually do what you think it does?
• In addition to what was done, is it clear why it was done?

Long-term goals:

• Can the code be used for other data?
• Can you extend the code to do other things?

Toolkit for reproducibility

• Scriptability \(\rightarrow\) R
• Literate programming (code, narrative, output in one place) \(\rightarrow\) Quarto
• Version control \(\rightarrow\) Git / GitHub

R and RStudio

R and RStudio

• R is an open-source statistical programming language
• R is also an environment for statistical computing and graphics
• It’s easily extensible with packages

• RStudio is a convenient interface for R called an IDE (integrated development environment), e.g. “I write R code in the RStudio IDE”
• RStudio is not a requirement for programming with R, but it’s very commonly used by R programmers and data scientists

R vs. RStudio

Source: Modern Dive.

R packages

• Packages: Fundamental units of reproducible R code, including reusable R functions, the documentation that describes how to use them, and sample data¹

• As of 27 August 2024, there are 21,168 R packages available on CRAN (the Comprehensive R Archive Network)²

• We’re going to work with a small (but important) subset of these!

¹ Wickham and Bryan, R Packages.

²CRAN contributed packages.

Tour: R + RStudio

Option 1:

Sit back and enjoy the show!

Option 2:

Go to your container and launch RStudio.

Tour recap: R + RStudio

A short list
(for now)
of R essentials

Packages

• Installed with install.packages(), once per system:

install.packages("palmerpenguins")

Note

We already pre-installed many of the package you’ll need for this course, so you might go the whole semester without needing to run install.packages()!

• Loaded with library(), once per session:

library(palmerpenguins)

Packages, an analogy

If data analysis was cooking…

• RStudio is your kitchen. It comes with a fridge, a stove, a sink, etc pre-installed;

• Installing a package would be like buying more appliances at the store: mixer, blender, toaster, instapot, air fryer;

• Loading a package would be like taking these things out of the cupboard;

• Your containers are like kitchens where we have already bought all of the extra appliances for you. In other words, “batteries included.”

tidyverse

aka the package you’ll hear about the most…

tidyverse.org

• The tidyverse is an opinionated collection of R packages designed for data science
• All packages share an underlying philosophy and a common grammar

Data frames and variables

• Each row of a data frame is an observation

• Each column of a data frame is a variable

• Columns (variables) in data frames can be accessed with $:

dataframe$variable_name

Help

Object documentation can be accessed with ?

?mean

Toolkit: Version control and collaboration

Git and GitHub

• Git is a version control system – like “Track Changes” features from Microsoft Word, on steroids
• It’s not the only version control system, but it’s a very popular one

• GitHub is the home for your Git-based projects on the internet – like DropBox but much, much better

• We will use GitHub as a platform for web hosting and collaboration (and as our course management system!)

Versioning - done badly

Versioning - done better

Versioning - done even better

with human readable messages

How will we use Git and GitHub?

How will we use Git and GitHub?

How will we use Git and GitHub?

How will we use Git and GitHub?

Git and GitHub tips

• There are millions of git commands – ok, that’s an exaggeration, but there are a lot of them – and very few people know them all. 99% of the time you will use git to add, commit, push, and pull.
• We will be doing Git things and interfacing with GitHub through RStudio, but if you google for help you might come across methods for doing these things in the command line – skip that and move on to the next resource unless you feel comfortable trying it out.
• There is a great resource for working with git and R: happygitwithr.com. Some of the content in there is beyond the scope of this course, but it’s a good place to look for help.

Tour: Git + GitHub

Option 1:

Sit back and enjoy the show!

Note

You’ll need to stick to this option if you haven’t yet accepted your GitHub invite and don’t have a repo created for you.

Option 2:

Go to the course GitHub organization and clone ae-your_github_name repo to your container.

Tour recap: Git + GitHub

• Find your application repo, that will always be named using the naming convention assignment_title-your_github_name

• Click on the green “Code” button, make sure SSH is selected, copy the repo URL

Tour recap: Git + GitHub

• In RStudio, File > New Project > From Version Control > Git
• Paste repo URL copied in previous step, then click tab to auto-fill the project name, then click Create Project
• For one time only, type yes in the pop-up dialogue

What could have gone wrong?

• Never received GitHub invite \(\rightarrow\) Fill out “Getting to know you survey

• Never accepted GitHub invite \(\rightarrow\) Look for it in your email and accept it

• Cloning repo fails \(\rightarrow\) Review/redo Lab 0 steps for setting up SSH key

• Still no luck? Visit OH or post on Ed.

Quarto

Quarto

• Fully reproducible reports – each time you render the analysis is ran from the beginning
• Code goes in chunks narrative goes outside of chunks
• A visual editor for a familiar / Google docs-like editing experience

Tour: Quarto

Option 1:

Sit back and enjoy the show!

Note

If you chose (or had to choose) this option for the previous tour, or if you couldn’t clone your repo for any reason, you’ll need to stick to this option.

Option 2:

Go to RStudio and open the document ae-01-meet-the-penguins.qmd.

Tour recap: Quarto

Tour recap: Git + GitHub

Once we made changes to our Quarto document, we

• went to the Git pane in RStudio

• staged our changes by clicking the checkboxes next to the relevant files

• committed our changes with an informative commit message

• pushed our changes to our application exercise repos

• confirmed on GitHub that we could see our changes pushed from RStudio

How will we use Quarto?

• Every application exercise, lab, project, etc. is an Quarto document
• You’ll always have a template Quarto document to start with
• The amount of scaffolding in the template will decrease over the semester