Beta
1 - Introduction to Git
# INTRODUCTION TO VERSION CONTROL WITH GIT
# What is a version?
# 1. Contents of a file at a given point in time
# 2. Metadata (information associated with the file):
# The author of a file
# Where it is located
# The file type
# When it was last saved
# What is version control?
# Version control is a group of systems and processes
# to manage changes made to documents, programs, and directories
# Version control is useful for anything that:
# changes over time, or
# needs to be shared
# --> not just for programs
# Track files in different states
# Simultaneous file development (Continuous Development)
# Combine different versions of files
# Identify a particular version
# Revert changes
# Why is version control important?
# To illustrate why version control is essential when working with data, consider a common scenario of modifying a dataset:
# We save separate copies at various time intervals, using fairly similar names.
# We then produce new analyses as the dataset changes, but matching these outputs to the correct data can be difficult as time goes by!
# Put another way, a data project without version control is like cooking without a recipe - it'll be difficult to remember how to produce the same results again.
# Git
# Popular version control system for computer programming and data projects
# Open source
# Scalable
# --> easily track everything from small solo projects to complex collaborative efforts with large teams!
# Git is not Github
# Github is a cloud-based Git repository hosting platform
# it's common to use Git with GitHub
# Benefits of Git
# Git stores everything, so nothing is lost
# Git notifies us when there is conflicting content in files
# Git synchronizes across different people and computers
# Using Git
# Git commands are run on the 'shell', also known as the 'terminal'
# The shell:
# is a program for executing commands
# can be used to easily preview, modify, or inspect files and directories
# Directory = folder
# Useful shell commands
# 'pwd'
# = print working directory
# to see our location
pwd
# /home/repl/Documents
# 'ls'
# To see what is in our current directory we can use the ls command.
# This returns a list of all files and directories.
ls
# archive finance.csv finance_data_clean.csv finance_data_modified.csv
# Changing directory
cd archive
pwd
# /home/repl/Documents/archive
# Editing a file
nano finance.csv
# Use 'nano' to:
# delete,
# add,
# or change contents of a file
# Save changes: 'Ctrl + O'
# Exit the text editor: 'Ctrl + X'
# 'echo' -create or edit a file
# Create a new file 'todo.txt'
echo "Review for duplicate records" > todo.txt
# Add content to existing file 'todo.txt'
echo "Review for duplicate records" >> todo.txt
# Checking Git version
# Different versions of software have different functionality, and Git is no exception.
git --version
# git version 2.17.1
# SAVING FILES
# Now let's explore how Git stores information, along with looking at a workflow to update files and check their status.
# A repository
# To start, let's discuss the components of a Git project. We'll be working with a project about mental health in tech throughout the course, which is shown here.
# There are two parts
# - the first is the files and directories that we create and edit, in this case:
# a funding document: funding.doc,
# a markdown file containing a report: report.md,
# and a directory called data.
# - The second part is the extra information that Git records about the project's history. The combination of these two things is called a repository, often referred to as a repo. Git stores all of its extra information in a directory called '.git', located in the main directory of the repo.
# Git expects this information to be laid out in a particular way:
# 'Do not edit or delete .git!'
# Staging and committing
# Saving a draft
# 'Staging area'
# Save files/update the repo
# 'Commit changes'
# 'Staging area'
# like placing a letter in an envelope
# 'Making a commit'
# like putting the envelope in a mailbox
# We can add more things to the envelope or take things out as often as we want, but once we put it in the mailbox we can't make further changes.
# Accessing the .git directory
# Although we shouldn't edit the dot-git directory, it may be helpful to see what's inside.
# It won't display when using the shell ls command, as it's a hidden directory.
# A hidden directory is a directory not displayed to users, typically because it stores information to enable programs to run.
ls
# data report.md
# But if we add the dash-a flag it shows up along with some hidden files!
ls -a
# . .DS_Store data
# .. .git report.md
# Making changes to files
# 'Working directory' 'Staging Area' .git Directory
# (Permanent Storage)
# report.md ----------> staged
# ----------> staged ------------> committed
# ----------> staged
# ----------> staged
# ----------> staged ------------> committed
# Let's visualize the Git storage workflow. Here, we modify a markdown file called report.md, and store five draft updates to the staging area as we progress.
# We commit, or save, the second and fifth versions of the file in the staging area, and with each commit our .git directory is modified to reflect the state of the repo.
# Git workflow
# Modify a file
# Save the draft
# Commit the updated file
# Repeat
# Modifiying a file
nano report.md
# Then we add 3 lines of text
# Mental Health in Tech Survey
# TODO: write executive summary.
# TODO: include link to raw data.
# Save using 'Ctrl + O' and 'Ctrl + X'
# Saving a file
# Adding a single file (to the staging area)
git add report.md
# Adding all modified files
git add .
# '.' = all files and directories in current location
# Making a commit
git commit -m "Updating TODO list in report.md"
# Log message is useful for reference
# Best practice = short and concise
# Check the status of files
# If we are making lots of changes then it's useful to know the status of our repo.
# which tells us which files are in the staging area, and which files have changes that aren't in the staging area yet.
git status
# on branch main
# Changes to be committed:
# (use "git restore --staged <file>..." to unstage)
# modified: report.md
# In this case, we see report-dot-md has been modified and is in the staging area, so we make a commit.
git commit -m "New TODO in report.md"
# Exercise: Adding a file
echo "49,M,No,Yes,Never,Yes,Yes,No" >> mental_health_survey.csv
git add mental_health_survey.csv
git commit -m "Adding one new participant's data"
# In three commands you've added a new row of data to a file, stored it as a draft, and made a commit to update the repo!
# Exercise: Adding multiple files
git status
git add .
git commit -m "Added 3 participants and a new section in report"
# 'git status' is a great way to see where you are in the version control workflow, allowing you to take steps to ensure everything is up-to-date. The commit output shows that two files were changed
# COMPARING FILES
# Why compare files?
# Perhaps we've made changes to a machine learning model, but we're getting poorer performance as a result. We want to revert our changes, but can't remember what the code for the model looked like previously.
# Comparing an unstaged file with the last commit
# We can compare the last committed version of a file with the unstaged version by using the git diff command followed by the filename.
git diff report.md
# The output shows two versions of report.md, where a is the first version, or the last one to be saved, and b is the second, or the one we have not added to the staging area.
# diff --git a/report.md b/report.md
# index 6218b4e..066f447 100644
# --- a/report.md
# +++ b/report.md
# @@ -1,5 +1,5 @@ <-- (in blue) Line changes : -1.5 shows one line was removed at line five, and +1.5 shows one line was added back in at line five.
# # Mental Health in Tech Survey
# -TODO: write executive summary. <-- (in red) Removed lines
# TODO: include link to raw data.
# TODO: add references.
# +TODO: cite funding sources. <-- (in green) Added lines
# What if we had already added the file to the staging area? We can use the git diff command again, but this time we add the dash-r flag to indicate we want to look at a particular revision of the file. Adding HEAD, which is a shortcut for the most recent commit, allows us to see a difference between the report file in the staging area and the version in the last commit.
git add report.md
git diff -r HEAD report.md
# 'git diff -r' won't work if it isn't followed by 'HEAD'
# As we can see, the output gives us the same information as using git diff for a file that hasn't been added to the staging area!
# Comparing multiple staged files with the last commit
cd data
nano mh_tech_survey.csv
# to add an extra participant's survey responses
git add mh_tech_survey.csv
git diff -r HEAD
# In the output, we can see two files have been modified. One new line was added to the end of the mh_tech_survey.csv file, shown in green, along with the two changes to the report.
# Recap
# Compare an unstaged file with the last committed version:
# git diff filename
# Compare a staged file with the last committed version:
# git diff -r HEAD filename
# Compare all staged files with the last committed versions:
# git diff -r HEAD
# Exercise: What's going to be committed?
git diff -r HEAD .
git add report.md
git commit -m "New participant data and reminder for analysis"
# in three steps you compared all files to their last committed version, and updated report.md!2 - Making changes
# The commit structure
# Git commits have 'three' parts:
# 'Commit'
# contains the medadata
# 'Tree'
# tracks the names and locations in the repo
# 'Blob'
# binary large object
# may contain data of any kind
# compressed snapshot of a file's contents
# Visualizing the commit structure
# Here, we visualize three commits to our repo to see these three individual components.
# In the first commit, we can see a unique identifier ending in six-five. This identifier is known as a hash, which we will discuss later in the video. In the tree, we see two files were modified - report.md and mental_health_survey.csv. The blob shows a snapshot of what the files contained at that time.
# In the second commit, there are three files in the tree, but only two were modified - mental_health_survey.csv and a newly created summary_statistics.csv. Therefore, the tree links report.md to the blob from the previous commit, as it wasn't modified in this commit.
# In the third and most recent commit, report.md and mental_health_survey.csv are modified, with updated blobs created and linked to the tree. The summary statistics file wasn't changed, so the tree links to the blob in the second commit.
# Git log
# We can view commit information using the git log command, which will display all commits made to the repo in chronological order, starting with the oldest. It will show the commit hash, author, date, and commit message. If the output doesn't fit in the terminal window, there will be a colon at the end of the output, indicating there are more commits.
git log
# commit ad8accdmiqhf4a4g6a5r4g6arga6f4fgagjahjga
# Author: Rep Loop <[email protected]>
# Date: Wed Jul 27 07:48:27 2022 +0000
#
# Added reminder to cite funding sources.
# :
# Press 'space' to show more recent commits
# Press 'q' to quit the log and return to the terminal
# Git hash
# Earlier, we mentioned a unique identifier called a hash. This is a 40 character string of numbers and letters, like this:
# Last commit: b22eb5dfht4h5dg4qgq54fg54fg4q6g6f4g6q46gf45
# It is called a hash because Git produces it using a pseudo-random number generator called a 'hash function'.
# Hashes enable Git to share data efficiently between repos.
# If two files are the same, their hashes will be the same.
# Therefore, Git can tell what information needs to be saved in which location by comparing hashes rather than entire files.
# Finding a particular commit
# Suppose we have observed issues since a particular date. We need to see what changed in a commit on that day to cause the issues. First we'll need to identify which commit caused the issue, so we run git log to view all commits and note the hashes for commits on the day in question.
git log
# We only need to note the first six-to-eight characters of the 'hash'.
git show c27fa856
# We can then run git show followed by the start of the hash for each commit in turn, until we find the commit we are looking for.
# Git show output
# The output of git show displays the log entry for that commit at the top, followed by a diff output showing changes between the file in that commit and the current version in the repo.
# At the bottom we see the added line appears to have data in the wrong order, with gender in the first column instead of the second. This looks like the source of the issue!
# Exercise: Viewing a repository's history
git log
# the git log command is very useful for tracking changes at a high level before diving deeper into more specific version control tasks
# Exercise: Viewing a specific commit
git log
git show 36b761
# It's common to refer to the log and then perform a follow-up command to find a specific piece of information! Did you notice that one line was added to report.md in this particular commit?
# VIEWING CHANGES
# The HEAD shortcut
git diff -r HEAD
# Compares staged files to the version in the last commit
# Use a tilde '~' to pick a specific commit to compare versions
# 'HEAD~1' = the second most recent commit
# 'HEAD~2' = the commit before that
# Must not use spaces before or after the tilde ~
# From commit hash to HEAD
# 'Commit' 'HEAD'
#
# bac5332a .............. HEAD~2
# First commit
# | |
# | |
# v v
# ebe93178 .............. HEAD~1
# Second-to-last
# commit
# | |
# | |
# v v
# a845edcb .............. HEAD
# Last commit
# Using HEAD with git show
git show HEAD~3
# to look at the fourth most recent commit.
# The output shows the commit hash, author, date, log message, and diff.
# We can see that three lines were added to the report-dot-md file.
# What changed between two commits?
# 'git show' is useful for viewing changes made in a particular commit
# 'git diff' compares changes between two commits
# To compare the fourth and third most recent commits
git diff 35f4b4d 186398f
# or
git diff HEAD~3 HEAD~2
# diff --git a/report.md b/report.md
# index 35f4b4d..186398f 100644
# --- a/report.md
# +++ b/report.md
# @@ -1.3 +1.4 @@ <-- Add a fourth line
# # Mental Health in Tech Survey
# TODO: write executive summary.
# TODO: include link to raw data.
# +TODO: remember to cite funding sources! <-- Contents of the new line
# The output shows that a fourth line was added at the end of the report-dot-md file in the more recent of the two commits, with the contents of the line shown at the bottom of the output.
# Changes per document by line
# Suppose we want to see who made the last change to each line of a file, and when the change took place. We can use the git annotate command followed by the filename.
git annotate report.md
# 'Hash' 'Author' 'Time' 'Line #' 'Line Content'
# | | | | |
# | | | | |
# v v v v v
# 39aec30d ( Rep Loop 2022-07-06 14:36:44 1)# Mental Health in Tech Survey
# 39aec30d ( Rep Loop 2022-07-06 14:36:44 1)TODO: write executive summary.
# 39aec30d ( Rep Loop 2022-07-06 14:36:44 1)TODO: include link to raw data.
# 3bd310f7 ( Rep Loop 2022-07-06 14:36:45 1)TODO: remember to cite funding sources!
# Notice that three lines were added in one commit and one more line added in a separate commit.
# This may not seem particularly useful in this instance, but if we are working as part of a large team where multiple people are editing a file, then git annotate is a quick way to see who added a specific line of content and when.
# Summary
# 'Command' 'Function'
# git show HEAD~1 Show what changed in the second most recent commit
# git diff 35f4b4d 186398f Show changes between the commits
# git diff HEAD~1 HEAD~2 Show changes between the commits
# git annotate file Show line-by-line changes and associated metadata
# Exercise: Comparing to the second most recent commit
git log
git show 1182c2
# Awesome work—combining git show with a commit hash or HEAD plus the required ~ is a great way to see what happened in a specific commit!
# Exercise: Comparing commits
git diff HEAD~3 HEAD~1
# By using 'git diff' you get an output showing that both of these files have been changed between these two commits!
# UNDOING CHANGES BEFORE COMMITTING
# We've learned several ways to compare files, but what if we need to undo changes? Let's look at some scenarios where this may occur and how we can use Git commands to resolve the problem.
# Unstaging a file
# Suppose there are four files in our repo and we've been modifying three of them. We want to save two of them and continue working on the third file, summary_statistics.csv, before we save it. In this case, we don't want to commit that file yet. But what if we accidentally add this file to the staging area?
# In this scenario, we need to remove summary_statistics.csv from the staging area, or, to put it another way, we need to unstage the file so it is back in the repo.
# Making a commit
# Once the file has been unstaged, we can then commit the two modified files from the staging area.
# Committing the third file
# We are then free to update summary_statistics.csv, add it back to the staging area, and make another commit just for this file.
# Unstaging a single file in Git
# To unstage a single file:
git reset HEAD summary_statistics.csv
# We can then edit our file,
nano summary_statistics.csv
# restage it, or add it back into the staging area,
git add summary_statistics.csv
# and make a commit for that file only, including a log message.
git commit -m "Adding age summary statistics"
# Unstaging all files
# To unstage all files:
git reset HEAD
# Undo changes to an unstaged file
# Let's look at another common scenario. If we need to undo changes to a file, let's say summary_statistics.csv, that we haven't added to the staging area, we can do this using the git checkout command followed by two dashes and the filename. This reverts the file back to the version in the last commit.
git checkout -- summary_statistics.csv
# 'checkout' means switching to a different version,
# Defaults to the version in the last commit.
# This means losing all changes made to the unstaged file 'forever'
# Undo changes to all unstaged files
# If, rather than undoing changes to one unstaged file, we need to undo changes to all unstaged files in our repo, we can use git checkout as before, with one slight difference. Instead of the two dashes and the filename we add a period
git checkout .
# '.' refers to the current directory when used in a shell command
# Undo changes to all unstaged files in the current directory and subdirectories
# This command must be run in the main directory i.e., mh_survey
# Unstaging and undoing
# To unstage one or more files and then undo changes we can combine the commands we have learned.
# So, first we unstage all files using git reset HEAD.
git reset HEAD
# Then we execute git checkout dot to checkout the last commit, replacing all versions of files in the repo.
git checkout .
# To save the repo in this state we then need to add all files to the staging area,
git add .
# and make a commit.
git commit -m "Restore repo to previous commit"
# Exercise: How to unstage a file
git reset HEAD mental_health_survey.csv
echo "41,M,Yes,No,No,No,Often,Yes" >> mental_health_survey.csv
git add mental_health_survey.csv
git commit -m "Extra participant"
# knowing how to unstage files is helpful if you need to make more edits before saving! Now let's look at how to restore a previous version of an unstaged file.
# Exercise: Undoing changes to unstaged files
git checkout -- report.md
# git checkout is a powerful command for undoing changes made to unstaged files! But what if you need to undo changes to all files in the repository?
# Exercise: Undoing all changes
git reset HEAD
git checkout .
# in two commands you moved files from the staging area back into the repo and restored their state to versions in the last commit! Now let's go one step further by looking at how to restore previous versions of files and repositories, along with removing untracked files.
# RESTORING AND REVERTING
# So far we've looked at undoing changes prior to making commits. Now we'll look at how to customize our log commands as our projects scale, restore files to previous versions from particular commits, and delete all files that are not being tracked by Git
# Project scale
# 'git log' is useful to find the commit we want to revert to
# Larger project = more commits = larger output
# To illustrate this point, here is a project with only eight commits. We can imagine even at this scale it becomes difficult to pinpoint the commit we are looking for!
# Customizing the log ouput
# We can restrict the number of commits displayed using '-':
git log -3
# To only look at the commit history of one file:
git log -3 report.md
# Restrict 'git log' by date:
git log --since='Month Day Year'
# ex: Commits since 2nd April 2022:
git log --since='Apr 2 2022'
# Commits between 2nd and 11th April:
git log --since='Apr 2 2022' --until='Apr 11 2022'
# Restoring an old version of a file
# Suppose we want to restore the mental_health_survey.csv file to a version from the sixth of July when we added some new data. We can run git log and find the commit we need to restore the file version to.
git log --since='Jul 6 2022'
# commit dc9d8fac.......
# Author: Rep Loop <[email protected]>
# Date: Wed Jul 6 14:44:31 2022 +0000
#
# Adding fresh data for the survey
# We previously saw how git checkout can be used to revert to the version of a file in the last commit.
git checkout -- filename
# If we want to revert to a version of a file from a specific commit, then all we need to do is point to that commit. We can do this by replacing the two dashes with the git hash.
git checkout dc9d8fac mental_health_survey.csv
# Here, we restore the mental_health_survey.csv file to the version in the commit with the hash starting with dc9.
# Since this was the second to last commit, we can also revert the file using git checkout HEAD~1
git checkout HEAD~1 mental_health_survey.csv
# Restoring a repo to a previous state
git checkout dc9d8fac
# Alternatively
git checkout HEAD~1
# Cleaning a repository
# See what files are not being tracked:
# By definition, this means we have not saved them, so perhaps we created some files, or copied them in from other projects, and have now decided we don't need them
git clean -n
# Delete those files:
git clean -f
# BEWARE: 'git clean -f' cannot be undone!
# Exercise: Restoring an old version of a repo
git checkout 7f71eade
# Hopefully, you won't need to restore an entire repository to an earlier state, but it is worth knowing how to do this just in case things go drastically wrong!
# Exercise: Restoring an old version of a file
git log -2 report.md
git checkout 36b761 report.md
git add report.md
git commit -m "Restoring version from commit 36b761"
# by combining log, checkout, add, and commit you've been able to revert report.md and save it as the current version in Git! In the next chapter we will learn even more about Git, including the use of branches, and how to deal with conflicting versions of files!3 - Git workflows
# CONFIGURING GIT
# Why do we need to configure our settings?
# Git has customizable settings to speed up or improve how we work!
# Levels of settings
git config --list
# To access a list of customizable settings
# Git has three levels of settings:
# --local: settings for one specific project
# --global: settings for all of our projects
# --system: settings for every users on this computer
# Each level overrides the one above it, so local settings override global settings, and global settings override system settings
# Local settings
# |
# |
# v
# Global settings
# |
# |
# v
# System settings
# What we configure?
git config --list
# [email protected]
# user.name=Rep Loop
# core.editor=nano
# core.repositoryformatversion=0
# core.filemode=true
# core.bare=false
# core.logallrefupdates=true
# 'user.email' and 'user.name' are needed by some commands, so setting these saves time!
# Changing our settings
# We can modify a global setting using the syntax:
git config --global setting value
# Change email address to [email protected]
git config --global user.email [email protected]
# Change username to John Smith
git config --global user.name 'John Smith'
# If we don't use '' and our user.name has a space:
# Git would save user.name as John
# Using an alias
# Set up an alias through global settings
# Typically used to shorten a command
# To create an alias for committing files by executing 'ci':
git config --global alias.ci 'commit -m'
# Again we use '' so Git processes characters after the space
# We can now commit files by executing:
git ci 'any comment'
# Creating a custom alias
# We can create an alias for any command
# If we often unstage files:
git config --global alias.unstage 'reset HEAD'
# Be careful not to overwrite existing commands!
# Tracking aliases
# .gitconfig file
git config --global --list
# Output format: alias.aliascommand=command
# alias.ci=commit -m
# alias.unstage=reset HEAD
# Ignoring specific files
# Another helpful trick is to instruct Git to ignore certain files. We do this by creating a file called '.gitignore'
nano .gitignore
# *.log
# * = Wildcard
# --> Ignore all files with the extension .log
# Commonly ignored files: APIs, credentials, system files, software dependencies
# Exercise: Modifying your email address in Git
git config --list
# [email protected]
# user.name=Rep Loopcore.editor=nano
# core.repositoryformatversion=0
# core.filemode=true
# core.bare=false
# core.logallrefupdates=true
git config --global user.email [email protected]
git config --global --list
# using git config allowed you to add your cool new email address! Updating your settings as you get started with Git will save you a lot of time in the long run!
# Exercise: Creating an alias
git config --global alias.st status
git st
# Entering st instead of status might not seem like a big deal, but this small change will add up to substantial improvements in efficiency over time!
# BRANCHES
# Too many subdirectories
# If we are working locally and not using version control, it's common to create subdirectories to store different versions of files.
# We'll likely end up with extra files and sub-directories.
# This image shows a draft, final subdirectory, and final-v2 subdirectories! So it becomes difficult to locate the correct version of each file.
# Branches to the rescue!
# One of the reasons Git is so popular is that it uses branches, which helps avoid this issue of excessive subdirectories.
# Git uses 'branches' to systematically track multiple versions of files
# In each branch:
# Some files might be the same
# Others might be different
# Some may not exist at all
# (Each branch is like a parallel universe)
# Merging report into main
# Main |___| ------------------------------> |___| -------------> |___|
# | | |
# Analysis |-----> |___| --> |___| --> |___| ----| |
# | |
# Report |----> |___| --> |___| --> |___|-------|
# Source and destination
# When merging two branches:
# the commits are called parent commits
# 'source' - the branch we want to merge 'from'
# 'destination' - the branch we want to merge 'into'
# When merging 'Analysis' into 'Main':
# 'Analysis' = 'source'
# 'Main' = 'destination'
# Identifying branches
# We can see what branches exist for our project by executing the 'git branch' command in the terminal.
git branch
# alter-report-title
# main
# * summary-statistics
# * = current branch
# Creating a new branch
# We've decided we need another branch called report, where we will work on the project report. We want to create it from the summary-statistics branch, where we are currently located. We can create a new branch called report by executing the git checkout command with the dash-b flag followed by our branch name.
git checkout -b report
# Switched to a new branch 'report'
# The output shows Git has created the new branch and moved us into it.
# By executing git branch, we see the report branch listed with an asterisk.
git branch
# alter-report-title
# main
# summary-statistics
# * report
# The difference between branches
# Previously we've compared repos from different points in time. When working with branches, we will often want to see the difference between branches. We can rely on the git diff command for this too! We provide the names of the two branches we want to compare. In this case, we examine main versus summary-statistics.
git diff main summary-statistics
# This produces a standard git diff output as we've seen previously. This output shows that summary statistics were added in the second branch!
# diff --git a/bin/summary b/bin/summary
# new file mode 100755
# index 0000000..9d6e2fa
# --- /dev/null
# +++ b/bin/summary
# @@ -0,0 +1,44 @@
# +Summary statistics
# +
# +Age:
# +count: 49.00
# +mean: 31.82
# +std: 6.72
# +min: 18.00
# +25%: 28.00
# +50%: 31.00
# +75%: 35.00
# +max: 46.00
# Exercise: Creating new branches
echo "Mean age is 32 years old, with a standard deviation of 6.72" >> summary_statistics.txt
git add summary_statistics.txt
git commit -m "Adding age summary statistics"
# [summary-statistics d8dc8aa] Adding age summary statistics
# 1 file changed, 1 insertion(+)
# create mode 100644 summary_statistics.txt
git checkout -b report
# Switched to a new branch 'report'
# you modified a file in one branch before using git checkout to create a new branch called report and switch to it!
# Exercise: Comparing branches
git diff alter-report-title summary-statistics
# diff --git a/bin/summary b/bin/summary
# new file mode 100755index 0000000..9d6e2fa
# --- /dev/null
# +++ b/bin/summary
# @@ -0,0 +1,44 @@
# +Summary statistics
# +
# +Age:
# +count: 49.00
# +mean: 31.82
# +std: 6.72
# ...
# git diff allows you to compare files and branches! Looks like the summary statistics data have not been added in the alter-report-title branch.
# WORKING WITH BRANCHES
# Why do we need to switch branches?
# Common to work on different components of a project simultaneously
# Branches allow us to keep making progress concurrently
# Main |Current Code| |-->|Code |-->|Code |
# | | |Updated| |Updated|
# | | |
# Testing |--------> |Test 1| ----> |Test 2| ---> |Test 3| -|-> |Evaluate|------|
# | |
# debugging |---> |Analyse| --> |Update| --> |Test| ----------|
# How do we switch branches?
# to create a new branch
git checkout -b new_branch
# we can also use git checkout to switch to an existing branch by omitting the dash-b flag
git checkout debugging
# We can confirm we have successfully switched by running git branch, as the output provides an asterisk to indicate the branch we are in.
git branch
# * debugging
# main
# summary-statistics
# Nest step: merge
# we merge our debugging branch back into main and the debugged code is put into production
# After this, we finish evaluating our new code and, as it performs better than our current one, we merge the testing branch back into main too.
# Notice that in this diagram we merged the testing and debugging branches back into the main branch. But why?
# Why do we merge branches?
# 'main' = ground truth
# Each branch should be for a specific task
# Once the task is complete we should merge our changes into 'main'
# to keep it up to date and accurate
# Merging branches
# syntax
git merge source destination
# To merge 'summary-statistics' into 'main'
git merge summary-statistics main
# Merge output
# Updating dc9d8fa..cef5ad8
# Fast-forward
# bin/summary | 44 ++++++++++++++++++++++++++++++++++++++++
# results/summary.txt | 0
# 2 files changes, 44 insertions(+)
# create mode 100755 bin/summary
# create mode 100644 results/summary.txt
# The output shows the last two commit hashes from each branch at the top.
# Next is the type of merge. In this case it is a fast-forward merge, meaning additional commits were made on the summary-statistics branch, so Git brings the main branch up to date.
# The output also shows the number of lines added or deleted per file. In this case a subdirectory called summary was created with 44 extra lines.
# The summary.txt file in the results subdirectory was created but no lines were added,
# which is reiterated in the last two lines of the output showing the files modified.
# Exercise: Switching branches
git checkout reportSwitched to branch 'report'
echo "80% of participants were male, compared to the industry average of 67%." >> report.md
git add report.md
git commit -m "Add gender summary in report"
# [report f512064] Add gender summary in report
# 1 file changed, 1 insertion(+)
# in four commands you switched branches, edited a file, stored the draft, and updated the repo! This workflow enables you to work on the report in its respective branch!
# Exercise: Merging two branches
git merge report
# mainUpdating 7f71ead..0082272
# Fast-forward report.md | 1 +
# 1 file changed, 1 insertion(+)
# Unfortunately, merges aren't always straightforward as there can be conflicts between versions of files in different branches - let's look at how to deal with this scenario!
# HANDLING CONFLICT
# So far, we've been able to merge branches without any issues. However, this isn't always the case - sometimes, we will experience what Git calls a conflict. Let's discuss what a conflict is and how to deal with it.
# What is a conflict?
# A conflict occurs when a file in different branches has different contents that prevent them from automatically merging into a single version. For example, suppose we have a to-do list in the main branch with two tasks, as shown here.
# A) Write report.
# B) Submit report.
# We commit this file,
git add todo.txt
git commit -m "Add todo list"
# then, we switch to a new branch called update.
git checkout -b update
# We add a third task to the to-do list in the new update branch.
# A) Write report.
# B) Submit report.
# C) Submit expenses.
# After committing the modified file in the update branch, we switch back to main.
git add todo.txt
git commit -m "Reminder to submit expenses"
git checkout main
# We work away on our report and then submit it. We then decide to update our to-do list by removing the first two tasks we've completed. It now looks like this in the main branch
# C) Submit expenses.
# Conflict
# This file now has different versions in the main and update branches!
# Main branch 'todo.txt'
# C) Submit expenses.
# Update branch 'todo.txt'
# A) Write report.
# B) Submit report.
# C) Submit expenses.
# Attempting to merge a conflict
# Let's see what happens if we try to merge update into main.
git merge update main
# CONFLICT (add/add): Merge conflict in todo.txt
# Auto-merging todo.txt
# Automatic merge failed; fix conflicts and then commit the result.
# The output shows that Git attempted to figure out how to merge the branches but failed, so it asks us to fix the conflicts and then commit the result.
# Git conflicts
# Before merging, we need to follow Git's advice and resolve the conflict.
# We can resolve it by opening the conflicting file using the nano text editor.
nano todo.txt
# <<<<<<< HEAD
# =======
# A) Write report.
# B) Submit report.
# >>>>>>>
# C) Submit expenses.
# We see some extra content was added. This was actually put there by Git to indicate the difference between versions in the two branches.
# The first line starting with arrows pointing to the left and the word HEAD indicates that lines beneath it contain the file's contents in the latest commit of the current branch, in this case, main.
# The line with equal signs refers to the center of the conflict. As it comes straight after the first line, it indicates that the lines beneath are part of the file version in the latest commit of the current branch.
# However, if the equal signs came after some content, this would mean our files have different content on the same lines in different branches.
# The line with several arrows pointing to the right and the word update indicates additional content below that is in the update branch.
# Conflict web editor
# Here we take steps to resolve the conflict. We only want task C remaining, which is to submit our expenses. Therefore, we delete all of the other lines. Lastly, we save the modified file, which no longer contains any of the Git conflict syntax.
# C) Submit expenses.
# Merging the branches
# We add our updated to-do list to the staging area, then commit it to the main branch. Now, we can try to merge the update branch into main again.
git add todo.txt
git commit -m "Resolving todo.txt conflict"
git merge update main
# Already up to date
# As we've resolved the conflict, Git informs us that the two branches are already up to date, which is a relief!
# Large conflicts can be quite intimidating!
# We just saw how to deal with a minor conflict
# How do we avoid conflicts?
# Prevention is better than cure!
# the best approach is to lower the chances of conflicts occurring.
# Use each branch for a specific task
# Avoid editing a file in multiple branches
# Doesn't guarantee we'll avoid conflicts
# but it does reduce the risk4 - Collaborating with Git
# CREATING REPOS
# We know how to work with pre-existing Git repos, so let's look at how we can make new repos, and convert an existing project into a Git repo.
# Why make a repo?
# Benefits of repos
# Systematically track versions
# Collaborate with colleagues
# Git stores everything!
# Creating a new repo
# Imagine we've secured funding for a project following up on our mental health in tech companies project, which will compare mental health in the workplace across a range of sectors. Here, we create a new Git repo for this project using the git init command, followed by the name of our project called mental-health-workspace.
git init mental-health-workspace
# Git creates a new subdirectory in the directory we were located in when we ran the git init command, with the name we provided for the repo. We can use cd to change into this directory.
cd mental-health-workspace
# We can check that the Git repo has initialized correctly by running git status. Success! The output confirms we have a Git repo, and that no commits have been made yet.
git status
# On branch main
#
# No commits yet
#
# nothing to commit (create/copy files and use "git add" to track)
# Converting a project
# Now we know how to create a Git repo from scratch, let's look at how to convert an existing project into a repo. Let's say we started working on our new mental health project without first creating a new Git repo. We can convert the directory to a Git repo by executing git init. We run the command from our project directory, and Git confirms that an empty repo has been created along with providing the location.
git init
# Initialized empty Git repository in /home/repl/mental-health-workspace/.git/
# What is being tracked?
# Something interesting happens if we check the status of the repo. We can see Git has immediately recognized that there are modified files, in this case, the report-dot-md file, so we are encouraged to add our files to the staging area and make a commit!
git status
# On branch main
#
# No commits yet
#
# Untracked files:
# (use "git add <file>..." to include what will be committed)
#
# data/
# report.md
#
# nothing added to commit but untracked files present (use "git add" to track)
# Nested repositories
# Don't create a Git repo inside another Git repo
# Known as nested repos
# There will be two .git directories
# Which .git directory should be updated?
# Not necessary in most circumstances
# Exercise: Setting up a new repo
git init anxiety_workplace
# Initialized empty Git repository in /home/repl/projects/anxiety_workplace/.git/
cd anxiety_workplace/
echo "TODO: Recap on existing research." > todo.txt
# in just three lines of code you created a new Git repository, moved into its main directory, and created a to-do list!
# WORKING WITH REMOTE
# Now we're going to learn about a fundamental feature for collaboration in Git - remote repos, also known as remotes
# Local repo
# So far, we have worked with repos stored on our computer. These are known as local repos.
# They are great if we are working on a local project, but what if we need to collaborate with colleagues? How will they access our repo?
# Remote repo
# This is where remotes solve our problems! A remote repo is a repo stored in the cloud through an online repo hosting service such as GitHub.
# Why use remote repos?
# Benefits of remote repos
# Everything is backed up
# If our computer breaks down or we lose it, we can use a different computer to access our project from the remote repo as it is backed up there!
# Collaboration, regardless of location
# Cloning locally
# We can copy existing remotes to our local computer by cloning them. If required, we can also clone a local repo. To do this we use the git clone command followed by the path to the project directory.
git clone path-to-project-directory
# Here, we clone a local project from home-slash-john-slash-repo.
git clone /home/john/repo
# We can also give the cloned repo a name by specifying it after the path, such as here, where we call it new-repo
git clone /home/john/repo new_repo
# Cloning a remote
# Remote repos are stored in an online hosting service e.g., GitHub, BitBucket, or Gitlab
# If we have an account:
# we can clone a remote repo on to our local computer
git clone [URL]
# e.g.
git clone https://github.com/datacamp/project
# Identifying a remote
# When cloning a repo
# Git remembers where the original was
# Git stores a remote 'tag' in the new repo's configuration
# If we are in a repo, we can list the names of its remotes using git remote.
git remote
# datacamp
# The output confirms we have cloned the datacamp GitHub project
# Getting more information
# If we want more information, such as the remotes URLs, we can add the dash-v flag, which stands for verbose. We see two outputs. They contain the same URL, but have either fetch or pull at the end of their respective rows. We will discuss these terms later in the chapter.
git remote -v
# datacamp https://github.com/datacamp/project (fetch)
# datacamp https://github.com/datacamp/project (pull)
# Creating a remote
# When cloning, Git will automatically name the remote 'origin'
# We can add more remotes by specifying a name for Git to assign. We do this using the git remote add command, and provide two pieces of information - the name that we would like to assign to the remote repo, and the URL, or the path to the directory.
git remote add name URL
# Here, we create a remote called george, which points to the url github.com/george_datacamp/repo.
git remote add george https://github.com/george_datacamp/repo
# Defining remote names is useful for merging branches
# rather than listing the URL or path.
# Exercise: Cloning a repo
git clone /home/john/repo john_anxiety_project
# git clone is a very useful command for copying other repos onto your local computer, whether from another local directory or remote storage such as GitHub. Now you can build on the work John has done so far!
# Exercise: Defining and identifying remotes
git remote add john /home/john/repo
git remote -v
# john /home/john/repo (fetch)
# john /home/john/repo (push)
# Adding the name john will save you time when accessing or merging the cloned repo! Now let's learn how to keep files in sync between local and remote branches.
# GATHERING FROM A REMOTE
# We've seen how to clone remote repos, now let's look at how to get content from a remote into our local repo.
# Remote vs. local
# Say we have been working in a branch of our local repo, but others have been working in the remote. Here, we can see the contents of the two repos, and it is clear that there are additional files and subdirectories in the remote.
# Collaborating on Git projects
# If several people are collaborating on a project then, in practice, they will access the remote, work on files locally, save them, and synchronize their changes between the remote and local repos.
# This means that the remote repo should be the source of truth for the project, where the latest versions of files that are not drafts can be located.
# Fetching from a remote
# To compare the files in a remote against the contents of a local repo we first need to fetch versions from the remote. We do this using the git fetch command, providing the name of the remote, and the local branch to fetch into. Here we fetch from the origin remote.
git fetch origin main
# From https://github.com/datacamp/project
# * branch main -> FETCH_HEAD
# The output shows us the URL or path we are fetching from, the branch, which is main, and that we are fetching the HEAD, or last commit.
# If we want to fetch into a different branch, we specify it. For example, here, we fetch from the remote origin repo's report branch.
git fetch origin report
# Synchronizing content
# After fetching, we now have the contents of the remote in our local repo.
# However, we need to synchronize contents between the two repos. To do this, we perform a merge of the remote into the local repo's main branch, where we are currently located.
git merge origin main
# Updating 9dcf4e5..887da2d
# Fast-forward
# data/mental_health_survey.csv | 3 +++
# report.md | 1 +
# 2 files changed, 4 insertions (+)
# The output shows the two commit hashes, followed by the type of merge. In this case it is a fast-forward, meaning the local repo was behind the remote, and this merge aligned it.
# We see two files were changed, mental_health_survey.csv had three lines added, and report.md had one line added. The last line summarizes the changes.
# Pulling from a remote
# 'remote' is often ahead of 'local' repos
# 'fetch' and 'merge' is a common workflow
# Git simplifies this process for us!
git pull origin main
# Here, we pull from origin into our local main branch.
# The output confirms this is a combination of the two commands, with the git fetch output shown in the first two lines, and the git merge output displayed in the remaining five lines!
# From https://github.com/datacamp/project
# * branch main -> FETCH_HEAD
# Updating 9dcf4e5..887da2d
# Fast-forward
# data/mental_health_survey.csv | 3 +++
# report.md | 1 +
# 2 files changed, 4 insertions (+)
# Pulling with unsaved local changes
# If we have been working locally and not yet committed our changes, then Git won't allow us to pull from a remote.
# Let's say we've added a new line to the report but not staged the file or made a commit.
# If we try to pull from origin then Git tells us that local changes would be overwritten.
# We are instructed to commit our changes and told that the pull command was aborted
git pull origin
# Updating 9dcf4e5..887da2d
# error: Your local changes to the following files would be overwritten by merge: report.md
# Please commit your changes or stash them before you merge.
# Aborting
# Important to save locally before pulling from a 'remote'
# Exercise: Fetching from a remote
git remote
# origin
git fetch origin main
# From /home/john/repo
# * branch main -> FETCH_HEAD
git diff origin main
# diff --git a/report.md b/report.md
# index 09640b5..186398f 100644
# --- a/report.md
# +++ b/report.md
# @@ -2,4 +2,3 @@
# TODO: write executive summary.
# TODO: include link to raw data.
# TODO: remember to cite funding sources!
# -TODO: add references.
# Looks like John added a reminder about references in report.md, as the local main branch doesn't have this content in the file.
# Exercise: Pulling from a remote
git pull origin main
# From /home/john/repo
# * branch main -> FETCH_HEADAlready up to date.
echo "No existing mental health diagnosis." >> protocol.md
git add protocol.md
git commit -m "Updating eligibility criteria"
# [master 2d0f871] Updating eligibility criteria
# 1 file changed, 1 insertion(+)
# create mode 100644 protocol.md
# Notice you received the output 1 file changed, 1 insertion(+), highlighting that the remote was ahead of your local repo!
# PUSHING TO A REMOTE
# We've seen how to bring our local branch up to date with a remote repo, but what about bringing our local changes into a remote repo?
# Pulling from a remote
# To recap, we use git pull to gather content from a remote and merge into our local repo.
# Pushing to a remote
# The opposite of this task is merging content from a local repo into a remote.
# Therefore, the opposite of pulling means that we push our changes to the remote.
# 'Remote Repo' 'Remote Repo'
# | ^
# Pull | Push |
# | |
# v |
# 'Local Repo' 'Local Repo'
# git push
# Save changes locally first!
# As with pulling contents from a remote, we need to commit local changes before we can push to the remote.
# Once we have done this, we can push to a remote using the git push command, supplying two arguments. First we provide the remote, followed by the local branch.
git push remote local_branch
# The order that we supply the remote repo and the local branch is the same as git pull. So, we can think of it as pushing content into the remote repo from the local branch. Here, we push changes into origin from our local main branch.
git push origin main
# Push/pull workflow
# The typical push/pull workflow is as follows. We start by pulling the remote into our local repo.
# We then work on our project locally, committing changes as we go.
# Lastly, we push our updated local repo to the remote. This workflow is repeated throughout our time working on the project.
# 'Remote Repo'
# | ^
# Pull | | Push
# | |
# v |
# 'Local Repo'
# Pushing first
# What happens if we don't start the workflow by pulling from the remote? Here, we try to push the contents of our local repo's main branch to the remote.
git push origin main
# To https://github.com/datacamp/project
# ! [rejected] main -> main (non-fast-forward)
# error: failed to push some refs to 'https://github.com/datacamp/project'
# hint: Updates were rejected because the tip of your branch is behind
# hint: its remote counterpart. Integrate the remote changes (e.g.
# hint: 'git pull ..') before pushing again.
# hint: See the 'Note about fast-forwards' in 'git push --help' for details.
# The first line shows us the remote repo URL we are trying to push to.
# The second and third lines show us the outcome of the command - it was rejected and failed.
# In the third to seventh lines, Git provides hints. This is where Git tells us why the command failed, and recommends what we should do to resolve the issue.
# In this case, we see that the tip, or the end, of our current branch is behind its remote counterpart and the suggestion is to pull from the remote before trying to push our local changes again. This can typically occur because while we've been working locally, our colleagues have been pushing their changes to the remote. So, if we don't pull from the remote at the start of the workflow then our local repo won't be synchronized with the remote.
# Resolving a conflict
# Previously we saw that git pull used a fast-forward merge.
# In this case it's slightly different, as there are different commits on the local and remote repos, so Git can't just bring them in line with one another.
# So, when we try to pull, Git will automatically open the nano text editor and ask us to add a message for the merge. We leave a message that we are pulling the latest report from the remote.
# We use Control and O to save, then Control and X to exit as usual.
# Avoid leaving a message
# We can avoid providing a message by including --no-edit when executing git pull, however, this is not recommended!
git pull --no-edit origin main
# Resolving a conflict
# After saving our message, we see that the merge has been made using a recursive strategy, and report.md was updated.
# Merge made by the 'recursive' strategy.
# report.md | 1 +
# 1 file changed, 1 insertion(+)
# Default merge strategy for Git v0.99.9 - v2.33.0
# Pushing to the remote
# As this change has now been incorporated locally, we can try to push our changes again.
git push origin main
# Counting objects: 3, done.
# Delta compression using up to 8 threads.
# Compressing objects: 10% (3/3), done.
# Writing objects: 100% (3/3), 325 bytes | 325.00 KiB/s, done.
# To https://github.com/datacamp/project
# 01384d2..0a1dbf6 main -> main
# The output provides a lot of information, but the key takeaway is that the changes have been written to the remote repo. The final line shows the commit hashes referenced for merging the local repos's main branch into the remote's main branch.
# Exercise: Pushing to a remote repo
git add .
git commit -m "Budget inaccuracy added to the issue log and report"
# [main f9c696f] Budget inaccuracy added to the issue log and report 2 files changed, 4 insertions(+)
# create mode 100644 issue_log.txt
git push origin main
# Counting objects: 4, done.
# Delta compression using up to 8 threads.
# Compressing objects: 100% (4/4), done.
# Writing objects: 100% (4/4), 496 bytes | 496.00 KiB/s, done.
# Total 4 (delta 1), reused 0 (delta 0)
# To /home/john/repo
# ebcea98..f9c696f main -> main
# Now you know how to pull changes from a remote, work locally, and push your changes back to the remote! But what happens if the remote is ahead of the local repo?
# Exercise: Handling push conflicts
git push origin main
# To /home/john/repo
# ! [rejected] main -> main (fetch first)
# error: failed to push some refs to '/home/john/repo'
# hint: Updates were rejected because the remote contains work that you do
# hint: not have locally. This is usually caused by another repository pushinghint: to the same ref. You may want to first integrate the remote changeshint: (e.g., 'git pull ...') before pushing again.
# hint: See the 'Note about fast-forwards' in 'git push --help' for details.
git pull origin main
# remote: Counting objects: 4, done.
# remote: Compressing objects: 100% (4/4), done.
# remote: Total 4 (delta 1), reused 0 (delta 0)
# Unpacking objects: 100% (4/4), done.
# From /home/john/repo
# * branch main -> FETCH_HEAD
# f9c696f..a9587cc main -> origin/main
# Removing issue_log.txt
# Merge made by the 'recursive' strategy.
# issue_log.txt | 2 --
# proposal.md | 2 ++
# report.md | 2 --
# 3 files changed, 2 insertions(+), 4 deletions(-)
# delete mode 100644 issue_log.txt
# create mode 100644 proposal.md
git push origin main
# Counting objects: 5, done.
# Delta compression using up to 8 threads.
# Compressing objects: 100% (4/4), done.
# Writing objects: 100% (5/5), 731 bytes | 731.00 KiB/s, done.
# Total 5 (delta 0), reused 0 (delta 0)
# To /home/john/repo
# a9587cc..7f0545f main -> main
# you've learned how to use git push and git pull to deal with a remote repo that has different content to your local repo. Let's quickly recap on what we've covered in the course!
# WRAP UP
# What you've covered
# What a version is
# Why version control is important
# Using Git to:
# save files
# compare files
# How Git stores data
# Configuration
git config --global alias.unstage 'reset HEAD'
# You also saw how to recognize Git's conflict syntax to handle merge conflicts!
# Lastly, you learned how to clone and synchronize content across local and remote repos - a fundamental skill for collaboration!
# Git cheat sheet
# https://www.datacamp.com/cheat-sheet/git-cheat-sheet