Chapter 33 Exercises: Reproducibility and Collaboration: Git, Environments, and Working with Teams

How to use these exercises: Part A tests conceptual understanding of version control and reproducibility. Part B is hands-on — you will actually use git, create environments, and write documentation. Part C involves team simulation and workflow design. Part D explores broader topics.

Difficulty key: ⭐ Foundational | ⭐⭐ Intermediate | ⭐⭐⭐ Advanced | ⭐⭐⭐⭐ Extension

Important: Many of these exercises require you to use the command line. If you get stuck on a git command, git help <command> will show you the documentation, and git status will always tell you where you are.

Part A: Conceptual Understanding ⭐

Exercise 33.1 — Why reproducibility matters

For each scenario, explain why reproducibility is critical and what could go wrong without it:

1. A pharmaceutical company uses a data science model to determine drug dosages.
2. A student submits a data analysis for a class project, and the professor wants to verify the results.
3. A data scientist leaves a company, and their replacement needs to update a quarterly report.
4. A research paper claims that a new treatment reduces hospital readmissions by 15%.
5. A machine learning model was trained six months ago and now needs to be retrained with new data.

Exercise 33.2 — Git vocabulary ⭐

Match each term with its definition:

Exercise 33.3 — Identifying reproducibility problems ⭐

For each code snippet, identify the reproducibility problem:

1.

df = pd.read_csv('C:/Users/alex/Desktop/data/vaccination_data.csv')

2.

train, test = train_test_split(data, test_size=0.2)

3.

import pandas  # no version specified anywhere in the project

4.

# I removed some outliers here
df = df[df['rate'] < 100]

5.

results = model.fit(X, y)
# model accuracy: 0.87 (I wrote this down on a sticky note)

Exercise 33.4 — The .gitignore file ⭐

For each file or directory, decide whether it should be tracked by git (YES) or excluded via .gitignore (NO), and explain why:

1. analysis.ipynb
2. data/raw/census_2020.csv (500MB)
3. requirements.txt
4. .env (contains API keys)
5. __pycache__/
6. README.md
7. venv/ (virtual environment directory)
8. results/figures/vaccination_trend.png
9. .ipynb_checkpoints/
10. src/data_utils.py

Part B: Hands-On Practice ⭐⭐

Exercise 33.5 — Initialize a repository ⭐⭐

Create a new directory for a practice project, initialize a git repository, and make your first three commits:

1. Commit 1: Add a README.md with a project title and one-sentence description
2. Commit 2: Add a .gitignore file that excludes Python bytecode and Jupyter checkpoints
3. Commit 3: Add a simple Python script or notebook that prints "Hello, reproducible world!"

After all three commits, run git log --oneline and verify you see three commits with descriptive messages.

# Create and enter the project directory
mkdir practice-project
cd practice-project

# Initialize git
git init

# Create and commit the README
echo "# Practice Project" > README.md
echo "A practice repository for learning git." >> README.md
git add README.md
git commit -m "Add README with project description"

# Create and commit the .gitignore
echo "__pycache__/" > .gitignore
echo ".ipynb_checkpoints/" >> .gitignore
echo "*.pyc" >> .gitignore
git add .gitignore
git commit -m "Add .gitignore to exclude Python bytecode and Jupyter checkpoints"

# Create and commit a simple script
echo 'print("Hello, reproducible world!")' > hello.py
git add hello.py
git commit -m "Add initial hello script"

# Verify
git log --oneline

Exercise 33.6 — Branching and merging ⭐⭐

Starting from the repository you created in Exercise 33.5:

1. Create a branch called feature/add-analysis
2. On that branch, add a new file called analysis.py that creates a small pandas DataFrame and prints its summary statistics
3. Commit the new file on the branch
4. Switch back to main and verify that analysis.py does not exist there
5. Merge the branch into main
6. Verify that analysis.py now exists on main
7. Delete the branch (it has been merged, so it is no longer needed)

# Create and switch to the branch
git checkout -b feature/add-analysis

# Create the analysis file
cat > analysis.py << 'EOF'
import pandas as pd

data = {'city': ['New York', 'Chicago', 'Houston', 'Phoenix'],
        'population': [8336817, 2693976, 2320268, 1680992],
        'vaccination_rate': [82.3, 78.1, 71.5, 68.9]}

df = pd.DataFrame(data)
print(df.describe())
EOF

# Commit
git add analysis.py
git commit -m "Add analysis script with city vaccination data summary"

# Switch to main and verify analysis.py is not there
git checkout main
ls analysis.py  # Should show "No such file"

# Merge
git merge feature/add-analysis

# Verify analysis.py exists
ls analysis.py  # Should show the file

# Delete the branch
git branch -d feature/add-analysis

Exercise 33.7 — Creating a virtual environment ⭐⭐

1. Create a new virtual environment (using either conda or venv)
2. Install pandas, numpy, and matplotlib
3. Generate a requirements.txt file
4. Deactivate the environment
5. Delete the environment
6. Recreate it using only the requirements.txt file
7. Verify that the same packages are installed

# Create
conda create --name test-env python=3.11 -y
conda activate test-env

# Install
conda install pandas numpy matplotlib -y

# Save
pip freeze > requirements.txt

# Deactivate and remove
conda deactivate
conda env remove --name test-env

# Recreate
conda create --name test-env python=3.11 -y
conda activate test-env
pip install -r requirements.txt

# Verify
pip list

# Create
python -m venv test-env
source test-env/bin/activate  # or test-env\Scripts\activate on Windows

# Install
pip install pandas numpy matplotlib

# Save
pip freeze > requirements.txt

# Deactivate and remove
deactivate
rm -rf test-env

# Recreate
python -m venv test-env
source test-env/bin/activate
pip install -r requirements.txt

# Verify
pip list

Exercise 33.8 — Writing a README ⭐⭐

Write a README.md for a hypothetical data science project that analyzes air quality data from a city's sensor network. Include all essential sections:

1. Project title and one-paragraph description
2. Key findings (make up 3 plausible findings)
3. Setup instructions (prerequisites, installation, data download)
4. Project structure (directory tree)
5. Usage instructions (how to run the analysis)
6. License and author information

Exercise 33.9 — Setting random seeds ⭐⭐

Write a Python script that demonstrates the importance of random seeds:

1. Without a seed, generate two random samples and show they are different
2. With a seed, generate two random samples and show they are identical
3. Show that train_test_split produces different results without random_state and identical results with it

import numpy as np
from sklearn.model_selection import train_test_split

# Part 1: Without seed
sample_1 = np.random.randn(5)
sample_2 = np.random.randn(5)
print("Without seed:")
print(f"  Sample 1: {sample_1}")
print(f"  Sample 2: {sample_2}")
print(f"  Are they equal? {np.array_equal(sample_1, sample_2)}")

# Part 2: With seed
np.random.seed(42)
sample_3 = np.random.randn(5)
np.random.seed(42)
sample_4 = np.random.randn(5)
print("\nWith seed:")
print(f"  Sample 3: {sample_3}")
print(f"  Sample 4: {sample_4}")
print(f"  Are they equal? {np.array_equal(sample_3, sample_4)}")

# Part 3: train_test_split
X = np.arange(20).reshape(10, 2)
y = np.arange(10)

# Without random_state (run twice to see different results)
_, X_test_1, _, _ = train_test_split(X, y, test_size=0.3)
_, X_test_2, _, _ = train_test_split(X, y, test_size=0.3)
print(f"\nSplit without seed - same? {np.array_equal(X_test_1, X_test_2)}")

# With random_state
_, X_test_3, _, _ = train_test_split(X, y, test_size=0.3, random_state=42)
_, X_test_4, _, _ = train_test_split(X, y, test_size=0.3, random_state=42)
print(f"Split with seed - same? {np.array_equal(X_test_3, X_test_4)}")

Run this script and verify the output matches your expectations.

Exercise 33.10 — Viewing git diffs ⭐⭐

In your practice repository:

1. Modify an existing file (change a number, fix a typo, add a line)
2. Run git diff to see the unstaged changes
3. Stage the file with git add
4. Run git diff --staged to see the staged changes
5. Commit with a descriptive message
6. Run git log -1 --stat to see what was changed in the last commit

Describe what information each command provides and when you would use it.

Exercise 33.11 — Undoing mistakes in git ⭐⭐

Practice these common "undo" operations in your practice repository:

1. Unstage a file: Stage a file, then remove it from the staging area without losing the changes
2. Discard changes: Modify a file, then discard the modifications (return to the last committed version)
3. Amend a commit message: Make a commit with a typo in the message, then fix the message

For each, write the git commands you used.

# Stage a file
git add some_file.py
# Unstage it (changes remain in working directory)
git restore --staged some_file.py

# Modify a file, then discard modifications
git restore some_file.py
# WARNING: This permanently discards your changes!

# Make a commit with a typo
git commit -m "Aded analysis script"
# Fix the message (only use for unpushed commits!)
git commit --amend -m "Added analysis script"

Part C: Team Simulation ⭐⭐⭐

Exercise 33.12 — Simulating a merge conflict ⭐⭐⭐

Create a merge conflict on purpose and resolve it:

1. Create a file with a few lines of text and commit it to main
2. Create a branch called feature-a and modify line 3
3. Switch to main and modify the same line 3 differently
4. Try to merge feature-a into main
5. Resolve the conflict by choosing the correct version
6. Complete the merge with a commit

# Setup: create and commit a file
cat > data_config.py << 'EOF'
# Data configuration
DATA_DIR = "data/"
OUTPUT_FORMAT = "csv"
RANDOM_SEED = 42
EOF
git add data_config.py
git commit -m "Add data configuration file"

# Branch A: change OUTPUT_FORMAT
git checkout -b feature-a
# (Edit data_config.py to change OUTPUT_FORMAT = "parquet")
git add data_config.py
git commit -m "Change output format to parquet for better performance"

# Back to main: change the same line differently
git checkout main
# (Edit data_config.py to change OUTPUT_FORMAT = "json")
git add data_config.py
git commit -m "Change output format to json for web compatibility"

# Try to merge — this will conflict
git merge feature-a

<<<<<<< HEAD
OUTPUT_FORMAT = "json"
=======
OUTPUT_FORMAT = "parquet"
>>>>>>> feature-a

git add data_config.py
git commit -m "Resolve conflict: use parquet format for performance (see PR discussion)"

Exercise 33.13 — Pull request simulation ⭐⭐⭐

If you have a GitHub account, practice the pull request workflow:

1. Create a new repository on GitHub
2. Clone it locally
3. Create a branch, make changes, and push the branch
4. Open a pull request on GitHub
5. Add a description explaining what changed and why
6. (If working with a partner) Have them review and approve
7. Merge the pull request on GitHub
8. Pull the merged changes locally

If you do not have a GitHub account, write out the steps you would take and explain what each step accomplishes.

# Clone
git clone https://github.com/yourusername/practice-repo.git
cd practice-repo

# Create branch
git checkout -b feature/add-data-cleaning

# Make changes (add a file, edit code, etc.)
echo "# Data Cleaning Script" > clean_data.py
git add clean_data.py
git commit -m "Add data cleaning script skeleton"

# Push branch to remote
git push -u origin feature/add-data-cleaning

git checkout main
git pull origin main

Exercise 33.14 — Code review practice ⭐⭐⭐

Review the following code as if it were submitted in a pull request. For each issue you find, explain the problem and suggest a fix. Look for reproducibility problems, unclear code, and missing documentation.

import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split

df = pd.read_csv('/home/sarah/Desktop/project/data.csv')

# clean
df = df.dropna()
df = df[df.age > 0]
df = df[df.age < 120]

X = df[['age', 'income', 'zipcode']]
y = df['purchased']

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier(n_estimators=100)
model.fit(X_train, y_train)
print(model.score(X_test, y_test))

Exercise 33.15 — Writing a CONTRIBUTING.md ⭐⭐⭐

Write a CONTRIBUTING.md file for a team data science project. Include:

1. How to set up the development environment
2. The branching strategy (what to name branches, when to create them)
3. Commit message format requirements
4. The pull request process (who reviews, how long to wait)
5. Code style conventions (naming, comments, notebook formatting)
6. How to report bugs or request features

# Contributing to [Project Name]

## Environment Setup
1. Clone the repository
2. Create a conda environment: `conda env create -f environment.yml`
3. Activate: `conda activate project-name`

## Branching Strategy
- Create a branch for each feature or fix
- Name branches: `feature/description`, `fix/description`, `docs/description`
- Never commit directly to `main`

## Commit Messages
- Start with a verb: Add, Fix, Update, Remove, Refactor
- Keep the first line under 72 characters
- Reference issue numbers when applicable: "Fix #42: correct rate calculation"

## Pull Requests
- Include a description of what changed and why
- Request review from at least one team member
- Address all review comments before merging
- Reviewers should respond within 2 business days

## Code Style
- Python: follow PEP 8
- Notebooks: use Markdown cells to narrate; clean up before committing
- Variable names: descriptive (`vaccination_rate`, not `vr`)
- Set random seeds: `np.random.seed(42)` at the top of every notebook

## Issues
- Use GitHub Issues to report bugs or request features
- Include: expected behavior, actual behavior, steps to reproduce

Part D: Synthesis and Extension ⭐⭐⭐–⭐⭐⭐⭐

Exercise 33.16 — Full project setup ⭐⭐⭐

Set up a complete, reproducible project from scratch:

1. Create a project directory with the standard structure (notebooks, data, src, results)
2. Initialize a git repository
3. Create a virtual environment and install necessary packages
4. Generate a requirements.txt
5. Write a .gitignore
6. Write a README
7. Create a simple analysis notebook with random seeds set
8. Make three well-messaged commits (initial setup, add analysis, add documentation)
9. Verify the entire project can be set up from scratch by a new user

This exercise brings together everything in the chapter.

Exercise 33.17 — Reproducibility forensics ⭐⭐⭐

You receive a Jupyter notebook from a colleague with these issues: - No requirements.txt or environment.yml - Cells run out of order (cell 5 depends on cell 12) - Absolute file paths for data - No random seeds - Results are described in text ("accuracy was about 87%") but not produced by visible code - Several cells produce warnings that are not addressed

Write a plan to make this notebook reproducible. List every specific action you would take, in order.

Exercise 33.18 — Git history detective ⭐⭐⭐

Using a public GitHub repository (any popular open-source project), explore the git history:

1. How many commits does the repository have?
2. Find a commit message that is particularly informative. What makes it good?
3. Find a commit message that is uninformative. What would improve it?
4. Find a pull request with code review comments. What did the reviewer catch?
5. Look at the .gitignore file. What types of files are excluded?

Write brief answers for each. This exercise builds your ability to read and navigate repository history.

Exercise 33.19 — The reproducibility report card ⭐⭐⭐⭐

Go back to an analysis you completed earlier in this course (any chapter's exercise or project milestone). Grade it on reproducibility using this rubric:

Score each criterion: 0 = not done, 1 = partially done, 2 = mostly done, 3 = fully done.

Then: pick the two lowest-scoring criteria and fix them. Document what you changed.

Exercise 33.20 — Environment debugging ⭐⭐⭐

Your colleague sends you a project with a requirements.txt. When you run pip install -r requirements.txt, you get errors because:

1. One package has been removed from PyPI
2. Two packages have version conflicts (Package A requires numpy<1.24, Package B requires numpy>=1.25)
3. One package requires a C compiler that is not installed on your system

For each problem, describe what the error message might look like and propose a solution.

Exercise 33.21 — Reproducibility across programming languages ⭐⭐⭐⭐

Research how reproducibility and environment management work in ONE of the following ecosystems (choose one that is NOT Python):

1. R (renv, packrat, Docker)
2. Julia (Pkg, Project.toml)
3. JavaScript/Node.js (package.json, npm)

Write a 200-word comparison with Python's approach. What is similar? What is different? What could Python learn from the other ecosystem?

Exercise 33.22 — Docker for reproducibility ⭐⭐⭐⭐

Research Docker containers and write a 150-word explanation of how they solve reproducibility problems that virtual environments cannot. Specifically address:

1. What is a Docker container?
2. How does it differ from a virtual environment?
3. What reproducibility problems does it solve that requirements.txt alone cannot?

Exercise 33.23 — Data versioning ⭐⭐⭐⭐

Git tracks code changes well, but data changes are harder. Research ONE tool for data versioning:

1. DVC (Data Version Control)
2. Git LFS (Large File Storage)
3. Delta Lake

Write a 150-word summary of how the tool works and when you would use it.

Exercise 33.24 — Team workflow design ⭐⭐⭐⭐

You are leading a data science team of four people working on a six-month project. Design a complete workflow covering:

1. Repository structure
2. Branching strategy
3. Code review process
4. Environment management
5. Documentation standards
6. How to handle large datasets
7. Meeting cadence and communication tools

Write this as a one-page team agreement (about 400 words).

Exercise 33.25 — The reproducibility pledge ⭐

Write a three-sentence personal commitment to reproducibility. What specific practices will you adopt from this chapter, starting with your next project?

This is a reflection exercise — there is no wrong answer, but be specific.

Reflection

The tools in this chapter — git, virtual environments, documentation — are not exciting. They do not produce cool visualizations or impressive models. But they are the infrastructure that makes everything else possible. Without version control, you lose work. Without environment management, your code breaks. Without documentation, your analysis is a mystery even to you.

The best time to adopt these practices was at the beginning of the course. The second best time is now. Every project from this point forward should start with git init, a virtual environment, and a README.