Affiliate disclosure

Book titles on this page link to Amazon. As an Amazon Associate, DataField.Dev earns from qualifying purchases — at no additional cost to you.

Further Reading: Libraries and Virtual Environments

Virtual Environments and Package Management

  • Python Documentation: "Virtual Environments and Packages" Tutorial. (Tier 1) The official Python tutorial on venv and pip. Concise, authoritative, and always up-to-date with the latest Python version. Start here for any questions about venv syntax or behavior. https://docs.python.org/3/tutorial/venv.html

  • Python Documentation: venv — Creation of Virtual Environments. (Tier 1) The full module reference for venv, including advanced options like --system-site-packages, --clear, and --upgrade-deps. More detailed than the tutorial. https://docs.python.org/3/library/venv.html

  • Python Packaging User Guide: "Installing Packages." (Tier 1) The official guide from the Python Packaging Authority (PyPA). Covers pip, virtual environments, requirements.txt, and pyproject.toml with clear examples. This is the canonical reference for Python packaging. https://packaging.python.org/en/latest/tutorials/installing-packages/

  • Python Packaging User Guide: "Managing Application Dependencies." (Tier 1) Walks through managing an application's dependencies with Pipenv, a higher-level tool that combines a virtual environment with a lock file, and points to alternatives such as pip-tools and Poetry. Read it once you're comfortable with plain pip and venv. https://packaging.python.org/en/latest/tutorials/managing-dependencies/

  • Python Packaging User Guide: "Writing your pyproject.toml." (Tier 1) A practical, section-by-section guide to the [build-system] and [project] tables used in this chapter's pyproject.toml example, including dependencies and optional dependencies. https://packaging.python.org/en/latest/guides/writing-pyproject-toml/

Dependency Management Tools

  • pip Documentation: User Guide. (Tier 1) The complete reference for pip commands, configuration, and behavior. Especially useful: the sections on requirements file format, version specifiers, and dependency resolution. https://pip.pypa.io/en/stable/user_guide/

  • pip-tools Documentation. (Tier 2) pip-tools provides pip-compile (generates pinned requirements.txt from a requirements.in with loose constraints) and pip-sync (ensures your environment matches the requirements file exactly). A step up from basic pip for serious projects. https://pip-tools.readthedocs.io/

  • Poetry Documentation. (Tier 2) Poetry is an alternative to pip + venv that handles virtual environment creation, dependency resolution, and package publishing in one tool. Its pyproject.toml-based workflow is increasingly popular in the Python community. https://python-poetry.org/docs/

  • Conda Documentation: "Managing Environments." (Tier 2) Conda is the package manager used in the Anaconda distribution, popular in data science and scientific computing. Unlike pip, conda can install non-Python dependencies (C libraries, R packages). Worth knowing if you're heading into data science. https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html

Semantic Versioning

  • semver.org. (Tier 1) The specification for Semantic Versioning 2.0.0. Short, clear, and definitive. If you want to understand exactly what MAJOR.MINOR.PATCH means and the rules for incrementing each, this is the source. https://semver.org/

  • Python Documentation: PEP 440 — Version Identification and Dependency Specification. (Tier 2) Python's version numbering scheme, which is compatible with but slightly different from strict semver. Covers pre-releases, post-releases, dev releases, and the version specifiers used in requirements.txt and pyproject.toml. https://peps.python.org/pep-0440/

Supply Chain Security

  • Birsan, A. (2021). "Dependency Confusion: How I Hacked Into Apple, Microsoft, and Dozens of Other Companies." (Tier 1) The blog post that introduced the "dependency confusion" attack to the world. Accessible writing with concrete examples. Eye-opening for anyone who uses pip. https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610

  • The PyPI Blog. (Tier 1) The official blog of the Python Package Index. PyPI's administrators and safety team write here about protections such as Trusted Publishing and mandatory two-factor authentication, analyses of real malware attacks, and yearly reviews. Understanding these protections helps you evaluate the trustworthiness of packages. https://blog.pypi.org/

  • pip-audit. (Tier 2) A tool maintained by the Python Packaging Authority that scans your environment or requirements.txt for packages with known security vulnerabilities. Install it with pip install pip-audit. https://pypi.org/project/pip-audit/

  • OpenSSF Scorecard. (Tier 2) Automated health checks for open-source projects (Is the project maintained? Does it review code? Does it pin its own dependencies?). A useful companion to the package evaluation checklist in section 23.6.1. https://scorecard.dev/

Key Libraries

  • Requests Documentation. (Tier 1) Beautifully written documentation for the most popular Python HTTP library. The "Quickstart" guide is one of the best examples of library documentation in the Python ecosystem. https://requests.readthedocs.io/

  • pandas Documentation: "Getting Started Tutorials." (Tier 1) A gentle introduction to pandas DataFrames. Start with the "10 Minutes to pandas" tutorial if you want a quick overview, or work through the full getting-started series for a thorough introduction. https://pandas.pydata.org/docs/getting_started/

  • matplotlib: "Tutorials." (Tier 1) Matplotlib's official tutorials, organized from introductory to advanced. The "Pyplot tutorial" covers the basics you need for most visualizations. https://matplotlib.org/stable/tutorials/

  • Rich Documentation. (Tier 1) The rich library's documentation includes a gallery of output examples, making it easy to see what's possible. The "Introduction" page is a great starting point. https://rich.readthedocs.io/

  • Flask Tutorial: "Quickstart." (Tier 2) Flask's quickstart guide walks you through creating your first web application in about 20 minutes. A natural next step if web development interests you. https://flask.palletsprojects.com/en/latest/quickstart/

The Left-Pad Incident

  • Collins, K. (2016). "How one programmer broke the internet by deleting a tiny piece of code." Quartz. (Tier 1) Accessible reporting on the left-pad incident, including the trademark dispute that started it. Clear, non-technical writing that captures the absurdity and the lessons. https://qz.com/646467/how-one-programmer-broke-the-internet-by-deleting-a-tiny-piece-of-code

  • Williams, C. (2016). "How one developer just broke Node, Babel and thousands of projects in 11 lines of JavaScript." The Register. (Tier 1) Same-day technical reporting with more detail about the dependency chains involved. Good for understanding the cascade effect. https://www.theregister.com/2016/03/23/npm_left_pad_chaos/

For the Curious

  • Eghbal, N. (2020). Working in Public: The Making and Maintenance of Open Source Software. Stripe Press. (Tier 2) A readable study of who actually maintains the open-source packages everyone depends on, and why so many critical projects rest on a handful of volunteers. Great context for the "bus factor" discussion in Case Study 2. https://press.stripe.com/working-in-public

  • Spotify Engineering (2023). "Fleet Management at Spotify (Part 1): Spotify's Shift to a Fleet-First Mindset." (Tier 2) How Spotify uses automation to keep the dependencies of thousands of repositories up to date. The source for the Spotify figures in Case Study 1. https://engineering.atspotify.com/2023/4/spotifys-shift-to-a-fleet-first-mindset-part-1

  • Python Documentation: PEP 518 — Specifying Minimum Build System Requirements. (Tier 2) The PEP that introduced pyproject.toml, originally to declare which tools are needed to build a project; PEP 621 later added the [project] table for metadata and dependencies. Technical but important if you want to understand why pyproject.toml has largely replaced setup.py. https://peps.python.org/pep-0518/