Resolving Module Version Chaos: Locking Down Dependencies in Python Projects with Poetry

July 13, 2024

This content originally appeared on DEV Community and was authored by Mazen Alotaibi

Hey there! I’ve got a nifty trick to share about managing Python dependencies, especially when they’re not version-locked. Let me walk you through how I tackled it using Poetry.

Problem

Ever faced a requirements.txt that looks like this?

tqdm
matplotlib

No version numbers can be a recipe for chaos during builds or at runtime due to inconsistencies. I needed to lock these dependencies to specific versions to keep things smooth and reliable, like this:

tqdm==4.64.0
matplotlib==3.5.3

Solution

Why Poetry?

I chose Poetry because it’s like the npm of the Python world—it respects semantic versioning and creates a lock file so every install is consistent. No more “works on my machine” issues!

Step-by-Step Guide

1) Install Poetry:

   curl -sSL https://install.python-poetry.org | python3 -

2) Grab a simple `pyproject.toml` template:

   wget https://gist.githubusercontent.com/ma7dev/7298ffc4409032edd4d18a57b4c38f3a/raw/1c32efcbde31aaf896c6d47b32dac19ed44d14a4/pyproject.toml

3) Install those unversioned dependencies:

   cat requirements.txt | xargs poetry add

4) Export the installed dependencies in a more structured format:

   poetry export -f requirements.txt --output long_requirements.txt --without-hashes

5) Clean up the exported file:

   # Strip unwanted python version constraints
   cat long_requirements.txt | cut -d ";" -f 1 > with_dep_requirements.txt
   # Filter out extraneous dependencies
   cat requirements.txt | while read line   do echo $(grep -n $line'==' with_dep_requirements.txt | cut -d ":" -f 2) >> final_requirements.txt done

Result

Here’s what you end up with, all dependencies neatly versioned (final_requirements.txt):

tqdm==4.64.0
matplotlib==3.5.3
... (rest of your dependencies)

This setup ensures that all packages are locked to specific versions, making your project stable and reproducible wherever it goes.

Explore and Connect with Me! Discover my thoughts on career and personal growth on Medium, deep dives into software and tech on Hashnode, and quick tech tips on Dev.to. Follow my journey and join the conversation on Twitter or LinkedIn. For project updates and coding streams, check out GitHub and Twitch. Let’s connect and create together!