Check requirements.txt against existing packages

Given pip requirements.txt file, is there a command to check which Python packages there are already packed for Fedora ?

1 Like

FPM is somehow able to detect Fedora packages from Python dependencies.

https://fpm.readthedocs.io/en/latest/source/python.html

It doesn’t detect anything, near as I can tell. It just reads in requirements.txt and generates its internal dependency list, then blindly munges that dependency data into Requires: lines.

(There’s even some special-casing in rpm.rb for rubygem dependencies, specifically:)

# Convert 'rubygem-foo' provides values to 'rubygem(foo)'
# since that's what most rpm packagers seem to do.

There’s nothing I can find in the code that verifies the actual existence of those dependencies as available packages, and their names will only be correct if the packager followed the same formula in generating the package name.

1 Like

All Python packages provide a standardized name. So given a name in the file, you can look up the equivalent Provides with:

$ rpm -E '%{py3_dist a-name-from-the-list}'
python3dist(a-name-from-the-list)

and you can see if a package exists by checking:

$ dnf repoquery --whatprovides 'python3dist(a-name-from-the-list)'
# Should return nothing since this is a made up name.

So with a bit of scripting around the requirements list, you should be able to determine whether all the packages are available.

2 Likes

Is there any way to query Provides: info from the web?

Up to now I found only .spec https://apps.fedoraproject.org/packages/python-straight-plugin/sources/spec/ which doesn’t include this field.

repoquery --provides didn’t work for https://pypi.org/project/straight.plugin/ which is https://apps.fedoraproject.org/packages/python-straight-plugin

➜  ~ sudo dnf repoquery --provides 'python3dist(straight.plugin)'
Last metadata expiration check: 0:36:53 ago on Fri 23 Aug 2019 05:14:25 AM +03.
➜  ~

Sorry, made a mistake there; --provides tells you what a package provides. To go from a provide back to a package name, use --whatprovides.

It works!

➜  ~ sudo dnf repoquery --whatprovides 'python3dist(straight.plugin)' -q
python3-straight-plugin-0:1.5.0-6.fc30.noarch

I used https://repology.org/ to lookup Fedora 30 package names for PyPI requirements file. This has an advantage that it doesn’t require to be run from local Fedora.

https://colab.research.google.com/drive/1AGzKal4EFdtZidCT23ANvYKk-fkO83Ft#scrollTo=lQOCMnI_03I4

Repology still can not correctly detect Python package names if they do not fit the pattern used for Fedora package names. I can not find where dnf repoquery --whatprovides get the value of Provides: header. I traced the call to hawkey C library and can not get further.

I found needed into in /var/cache/dnf/fedora-8c9e363c32277c01/repodata/912f062d93e096c75901055ffca02a0c3961b33b8e1dd65319d97d493d3e49d5-primary.xml.gz. Now I need to understand how to fetch that from the web.

<rpm:entry name="python3dist(straight.plugin)" flags="EQ" epoch="0" ver="1.5.0"/>