I want a command which works in this way but in case of python packages:
dnf repoquery --userinstalled
I am looking for a command which looks something like this:
pip list --userinstalled
I’m in the cleaning process where I’m removing packages that are idle (stale packages which are no longer used/required). I’m looking for ways that help me to see all pip packages which are installed manually by me in the past for testing propose.
You really should be using virtual environments when installing packages using pip so that they don’t interfere with your system packages. When you don’t need a virtual environment, you just delete it and whatever was installed in there is gone. No need to find a list using pip and all that.
If you haven’t been using virtual environments and have been using pip but without sudo, pip will generally install user packages in ~/.local/ (see man pip-install). If you’ve been using sudo, pip has installed stuff in the system locations: /usr/lib/pythonX.Y and /usr/lib64/pythonX.Y, and there’s no easy way of “undoing” any changes it has made to packages you installed using dnf. You’ll have to manually find the packages, then remove them using pip, and if that doesn’t break dnf, reinstall the system packages using dnf. If it breaks dnf, the simplest thing to do is to backup your data and reinstall afresh.
Without virtual environments pip knows about all packages installed in both the default system and user locations, and it cannot tell the difference between stuff you installed using pip and stuff you installed using dnf. dnf, however, only installs packages in system wide locations. So, for packages installed with pip using --user, you can use the corresponding command to get a complete list:
pip freeze --user
This will not include system wide packages (ones installed with sudo and without --user).
If you did install packages system wide with pip, you can get the list of python packages installed by pip and dnf and then compare the two:
# get all Python packages pip can find in the default paths
$ pip freeze | cut -d "=" -f1 | sort -h | uniq
aiodns
aiofiles
aiohttp
aiohttp-cors
aiohttp-socks
airspeed
...
# get list of all dnf installed python packages
$ rpm --qf '%{NAME}\n' -qa python3\* | cut -d "-" -f2 | sort -h | uniq
abrt
aiodns
aiofiles
aiohttp
airspeed
apipkg
appdirs
argcomplete
argon2
astroid
async
atomicwrites
attrs
audit
augeas
...
I’d just save these to files (add > filename.txt at the end of the commands to save to filename.txt) and then use diff to see what’s in one and not in the other:
diff file1 file2
If you’ve been using sudo, pip has installed stuff in the system locations
usr/lib/python2.7/site-packages🔒
⟹ ls
usr/lib/python3.9/site-packages🔒
⟹ ls
chardet py_swtpm_localca pygments
swtpm_localca-0.5.2-py3.9.egg-info torbrowser_launcher
packaging py_swtpm_setup
requests swtpm_setup-0.5.2-py3.9.egg-info
Python2 doesn’t have any packages installed from pip. But Python3 does seem to have some of the packages. Are they okay/normal packages, or should I uninstall them all?
[I will post a follow-up-comment about your comment’s second part later in few hours by sorting the packages names.]
is this in the system packages or in your user packages (in ~/.local/lib)?
dnf will never install anything into your ~/, so these will all be from pip. you can remove them without any worries. dnf only installs in system locations.
I was looking at the pip freeze man page, and saw that it also has a --user option. That should tell you what’s installed by pip in your user directories.
pip freeze --user
The sorting etc. commands above is only required if you’ve used sudo to install in the system wide locations. (I’ve also edited my initial answer to note this now).
That –user option is a gem. Thanks a lot.
But why do pip freeze --user when we can do pip list --user which makes more sense!
Anyway thanks.
is this in the system packages or in your user packages
Abovels command output is used inside “usr/lib/python2.7/site-packages” and “usr/lib/python3.9” directory. So they are system directories. I forget to put usr path which I have fixed it now.
A new thing that I didn’t see is these massive packages inside python3.10 directory:
⟹ ls
__pycache__ firewall pyenchant-3.2.2-py3.10.egg-info
_distutils_hack fros-1.1-py3.10.egg-info pyfros
_version.py gi pygments
abrt3.pth gtweak Pygments-2.9.0.dist-info
abrt_exception_handler3.py html2text pygtkcompat
abrtcli html2text-2020.1.16-py3.10.egg-info pyOpenSSL-21.0.0-py3.10.egg-info
aiodns html5lib pyparsing-2.4.7.dist-info
aiodns-3.0.0-py3.10.egg-info html5lib-1.1-py3.10.egg-info pyparsing.py
aiofiles humanize PySocks-1.7.1-py3.10.egg-info
aiofiles-0.7.0.dist-info humanize-0.5.1-py3.10.egg-info python_augeas-1.1.0-py3.10.egg-info
aiohttp_cors idna python_dateutil-2.8.1-py3.10.egg-info
aiohttp_cors-0.7.0-py3.10.egg-info idna-3.2-py3.10.egg-info python_utils
argcomplete ifaddr python_utils-2.5.6-py3.10.egg-info
argcomplete-1.12.3-py3.10.egg-info ifaddr-0.1.7-py3.10.egg-info pytz
async_timeout init_calibre.py pytz-2021.3-py3.10.egg-info
async_timeout-3.0.1-py3.10.egg-info jeepney pyxdg-0.27-py3.10.egg-info
attr jeepney-0.7.1-py3.10.egg-info raven
attrs-21.2.0-py3.10.egg-info jinja2 raven-6.10.0-py3.10.egg-info
augeas Jinja2-3.0.1-py3.10.egg-info requests
augeas.py jsonschema requests-2.26.0.dist-info
babel jsonschema-3.2.0-py3.10.egg-info requests_gssapi
Babel-2.9.1-py3.10.egg-info koji requests_gssapi-1.2.3-py3.10.egg-info
beaker koji-1.26.1.dist-info rpmautospec
Beaker-1.10.0-py3.10.egg-info koji_cli rpmautospec-0.2.5-py3.10.egg-info
beautifulsoup4-4.9.3-py3.10.egg-info louis rpmconf
bs4 mako rpmconf-1.1.4-py3.10.egg-info
chardet Mako-1.1.4-py3.10.egg-info seobject.py
chardet-4.0.0.dist-info markdown sepolgen
charset_normalizer Markdown-3.3.4-py3.10.egg-info sepolicy
charset_normalizer-2.0.4.dist-info mechanize sepolicy-3.3-py3.10.egg-info
chrome_gnome_shell-0.0.0-py3.10.egg-info mechanize-0.4.5-py3.10.egg-info setuptools
click nftables setuptools-57.4.0.dist-info
click-8.0.1.dist-info nftables-0.1-py3.10.egg-info sgmllib.py
configobj-5.0.6-py3.10.egg-info odf sgmllib3k-1.0.0-py3.10.egg-info
configobj.py odfpy-1.4.1-py3.10.egg-info six-1.16.0.dist-info
cpuinfo olefile six.py
css_parser olefile-0.46-py3.10.egg-info socks.py
css_parser-1.0.6-py3.10.egg-info OpenSSL sockshandler.py
cupshelpers orca sos
cupshelpers-1.0-py3.10.egg-info packaging sos-4.2-py3.10.egg-info
dateutil packaging-21.0.dist-info soupsieve
decorator-5.0.9-py3.10.egg-info paste soupsieve-2.3.1-py3.10.egg-info
decorator.py Paste-3.5.0-py3.10-nspkg.pth tempita
defusedxml Paste-3.5.0-py3.10.egg-info Tempita-0.5.2-py3.10.egg-info
defusedxml-0.7.1-py3.10.egg-info pexpect test
distro-1.6.0-py3.10.egg-info pexpect-4.8.0-py3.10.egg-info torbrowser_launcher
distro.py pip torbrowser_launcher-0.3.5-py3.10.egg-info
distutils-precedence.pth pip-21.2.3.dist-info typing_extensions-3.7.4.3-py3.10.egg-info
dnf pkg_resources typing_extensions.py
dnf-plugins ply urllib3
dnfpluginscore ply-3.11-py3.10.egg-info urllib3-1.26.6-py3.10.egg-info
dnfpluginsextras progressbar validate.py
dns progressbar2-3.53.2-py3.10.egg-info webencodings
dnspython-2.1.0-py3.10.egg-info ptyprocess webencodings-0.5.1-py3.10.egg-info
enchant ptyprocess-0.6.0-py3.10.egg-info xdg
fedora_third_party py_cpuinfo-8.0.0-py3.10.egg-info zeroconf
fedora_third_party-0.8-py3.10.egg-info pyatspi zeroconf-0.36.9-py3.10.egg-info
feedparser pycparser
feedparser-6.0.6-py3.10.egg-info pycparser-2.20-py3.10.egg-info
⟹
But none of these packages are installed by me. Because Python3.10 came with the release of Fedora 35, AKAICS.
Another good thing is there is no user-installed packages that I see when I type:
⟹ pip list --user
~
⟹
Which seems a bit weird. But it is what it is.
pip list --user lists only packages installed in your user space, that is, the packages in $HOME/.local/lib/python3.x/site-packages. The fact that you have none means that you haven’t used pip install --user to install any packages. But that’s a fairly different thing than how your question started out.
As far as packages in /usr/lib*/python3.10/site-packages/, while you can use pip to examine those, you shouldn’t use pip to manage them at all, because those packages are under the control of rpm. (By the same token, you should never sudo pip install ____ anything, as you’ll be messing up the system-controlled package collection.)
Pip does have a --not-required flag to list, which will exclude packages that are required by another package. However, most of the packages in /usr/lib*/python3.x/site-packages/ will show up as not-required, because they aren’t required by another Python package… that doesn’t mean they aren’t required by one or more RPMs. Again, pip isn’t really the tool for managing the system package collection.
Note, also, that dnf repoquery --userinstalled doesn’t completely mean what it appears to. Packages are automatically marked as “user installed” by dnf if they were installed by direct request of the user, and not marked if they were pulled in as dependencies. But any package can be marked as user-installed by running sudo dnf mark install <package>, and any package can be marked as non-user-installed by running sudo dnf mark remove <package>.
Those flags are used by the dnf autoremove system, which ignores any package listed as user-installed (along with any package that’s a dependency of another package) when computing which packages can be autoremoved.
If you wanted to remove all packages that aren’t required by another package from /usr/lib*/python3.x/site-packages, the easiest way would be to run sudo dnf mark remove python3-\* – that will unmark ALL Python RPMs as being user-installed, which means that any which aren’t required by another RPM will now be on the autoremove chopping block. (You can see what’s auto-removable by running sudo dnf list autoremove, and remove them with sudo dnf autoremove.)
Pip doesn’t have a concept of “user-installed” vs. not, because every package is user-installed as far as pip is concerned. The “system” package collection is totally a construct of the RPM packaging system, which is why those packages should be managed the same way.
A post was split to a new topic: Python: venvs vs --user installs