Dnf info returns for only one of local or COPR repo, not both?

pgfed · June 30, 2020, 6:24pm

I’ve built/installed a into both a local-only repo and a COPR repo

my relevant repo files are

/etc/yum.repos.d/nginx-mainline-local.repo
	[nginx-mainline-local]
	name=nginx-mainline-local
	baseurl=file:///usr/local/LOCAL_REPO/nginx-mainline
	enabled=1
	module_hotfixes=1
	gpgcheck=0
	repo_gpgcheck=0
	priority=90

/etc/yum.repos.d/_copr:copr.fedorainfracloud.org:pgfed:nginx-mainline.repo
	[copr:copr.fedorainfracloud.org:pgfed:nginx-mainline]
	name=Copr repo for nginx-mainline owned by pgfed
	baseurl=https://download.copr.fedorainfracloud.org/results/pgfed/nginx-mainline/fedora-$releasever-$basearch/
	type=rpm-md
	skip_if_unavailable=True
	gpgcheck=1
	gpgkey=https://download.copr.fedorainfracloud.org/results/pgfed/nginx-mainline/pubkey.gpg
	repo_gpgcheck=0
	enabled=1
	enabled_metadata=1
	priority=80

both repos are enabled

dnf repolist nginx-mainline-local copr:copr.fedorainfracloud.org:pgfed:nginx-mainline
	repo id                                                        repo name                                              status
	copr:copr.fedorainfracloud.org:pgfed:nginx-mainline            Copr repo for nginx-mainline owned by pgfed            enabled
	nginx-mainline-local                                           nginx-mainline-local                                   enabled

I can see ‘info’ re: my available package in the local repo

dnf --disablerepo='*' --enablerepo=nginx-mainline-local info compat-luarocks  --refresh 
	Last metadata expiration check: 1:07:16 ago on Tue 30 Jun 2020 08:14:01 AM PDT.
	Available Packages
	Name         : compat-luarocks
	Version      : v3.3.1
	Release      : 1.fc32
	Architecture : x86_64
	Size         : 153 k
	Source       : compat-luarocks-v3.3.1-1.fc32.src.rpm
	Repository   : nginx-mainline-local
	Summary      : Lua Module Deployment & Management
	URL          : https://github.com/luarocks/luarocks
	License      : MIT
	Description  : Lua Module Deployment & Management

and in the COPR repo

dnf --disablerepo='*' --enablerepo=copr:copr.fedorainfracloud.org:pgfed:nginx-mainline info compat-luarocks --refresh
	Copr repo for nginx-mainline owned by pgfed                                                  9.8 kB/s | 3.3 kB     00:00
	Available Packages
	Name         : compat-luarocks
	Version      : v3.3.1
	Release      : 1.fc32
	Architecture : src
	Size         : 5.1 M
	Source       : None
	Repository   : copr:copr.fedorainfracloud.org:pgfed:nginx-mainline
	Summary      : Lua Module Deployment & Management
	URL          : https://github.com/luarocks/luarocks
	License      : MIT
	Description  : Lua Module Deployment & Management

	Name         : compat-luarocks
	Version      : v3.3.1
	Release      : 1.fc32
	Architecture : x86_64
	Size         : 153 k
	Source       : compat-luarocks-v3.3.1-1.fc32.src.rpm
	Repository   : copr:copr.fedorainfracloud.org:pgfed:nginx-mainline
	Summary      : Lua Module Deployment & Management
	URL          : https://github.com/luarocks/luarocks
	License      : MIT
	Description  : Lua Module Deployment & Management

BUT, if I simply check for ‘–all’ available packages,

dnf info compat-luarocks --all --refresh

it only returns info for the COPR repo,

Available Packages
Name         : compat-luarocks
Version      : v3.3.1
Release      : 1.fc32
Architecture : src
Size         : 5.1 M
Source       : None
Repository   : copr:copr.fedorainfracloud.org:pgfed:nginx-mainline
Summary      : Lua Module Deployment & Management
URL          : https://github.com/luarocks/luarocks
License      : MIT
Description  : Lua Module Deployment & Management

Name         : compat-luarocks
Version      : v3.3.1
Release      : 1.fc32
Architecture : x86_64
Size         : 153 k
Source       : compat-luarocks-v3.3.1-1.fc32.src.rpm
Repository   : copr:copr.fedorainfracloud.org:pgfed:nginx-mainline
Summary      : Lua Module Deployment & Management
URL          : https://github.com/luarocks/luarocks
License      : MIT
Description  : Lua Module Deployment & Management

not ‘all’ including the local repo, which is still clearly enabled & populated.

what ‘dnf’ syntax, add’l step &/or config, is required to return the intended “-all” available packages?

ankursinha · July 1, 2020, 10:14am

I don’t think DNF will list all the instances of the same package with the same version if they are served by different repositories. This is not quite the use case for DNF. DNF picks the best candidate. In this scenario, since the COPR repo is at a lower priority, the package will be picked from there. It’s explained in man dnf.conf.

If the package versions differ, DNF should list both. What is the use case for you providing the same package, with the same version from two repositories (and manually creating a conflict)?

pgfed · July 1, 2020, 1:42pm

use case

Simply, I build it ‘here’ locally for my occassional use, and promote it to COPR for wider use.

If the package versions differ, DNF should list both

I can play some games with that, I suppose; in my usual workflow, that’s basically the opposite of what I do. Perhaps tacking on a build id/# … will explore.

Coming to Fedora, I’m just figuring out what’s doable.

The ‘DNF picks the best candidate’ carries with it an assumption. Namely, what ‘best’ is – as opposed to ‘all the info’. I’m used to the capabilities of OBS/Zypper. WIth those tools I’m able to specify any number of repos, containing any packages of any version, and have zypper query/list their available name/arch/version/etc. In that scenario ‘best’ is what I say it is – either by repo priority, or by explicit selection by version, or repo to install from. Only works well when you can see all the info easily.

ankursinha · July 1, 2020, 2:35pm

What happens when your local repo and COPR have different versions? (dnf should list all available versions)

A repository can have multiple versions of packages, and if the exact versions are provided to dnf, it will install them without issues. (I haven’t had to do this for a long time, but I don’t see why this would be an issue) In your particular case, I expect only one copy of the package is being shown because the two copies have the same version-release, which tells dnf that they are identical. So, here dnf chooses the repo with the lower priority setting.

What is your workflow—why does the package in the local repo, which I expect is a newer build than the version in the COPR repo, have the same version-release? Your local repo should always have a newer version of the package (with your improvements)—every new “build” should have an increased release number which ensures this:

https://docs.fedoraproject.org/en-US/packaging-guidelines/#changelogs

When you think the improved version is ready to share, you push that to the COPR repository as an update. At that point, you don’t need the same package from the local repo (even though they should be pretty identical).

pgfed · July 2, 2020, 12:08am

no, not in this particular case …

The current goal was certainly same version – initially built/tested locally, and then subsequently pushed to COPR for builds

As you pointed out version+release needs to be different.

Fwiw, this now works:

i added an ‘early’ timestamp %define to the spec,

	%define _owner pgnd
-->	%define _build_timestamp %( date +%Y%m%d_%H%M%S | head -c 15 )
	...
	%forgemeta -i -a
	%global dist %{_owner}_%{_build_timestamp}.fc%{fedora}

	Name:              %{luarocks_name}
	Release:           %{dist}
	...

after a successful local rpmbuild, mv’d rpm results to LOCAL repo

	/usr/bin/cp -af \
	 ~/rpmbuild/{RPMS/x86_64,SRPMS}/compat-luarocks*.rpm \
	 /usr/local/LOCAL_REPO/nginx-mainline/

	tree /usr/local/LOCAL_REPO/nginx-mainline -L 1
		/usr/local/LOCAL_REPO/nginx-mainline
		├── compat-luarocks-v3.3.1-pgnd_20200701_155718.fc32.src.rpm
		├── compat-luarocks-v3.3.1-pgnd_20200701_155718.fc32.x86_64.rpm
		└── repodata

updated it

	createrepo --database /usr/local/LOCAL_REPO/nginx-mainline

pushed the same spec to COPR,

	copr-cli build nginx-mainline compat-luarocks.spec
		...
		16:02:39 Build 1516025: succeeded

enabled the copr repo,

	dnf -y copr enable pgfed/nginx-mainline

& set relative repo priorities, LOCAL ‘higher’,

	dnf config-manager --save --setopt="nginx-mainline-local.priority=80"
	dnf config-manager --save --setopt="copr:copr.fedorainfracloud.org:pgfed:nginx-mainline.priority=90"

Checking ‘info’ shows the higher-priority, LOCAL repo’s pkg contents as expected,

	dnf info compat-luarocks --refresh

		Available Packages
		Name         : compat-luarocks
		Version      : v3.3.1
		Release      : pgnd_20200701_161621.fc32
		Architecture : src
		Size         : 5.1 M
		Source       : None
		Repository   : nginx-mainline-local
		Summary      : Lua Module Deployment & Management
		URL          : https://github.com/luarocks/luarocks
		License      : MIT
		Description  : Lua Module Deployment & Management

		Name         : compat-luarocks
		Version      : v3.3.1
		Release      : pgnd_20200701_161621.fc32
		Architecture : x86_64
		Size         : 153 k
		Source       : compat-luarocks-v3.3.1-pgnd_20200701_161621.fc32.src.rpm
		Repository   : nginx-mainline-local
		Summary      : Lua Module Deployment & Management
		URL          : https://github.com/luarocks/luarocks
		License      : MIT
		Description  : Lua Module Deployment & Management

That’s good.

Next, switching repo priorities

	dnf config-manager --save --setopt="nginx-mainline-local.priority=90"
	dnf config-manager --save --setopt="copr:copr.fedorainfracloud.org:pgfed:nginx-mainline.priority=80"

Checking ‘info’ again, I see the expected, NOW-higher-priority COPR repo’s packages,

	dnf info compat-luarocks --refresh
		Available Packages
		Name         : compat-luarocks
		Version      : v3.3.1
		Release      : pgnd_20200701_234521.fc32
		Architecture : src
		Size         : 5.1 M
		Source       : None
		Repository   : copr:copr.fedorainfracloud.org:pgfed:nginx-mainline
		Summary      : Lua Module Deployment & Management
		URL          : https://github.com/luarocks/luarocks
		License      : MIT
		Description  : Lua Module Deployment & Management

		Name         : compat-luarocks
		Version      : v3.3.1
		Release      : pgnd_20200701_234529.fc32
		Architecture : x86_64
		Size         : 153 k
		Source       : compat-luarocks-v3.3.1-pgnd_20200701_234529.fc32.src.rpm
		Repository   : copr:copr.fedorainfracloud.org:pgfed:nginx-mainline
		Summary      : Lua Module Deployment & Management
		URL          : https://github.com/luarocks/luarocks
		License      : MIT
		Description  : Lua Module Deployment & Management

So that’s straightened out; thx !!

Still seems to me there should be some way to concurrently show ALL available releases of a same-versioned package across ALL enabled repos – the goal being to see ALL of what I’ve built, and where it is in my repo-space.

pgfed · July 2, 2020, 1:49am

looks like the ‘solution’ in Fedora most similar to my old solution is – my old solution: zypper !

dnf install zypper
( ... edit zypp.conf ...)
zypper ref
zypper se -s compat-luarocks

S | Name            | Type       | Version                          | Arch   | Repository
--+-----------------+------------+----------------------------------+--------+--------------------------------------------
  | compat-luarocks | package    | v3.3.1-pgnd_20200702_013304.fc32 | x86_64 | Copr repo for nginx-mainline owned by pgfed
  | compat-luarocks | package    | v3.3.1-pgnd_20200701_161621.fc32 | x86_64 | nginx-mainline-local
  | compat-luarocks | srcpackage | v3.3.1-pgnd_20200702_013256.fc32 | noarch | Copr repo for nginx-mainline owned by pgfed
  | compat-luarocks | srcpackage | v3.3.1-pgnd_20200702_013244.fc32 | noarch | Copr repo for nginx-mainline owned by pgfed
  | compat-luarocks | srcpackage | v3.3.1-pgnd_20200701_161621.fc32 | noarch | nginx-mainline-local

which is exactly what I’m looking for.