Infra and RelEng Update - Week 30 2025

[zlopez (Michal Konečný)]

Originally published at: Infra and RelEng Update - Week 30 2025 – Fedora Community Blog

This is a weekly report from the I&R (Infrastructure & Release Engineering) Team. We provide you both infographic and text version of the weekly report. If you just want to quickly look at what we did, just look at the infographic. If you are interested in more in depth details look below the infographic.

Week: 21st – 25th July 2025

Infrastructure & Release Engineering

The purpose of this team is to take care of day to day business regarding CentOS and Fedora Infrastructure and Fedora release engineering work.
It’s responsible for services running in Fedora and CentOS infrastructure and preparing things for the new Fedora release (mirrors, mass branching, new namespaces etc.).
List of planned/in-progress issues

Fedora Infra

• In progress:
  • reconfigure power10 machines
  • [FMTS] TLS certificate for centos-koji service is about to expire in 30 days
  • [FMTS] TLS certificate for git-hooks service is about to expire in 30 days
  • [FMTS] TLS certificate for batcave service is about to expire in 30 days
  • [FMTS] TLS certificate for gitlab-centos service is about to expire in 30 days
  • [FMTS] TLS certificate for fmc service is about to expire in 30 days
  • [RFE] expose metrics from rabbitmq queues for centos infra consumption (or others)
  • Forgejo: Owner access to @jflory7 for @CommOps
  • Move copr_hypervisor group from iptables to nftables
  • tmpwatch removed from ansible
  • Link in staging distgit instance leads to prod auth
  • re-add datagrepper nagios checks (and add to zabbix?)
  • Fix logrotate on kojipkgs01/02
  • Broken link for STG IPA CA certificate, needed for staging CentOS Koji cert
  • [CommOps] Open Data Hub on Communishift
  • retire easyfix
  • Deploy Element Server Suite operator in staging
  • Pagure returns error 500 trying to open a PR on https://src.fedoraproject.org/rpms/python-setuptools-gettext
  • maubot-meetings bot multi line paste is cut
  • Move OpenShift apps from deploymentconfig to deployment
  • Support allocation dedicated hosts for Testing Farm
  • EPEL minor version archive repos in MirrorManager
  • Add fedora-l10n pagure group as an admin to the fedora-l10n-docs namespace projects
  • vmhost-x86-copr01.rdu-cc.fedoraproject.org DOWN
  • Add yselkowitz to list to notify when ELN builds fail
  • Cleaning script for communishift
  • Move from iptables to firewalld
  • Port apps to OIDC
  • Help me move my discourse bots to production?
  • Migration of registry.fedoraproject.org to quay.io
• Done:
  • A request for a compatibility package for Intel’s intel-compute-runtime for the older hardware.
  • Login to seccdn.libravatar.org with Fedora Account fails: “Invalid redirect_uri”
  • Can’t activate account in IPA
  • Queue for cloud-image-upload missing in staging
  • SPAM on fedora-devel
  • Packager Dashboard out of sync
  • src.fp.o not sending emails for opened / closed / commented-on PRs?
  • mailman unsubscribing people multiple times
  • Request creation of copr-only group airsim
  • Multiple continuing bounces on server mailing list.
  • Please register my testing instance of Fedora Infrastructure apps to OIDC
  • centos-resilientstorage-9-stream sha512 mismatch
  • s390x images not found in mirrors.hostiserver.com/fedora
  • Random hangs when pushing to pkgs.fedoraproject.org
  • archive.linux.duke.edu ‘s fedora-enchilada module seems incorrect (wrong top level, missing development)
  • 2025 datacenter move (IAD2->RDU3)
  • The process to update the OpenH264 repos is broken
  • httpd 2.4.61 causing issue in fedora infrastructure
  • Need to delete user account in FAS that has been already deleted in Discourse (discussion.fedoraproject.org) due to violations (spam, AI, etc.)
  • Install AWX for testing proof of concept

CentOS Infra including CentOS CI

• In progress:
  • Release Improvements
• Done:
  • Add new mirror: plug-mirror.rcac.purdue.edu
  • [Promo SIG] Migrate some repos from git.centos.org to gitlab
  • Migrate centos-infra-docs to gitlab
  • Migrate CentOS Board tracker to gitlab
  • WSL image builds fail with “Unsupported file type”
  • [SN#1658] Wrapper to check / create projects on GitLab using the REST API
  • [SN#1715] Migrate CentOS infra tracket to gitlab (from pagure.io)
  • [SN#1717] New tags needed for Sama v4.22 and v4.21 in Storage SIG for CentOS Stream 10

Release Engineering

• In progress:
  • pybind11 PR cannot be merged
  • Fedora 43 Mass Rebuild Tracker
  • Improve the mass-tag script so it notify if something is broken and verify permissions
  • Please unblock python-daiquiri
  • When processing stalled EPEL package requests, make the requester EPEL bugzilla override
  • golang-helm-3 not retired properly
  • Packages containg a “+” do not have a f42 branch
  • Transfer/orphan packages owned by rdieter
  • `–no-git-branch` option to fedpkg request-branch and creating the branch manually doesn’t work as expected
  • Turn EPEL minor branching scripts into playbooks
  • Mass retirement of packages with uninitialized rawhide branch
  • 300+ F42FTBFS bugzillas block the F41FTBFS tracker
  • Packages that have not been rebuilt in a while or ever
  • Send compose reports to a to-be-created separate ML
  • Could we have fedoraproject-updates-archive.fedoraproject.org for Rawhide?
  • Investigate and untag packages that failed gating but were merged in via mass rebuild
  • Update pungi filters
  • Implement checks on package retirements
  • Untag containers-common-0.57.1-6.fc40
  • Provide stable names for images
  • Packages that fail to build SRPM are not reported during the mass rebuild bugzillas
  • When orphaning packages, keep the original owner as co-maintainer
  • Create an ansible playbook to do the mass-branching
  • RFE: Integration of Anitya to Packager Workflow
  • Fix tokens for ftbfs_weekly_reminder. script
  • Update bootloader components assignee to “Bootloader Engineering Team”for Improved collaboration
• Done:
  • Stalled EPEL package: daemonize
  • FEX image build fails in wine-core %postinstall
  • Package removal due to licensing
  • Create detached signatures for the ignition 2.22.0 release
  • f41-updates-testing compose failing sanity check and failing
  • Merge f43-perl side tag into f43
  • F43 system-wide change: GNU Toolchain update for F43 https://fedoraproject.org/wiki/Changes/GNUToolchainF43
  • Broken fork
  • F39 Archives are still not cleaned up
  • Not able to view info for any rpm in Koji
  • Uring IO disabled on ppc64le koji builders?
  • Fedora-KDE-42-1.1-x86_64-CHECKSUM has wrong ISO filename
  • F40 end of life
  • a few mass rebuild bumps failed to git push – script should retry or error
  • Package retirements are broken in rawhide

If you have any questions or feedback, please respond to this report or contact us on #redhat-cpe channel on matrix.