Originally published at: Infra and RelEng Update - Week 19 – Fedora Community Blog
This is a weekly report from the I&R (Infrastructure & Release Engineering) Team. We provide you both infographic and text version of the weekly report. If you just want to quickly look at what we did, just look at the infographic. If you are interested in more in depth details look below the infographic.
Week: 5th – 9th May 2025
Infrastructure & Release Engineering
The purpose of this team is to take care of day to day business regarding CentOS and Fedora Infrastructure and Fedora release engineering work.
It’s responsible for services running in Fedora and CentOS infrastructure and preparing things for the new Fedora release (mirrors, mass branching, new namespaces etc.).
List of planned/in-progress issues
Fedora Infra
- In progress:
- Spam in devel list
- [FMTS] TLS certificate for centos-odcs-private-queue service is about to expire in 30 days
- [FMTS] TLS certificate for centos-odcs service is about to expire in 30 days
- Move copr_hypervisor group from iptables to nftables
- tmpwatch removed from ansible
- FedoraQA – Blockerbugs – OIDC setup
- please don’t remove enrolled centos machines from IPA in staging
- Inactive packagers for the F42 cycle
- wiki upgrade before f40 eol
- FedoraQA – Testdays – OIDC setup
- redeploy aws proxies
- Disable self-serve project creation on pagure.io
- Link in staging distgit instance leads to prod auth
- re-add datagrepper nagios checks (and add to zabbix?)
- Fix logrotate on kojipkgs01/02
- 2025 datacenter move (IAD2->RDU3)
- Broken link for STG IPA CA certificate, needed for staging CentOS Koji cert
- [CommOps] Open Data Hub on Communishift
- retire easyfix
- Deploy Element Server Suite operator in staging
- Pagure returns error 500 trying to open a PR on https://src.fedoraproject.org/rpms/python-setuptools-gettext
- Create a POC integration in Konflux for fedora-infra/webhook-to-fedora-messaging
- maubot-meetings bot multi line paste is cut
- setup ipa02.stg and ipa03.stg again as replicas
- Move OpenShift apps from deploymentconfig to deployment
- The process to update the OpenH264 repos is broken
- httpd 2.4.61 causing issue in fedora infrastructure
- Support allocation dedicated hosts for Testing Farm
- EPEL minor version archive repos in MirrorManager
- Add fedora-l10n pagure group as an admin to the fedora-l10n-docs namespace projects
- vmhost-x86-copr01.rdu-cc.fedoraproject.org DOWN
- Add yselkowitz to list to notify when ELN builds fail
- Cleaning script for communishift
- Move from iptables to firewalld
- Port apps to OIDC
- Help me move my discourse bots to production?
- Replace Nagios with Zabbix in Fedora Infrastructure
- Migration of registry.fedoraproject.org to quay.io
- Done:
- cloud images are missing `saved_entry` in grubenv
- Issues using dnf failing on 503 errors getting mirrors from 18.159.254.57 and 2a05:d014:10:7803:f774:4d7c:e277:a457
- buildvm-ppc64le-10.iad2.fedoraproject.org Koji builder: Read-only file system
- Spam on Join ML
- [FMTS] TLS certificate for coreos-ostree-importer service is about to expire in 30 days
- [FMTS] TLS certificate for monitor-gating service is about to expire in 30 days
- Download Fedora IoT 42: 404 Not Found
- Please issue new fedora-messaging/rabbitmq TLS certs for CentOS Stream infra
- Several proxies are missing ipv6 addresses in DNS
- Discussion
- Please create #mobility-sdm845:fedoraproject.org and #mobility-pinephones:fedoraproject.org Matrix rooms
- Broken mirror
- Pagure DB: Name change request for @jflory7
- Bodhi API timeouts
- Request #centos-proposed-updates:fedoraproject.org as additional address for #centos-proposed-updates:fedora.im
- pkg rpm search for openssl incorrectly points to openssl3, misses openssl
- [x86_64] Firefox fails to build on Fedora 41
- Failure to log in to fedora discussion account with container.
- Help with Weblate Fedora Messaging
- Koschei does not rebuild rubygem-mysql2 since mid August
CentOS Infra including CentOS CI
- In progress:
- Release Improvements
- Wrapper to check / create projects on GitLab using the REST API
- Write a script to cleanup Duffy DB after retiring a pool
- create Oauth2 tokens for testing instance for OpenQA CentOS stream testing purposes
- Add proposed-updates c9s and c10s tags to hyperscale9s and hyperscale10s tags
- Missing s390x extras/extras-common repo
- Ensuring ansible automation in place can manage/control el10 nodes
- [spike] : investigating ppc64le kvm host option with el9/el10 (for Power10)
- (Cloud SIG) OKDerator CI/CD space
- [spike] : investigating needed deps (poetry) for duffy on el9
- [spike] : investigating options/alternatives for the upcoming DC move
- Done:
- OOM on x86-03.stream.rdu2.redhat.com, ppc64le-01.stream.rdu2.redhat.com
- [Auto-SIG] Add IP to centos-sigs-ami-images
- Broken mirror in mirrorlist
- Init new wireguard role for quick/easy tunnel between DCs
- Please upgrade CBS Koji to 1.35+ for kiwi plugin enhancements
- Request for machine intended for CentOS stream OpenQA instance
Release Engineering
- In progress:
- Please tag EPEL 10 Haskell builds also into epel10.1
- consider adding iot iso image to torrents
- F42 Post Release Cleanup
- Fix OpenH264 tagging issues
- Turn EPEL minor branching scripts into playbooks
- Mass retirement of packages with uninitialized rawhide branch
- Please send openh264-2.6.0 to Cisco
- 300+ F42FTBFS bugzillas block the F41FTBFS tracker
- Packages that have not been rebuilt in a while or ever
- Send compose reports to a to-be-created separate ML
- Could we have fedoraproject-updates-archive.fedoraproject.org for Rawhide?
- Investigate and untag packages that failed gating but were merged in via mass rebuild
- a few mass rebuild bumps failed to git push – script should retry or error
- Package retirements are broken in rawhide
- Update pungi filters
- Implement checks on package retirements
- Untag containers-common-0.57.1-6.fc40
- Provide stable names for images
- Packages that fail to build SRPM are not reported during the mass rebuild bugzillas
- When orphaning packages, keep the original owner as co-maintainer
- Create an ansible playbook to do the mass-branching
- RFE: Integration of Anitya to Packager Workflow
- Fix tokens for ftbfs_weekly_reminder. script
- Update bootloader components assignee to “Bootloader Engineering Team”for Improved collaboration
- Done:
- Stalled EPEL packages: xdpyinfo, libXxf86dga and libdmx
- Stalled EPEL package: python-binaryornot
- Unretire marker
- Missing Rawhide compose reports
- stalled EPEL request: python-apsw
- Stalled EPEL package: python-google-cloud-core
- Deletion of accidently created branch
- Stalled EPEL package: python-proto-plus
- BuildError: package perl-Date-Holidays-DE not in list for tag epel10.{1,0}-testing-candidate
- Stalled EPEL request pyOpenSSL
- New package yudit repository created but owned by releng-bot
- please unlock ghc/epel10.0
- Bodhi Pending/Current releases need adjustment
- Re-push root-6.34.08-4.fc42 to stable
- 10 builds still koji tagged with signing-pending
If you have any questions or feedback, please respond to this report or contact us on #redhat-cpe channel on matrix.