Originally published at: Infra and RelEng Update - Week 27, 2025 – Fedora Community Blog
This is a weekly report from the I&R (Infrastructure & Release Engineering) Team. We provide you both infographic and text version of the weekly report. If you just want to quickly look at what we did, just look at the infographic. If you are interested in more in depth details look below the infographic.
Week: 30 June – 04 July 2025
Infrastructure & Release Engineering
The purpose of this team is to take care of day to day business regarding CentOS and Fedora Infrastructure and Fedora release engineering work.
It’s responsible for services running in Fedora and CentOS infrastructure and preparing things for the new Fedora release (mirrors, mass branching, new namespaces etc.).
List of planned/in-progress issues
Fedora Infra
- In progress:
- no Matrix notifications from FMN since June 15
- [FMTS] TLS certificate for gitlab-centos service is about to expire in 30 days
- Decommission provisioning.fp.org
- Planned Outage – Datacenter Move outage – 2025-06-30 01:00 UTC
- Please register my testing instance of Fedora Infrastructure apps to OIDC
- Forgejo: Owner access to @jflory7 for @CommOps
- Move copr_hypervisor group from iptables to nftables
- tmpwatch removed from ansible
- Link in staging distgit instance leads to prod auth
- re-add datagrepper nagios checks (and add to zabbix?)
- Fix logrotate on kojipkgs01/02
- 2025 datacenter move (IAD2->RDU3)
- Broken link for STG IPA CA certificate, needed for staging CentOS Koji cert
- [CommOps] Open Data Hub on Communishift
- retire easyfix
- Deploy Element Server Suite operator in staging
- Pagure returns error 500 trying to open a PR on https://src.fedoraproject.org/rpms/python-setuptools-gettext
- maubot-meetings bot multi line paste is cut
- Move OpenShift apps from deploymentconfig to deployment
- The process to update the OpenH264 repos is broken
- httpd 2.4.61 causing issue in fedora infrastructure
- Support allocation dedicated hosts for Testing Farm
- EPEL minor version archive repos in MirrorManager
- Add fedora-l10n pagure group as an admin to the fedora-l10n-docs namespace projects
- vmhost-x86-copr01.rdu-cc.fedoraproject.org DOWN
- Add yselkowitz to list to notify when ELN builds fail
- Cleaning script for communishift
- Move from iptables to firewalld
- Port apps to OIDC
- Help me move my discourse bots to production?
- Migration of registry.fedoraproject.org to quay.io
- Done:
- Some links from src.fedoraproject.org are broken for packages with plus (+) in their names
- release-monitoring.org isn’t filing bugs
- please don’t remove enrolled centos machines from IPA in staging
- RFR: Requesting a new FAS Group: Phosh SIG
- Need to delete user account in FAS that has been already deleted in Discourse (discussion.fedoraproject.org) due to violations (spam, AI, etc.)
CentOS Infra including CentOS CI
- In progress:
- Done:
- Request for New Mailing List: genos – at – centos.org
- c7 builds failing for cksum mismatch of firewalld-filesystem-0.6.3-13.el7_9.noarch – centos7-updates
- Add siosm (myself) as a sponsor to sig-cloud group
- Stream 9 new mirror : mirror.clarkson.edu
- Verify/Change target of press email address
- Create a repo and FAS group for FRCL
- CBS koji permissions
- Update openinfra server to sync centos mirror
- resources for OpenQA test execution
- migrate id{.stg}.centos.org to new DC
- Hardware init new RDU3 servers (part 1 / wave 0)
- Prepare AWS new VPC for isolated builders
- Mailing lists broken with DMARC
Release Engineering
- In progress:
- Stalled package epel10 gtksourceview3
- F43 system-wide change: GNU Toolchain update for F43 https://fedoraproject.org/wiki/Changes/GNUToolchainF43
- evaluate proposed F43 change for preserving debuginfo in static .a libraries
- `–no-git-branch` option to fedpkg request-branch and creating the branch manually doesn’t work as expected
- EPEL 8 x86_64 won’t sync with Red Hat Satellite
- Broken fork
- msedit: Delete commit
- F39 Archives are still not cleaned up
- Fedora-KDE-42-1.1-x86_64-CHECKSUM has wrong ISO filename
- F40 end of life
- Turn EPEL minor branching scripts into playbooks
- Mass retirement of packages with uninitialized rawhide branch
- 300+ F42FTBFS bugzillas block the F41FTBFS tracker
- Packages that have not been rebuilt in a while or ever
- Send compose reports to a to-be-created separate ML
- Could we have fedoraproject-updates-archive.fedoraproject.org for Rawhide?
- Investigate and untag packages that failed gating but were merged in via mass rebuild
- a few mass rebuild bumps failed to git push – script should retry or error
- Package retirements are broken in rawhide
- Update pungi filters
- Implement checks on package retirements
- Untag containers-common-0.57.1-6.fc40
- Provide stable names for images
- Packages that fail to build SRPM are not reported during the mass rebuild bugzillas
- When orphaning packages, keep the original owner as co-maintainer
- Create an ansible playbook to do the mass-branching
- RFE: Integration of Anitya to Packager Workflow
- Fix tokens for ftbfs_weekly_reminder. script
- Update bootloader components assignee to “Bootloader Engineering Team”for Improved collaboration
- Done:
- Cannot build rust-debug-helper for epel9
- Side tag for Perl 5.42
- F43 System-wide change: Perl 5.42
- New package not in tag f43-updates-candidate
- “initial_commit”: false not respected in releng/fedora-scm-requests
- Fix incorrectly created epel10 branch for lcov package
- Untag graphviz-13.0.1-2.fc43 and graphviz-13.0.1-2.eln150 (and the 13.0.1-1 builds too)
- Removing oneself from a package does not reset bugzilla assignee
- Unretirement request: elementary-photos
- F42 Atomic Desktops `-testing` builds failing on pungi-make-ostree error
- Recent WSL image builds fail with “Unsupported file type: Fedora-WSL-Base-Rawhide-20250530.n.0.aarch64.wsl”
- Request for permissions to make changes on torrent server
- Fix OpenH264 tagging issues
Fedora Data Center Move: “It’s Move Time!” and Successful Progress!
This week was “move time” for the Fedora Data Center migration from IAD2 to RDU3, and thanks to the collective effort of the entire team, it’s been a significant success! We officially closed off the IAD2 datacenter, with core applications, databases, and the build pipeline successfully migrated to RDU3. This involved meticulously scaling down IAD2 OpenShift apps, migrating critical databases, and updating DNS, followed by the deployment and activation of numerous OpenShift applications in RDU3. While challenges arose, especially with networking and various service configurations, our dedicated team worked tirelessly to address them, ensuring most services are now operational in the new environment. We’ll continue validating and refining everything, but we’re thrilled with the progress made in establishing Fedora’s new home!
If you have any questions or feedback, please respond to this report or contact us on #redhat-cpe channel on matrix.