F40 Change Proposal: Transitioning to Zlib-ng as a compatible replacement for Zlib (System-Wide)

Transitioning to Zlib-ng as compatible replacement for Zlib

This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.

Annouced
Wiki

:link: Summary

Replace Zlib with Zlib-ng. This change is strongly tied to Changes/MinizipNGTransition.

:link: Owner

:link: Current status

:link: Detailed Description

Zlib has long served as a reliable compression library, but its performance on modern CPU architectures does not fully utilize their capabilities. The transition to Zlib-ng aims to enhance the compression efficiency and performance for Fedora while ensuring compatibility with existing packages and libraries.

:link: Feedback

:link: Benefit to Fedora

  • Zlib-ng supports hardware acceleration when available
    • Adler32 implementation using SSSE3, AVX2, AVX512, AVX512-VNNI, Neon, VMX & VSX
    • CRC32-B implementation using PCLMULQDQ, VPCLMULQDQ, ACLE, & IBM Z
    • Hash table implementation using CRC32-C intrinsics on x86 and ARM
    • Slide hash implementations using SSE2, AVX2, ARMv6, Neon, VMX & VSX
    • Compare256 implementations using SSE2, AVX2, Neon, POWER9 & RVV
    • Inflate chunk copying using SSE2, SSSE3, AVX, Neon & VSX
    • Support for hardware-accelerated deflate using IBM Z DFLTCC
  • Includes Intel and Cloudflare optimizations
  • Uses code sanitizers, fuzzing, and code coverage

:link: Scope

  • Proposal owners:
    • Zlib-ng will be rebuilt with zlib-compatible API, obsoleting zlib-1.2
      • Zlib-ng compat mode will be distributed in a new package called zlib-ng-compat, with its respective zlib-ng-compat-devel.
      • Initially, zlib-ng-compat will provide zlib = 1.2.13 and zlib-ng-compat-devel will provide zlib = 1.2.13.
      • These versions will be updated following the support added to zlib-ng upstream.
    • Considering the new packages promise to be ABI-compatible with zlib, the packages depending on zlib won’t be rebuilt.
      • This may help to catch ABI-compatibility issues soon.
  • Other developers: Help with build failures may be requested.
  • Release engineering: No action required
  • Policies and guidelines: N/A
  • Trademark approval: N/A
  • Alignment with Objectives: N/A

:link: Upgrade/compatibility impact

Zlib-ng has a zlib-compat mode that aims to preserve both API and ABI compatibility with the original Zlib. However certain application tests may rely on hardcoded checksums for the compressed output. These tests could fail due to various changes, even if the compressed output remains valid and decompresses correctly.

:link: How To Test

Update the system, verify that it does not break other packages.

:link: User Experience

This change will increase compression efficiency and performance

:link: Dependencies

Zlib is a dependency for over 100 packages. Most of them are expected not to fail or break.

:link: Contingency Plan

  • Contingency mechanism: Revert the change to Zlib and rebuild dependent packages.
  • Contingency deadline: Beta freeze
  • Blocks release? No

:link: Documentation

:link: Release Notes

This change proposal has now been submitted to FESCo with ticket #3109 for voting.

To find out more, please visit our Changes Policy documentation.

1 Like

Is this a typo, should it zlib-devel?

@tstellar , Yes. It should read:

… and zlib-ng-compat-devel will provide zlib-devel = 1.2.13

I am having a weird issue of segfaults in specific situations with a proprietary application using Mono. Apparently zlib-ng is involved. Users of zlib do not report this problem.
Where should I report? Upstream with zlib-ng? Would you consider problems with a proprietary application (game) even as a relevant regression?

How do you feel about the proposal as written?
  • Strongly in favor
  • In favor, with reservations
  • Neutral
  • Opposed, but could be convinced
  • Strongly opposed
0 voters

If you are in favor but have reservations, or are opposed but something could change your mind, please explain in a reply.

We want everyone to be heard, but many posts repeating the same thing actually makes that harder. If you have something new to say, please say it. If, instead, you find someone has already covered what you’d like to express, please simply giving that post a :heart: instead of reiterating. You can even do this by email, by replying with the heart emoji or just “+1”. This will make long topics easier to follow.

Please note that this is an advisory “straw poll” meant to gauge sentiment. It isn’t a vote or a scientific survey. See About the Change Proposals category for more about the Change Process and moderation policy.

Depends on the nature of the problems, I suppose. If it can be reproduced and doesn’t happen with zlib then filing an issue against zlib-ng would be appropriate.

1 Like

I filed a zlib-ng issue:

1 Like

Thanks for investigating this!
I see you proposed a fix at Correct the selection of alloc/free functions for zlib. by jo-oe · Pull Request #21759 · mono/mono · GitHub
Would you like some help in order to get it backported to Fedora 40?

I have never contributed to fedora before.

Also I while updating the mono packages in fedora would be nice, i does not fix the actual problem I had, where a proprietary software (“Kerbal Space Program” video game") with bundled mono libraries (from Unity) crashes. This software won’t use the system provided mono libraries anyways, but it does use the system provided zlib(-ng). That is probably something noone else than the game’s publisher could fix by updating the bundled library.

I am not sure if the code path causing the crash I mentioned would be used in any other software, so maybe it is not worth the fuss including the patch in fedora, is it?

John Obscurant via Fedora Discussion
notifications@fedoraproject.discoursemail.com writes:

I have never contributed to fedora before.

I can do this if you don’t plan to do this.
But if you prefer, I can help you through the process.
Let me know what you prefer.

I am not sure if the code path causing the crash I mentioned would be used in any other software, so maybe it is not worth the fuss including the patch in fedora, is it?

AFAIU this is affecting mono distributed by Fedora.
I can see the error in the source code from the package distributed by Fedora.
You were the first one to find it and other people may hit it too.
That means that fixing it downstream is still important for the Fedora
community even if it doesn’t solve the issue for you.

1 Like

Ok, yeah, then I’m happy if you’d like to teach me :slight_smile:
How should we start? Might I suggest we open a chat somewhere where you could guide me?

Ok, yeah, then I’m happy if you’d like to teach me :slight_smile:
How should we start?

I suggest to start by checking if you have followed steps 3 and 4 from:

Might I suggest we open a chat somewhere where you could guide me?

Sure! I’m usually tuliom on librera.chat and Fedora Chat.
If you prefer, send me an email: tuliom at ascii.art.br