"IHATECDN" at the end of a file payload causing problems for archive extractors

Ask Fedora
,
1

When analyzing Fedora update problems through our firewall, we noticed that when retrieving https://cdn.fwupd.org/downloads/firmware.xml.zst.jcat the payload of the response has a few extra bytes saying “IHATECDN” at the end.
This obviously causes problem in some archive extractors, could you please remove these unnecessary bytes?

2

I do not think that Fedora is responsible for fwupd.org

4

Maybe not, but Fedora is affected, and I could not find any contact address for fwupd.org.

5

@rhughes is one of the fwupd developers.

1 Like
6

you could file an issue here: Issues · LVFS / lvfs-website · GitLab to gain some attention. website without an imprint or contact address is really bad.

7

the IHATECDN bytes are very deliberate I’m afraid – removing them would break updates for tens of thousands of people seemingly using broken firewalls. What archive extractor are you using?

2 Likes
8

Spam, spam, spam I’m afraid.

9

libarchive reports a fatal error. Command line gunzip complains, but unpacks correctly.

10

So back up; what are you trying to do?

11

We scan archives for malware in our firewall, using libarchive to unpack them. As libarchive reports a fatal error, we block the response.

12

That sounds like a libarchive bug then, to be honest. The choice I have is to make fwupd work on tens of thousands of machines by adding the padding or to remove the padding to make your firewall work. I suspect adding an allow-list entry for the fwupd metadata signature might be a pragmatic workaround.

2 Likes
13

TBH to me (given jcat/jcatfile.py · master · LVFS / lvfs-website · GitLab) this sounds like a CDN bug. Having to add dummy data for some infrastructure to not break things is more likely a bug than a library being asked to process data that it cannot know about.

14

It is a CDN bug, but it affects tons of people. I’m stuck between a rock and a hard place.

15

hey, can you link me the bug if its know to the cdn?

16

gzip can unzip the file without any problem, which are these “some archive extractors” that you are using and that are causing problems ?

17

libarchive

18

Sorry, I had seen it in your reply since. Seems like it is a known bug of the python gzip library, but it has been there since 2011:

19

In fact, if you are actually using python (sorry if you don’t) the workaround to use zlib instead of gzip and passing a negative data stream length, works :

import zlib 
filename='firmware.xml.zstd' 
with open(filename, "rb") as gzipfile: 
    data = zlib.decompress(gzipfile.read()[10:], -zlib.MAX_WBITS) 
    print(data)
20

I think all the original debugging was in Failed to load signature: Need more input · Issue #6032 · fwupd/fwupd · GitHub – although there are other dupes too over the last 5 years or so while we didn’t know what was going on.