Need help using mock to build a binary rpm from a source rpm

Ask Fedora
1

I have setup a fedora-34 VM. I want to build the python2.7 binary debuginfo rpm from the python2.7 srpm. This has value to me in its own right but I am also using this to learn about rpm packaging tools in depth.

I can see that I have this version of python2.7 installed:

rpm -q python2.7
python2.7-2.7.18-21.fc34.x86_64

and I downloaded the SRPM like this:

dnf download --source python2.7

and then kicked off a build via mock like this:

mock -r fedora-34-x86_64 python2.7-2.7.18-21.fc34.src.rpm

this seemed to work, and produced these results:

ls -la /var/lib/mock/fedora-34-x86_64/result/

total 36368
drwxrwsr-x. 2 em7admin mock     4096 Aug  6 12:28 .
drwxrwsr-x. 4 root     mock       54 Aug  6 12:28 ..
-rw-rw-r--. 1 em7admin mock  5081696 Aug  6 12:28 build.log
-rw-rw-r--. 1 em7admin mock     2537 Aug  6 12:03 hw_info.log
-rw-rw-r--. 1 em7admin mock    31609 Aug  6 12:04 installed_pkgs.log
-rw-r--r--. 1 em7admin mock 13551113 Aug  6 12:03 python2.7-2.7.18-21.fc34.src.rpm
-rw-r--r--. 1 em7admin mock 13162772 Aug  6 12:28 python2.7-2.7.18-21.fc34.x86_64.rpm
-rw-r--r--. 1 em7admin mock  3152974 Aug  6 12:27 python2.7-debuginfo-2.7.18-21.fc34.x86_64.rpm
-rw-r--r--. 1 em7admin mock  2036041 Aug  6 12:27 python2.7-debugsource-2.7.18-21.fc34.x86_64.rpm
-rw-rw-r--. 1 em7admin mock   197064 Aug  6 12:28 root.log
-rw-rw-r--. 1 em7admin mock      841 Aug  6 12:28 state.log

However, when I try to use gdb to debug a core file generated from a simple python program I wrote to specifically dump a core file so I could test this, it seems that the debuginfo package I compiled does not match the one already installed in fc-34:

type or paste code here
gdb /usr/bin/python2.7 core
GNU gdb (GDB) Fedora 11.1-5.fc34
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/python2.7...

warning: the debug information found in "/usr/lib/debug//usr/bin/python2.7-2.7.18-21.fc34.x86_64.debug" does not match "/usr/bin/python2.7" (CRC mismatch).

Missing separate debuginfo for /usr/bin/python2.7
Try: dnf --enablerepo='*debug*' install /usr/lib/debug/.build-id/47/f710d3b0f56f0c36686069e8deb2a22060fdfa.debug
Reading symbols from .gnu_debugdata for /usr/bin/python2.7...
(No debugging symbols found in .gnu_debugdata for /usr/bin/python2.7)
[New LWP 132167]

warning: the debug information found in "/usr/lib/debug//lib64/libpython2.7.so.1.0-2.7.18-21.fc34.x86_64.debug" does not match "/lib64/libpython2.7.so.1.0" (CRC mismatch).

Missing separate debuginfo for /lib64/libpython2.7.so.1.0
Try: dnf --enablerepo='*debug*' install /usr/lib/debug/.build-id/ef/e5ee82c6c149599eee3eb2f44f2ad6120a9e02.debug

warning: the debug information found in "/usr/lib/debug//usr/lib64/python2.7/lib-dynload/_localemodule.so-2.7.18-21.fc34.x86_64.debug" does not match "/usr/lib64/python2.7/lib-dynload/_localemodule.so" (CRC mismatch).

Missing separate debuginfo for /usr/lib64/python2.7/lib-dynload/_localemodule.so
Try: dnf --enablerepo='*debug*' install /usr/lib/debug/.build-id/6a/48e70ee821577eabd23e7ebb1f8fb2ae22baae.debug

warning: the debug information found in "/usr/lib/debug//usr/lib64/python2.7/lib-dynload/readline.so-2.7.18-21.fc34.x86_64.debug" does not match "/usr/lib64/python2.7/lib-dynload/readline.so" (CRC mismatch).

Missing separate debuginfo for /usr/lib64/python2.7/lib-dynload/readline.so
Try: dnf --enablerepo='*debug*' install /usr/lib/debug/.build-id/11/bc9cdd27f2fe1f5fa5e951fb28d9a10b4c211a.debug
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `python2'.
Program terminated with signal SIGABRT, Aborted.
#0  0x00007f34e7a81292 in raise () from /lib64/libc.so.6
Missing separate debuginfos, use: dnf debuginfo-install glibc-2.33-5.fc34.x86_64 ncurses-libs-6.2-4.20200222.fc34.x86_64 readline-8.1-2.fc34.x86_64

I take it the key lines of output here are:

warning: the debug information found in "/usr/lib/debug//usr/bin/python2.7-2.7.18-21.fc34.x86_64.debug" does not match "/usr/bin/python2.7" (CRC mismatch).

When I install the debuginfo package from fedora’s repos with:

dnf install python2.7-debuginfo --enablerepo='*debug*'

I have no problem analyzing the core file with gdb. My ultimate goal here is to understand how to recreate binary builds so that, if all I have is a RPM derived distribution where the source package is available but the debuginfos package is not, I can produce the binary RPM myself and analyze core dumps.

2

Each time a src.rpm is built the binary (and the debug) will be different.

Aside: fully reproduceable builds is a ongoing goal what is being worked on by
some amazing people.

So if you want to use your debug package you will need to install the matching
python2.7 binary rpm as well.

If you want to go deeper into the RPM world then you can do this:

rpm -i python2.7-2.7.18-21.fc34.src.rpm
cd ~/rpmbuild/SPEC # I think ~/rpmbuild is where it is installed by default
edit python.spec
# change the Release: value to be 1 bigger then the installed python2.7
mock --root fedora-34-x86_64 --buildsrpm --source ../SOURCES --spec python.spec

I tend to use mock to build my .src.rpm so that it uses the tools that match the --root.

Now you have your .src.rpm use your mock recipe to build the bin rpm and then install
it and the debug rpm.

3

Barry:

Thanks so much! You have given me lots of threads to pull on here.

I like the idea of bumping the release number by 1 on the python build. Then I would be able to upgrade the python already installed, and use the debuginfo package I compiled with it. I can see scenarios where the would work well. I think I am going to experiment with that.

One of the problems I am trying to solve is that I have a customer using oracle 7.9, and my company’s software, running on their system, is producing core dumps. I want to be able to run gdb on a similar system I have in house to I can analyze the core dump. Oracle does not publish the debuginfo package for the version of python they shipped in 7.9.

I see how I can, using fedora 34, and mock, build a binary rpm for oracle 7.9, from their python srpm. That debuginfos package, however, is going to have a different debug symbols CRC than the python package that I built. And is that even the CRC that matters (is the CRC written into the core file?)

Does the CRC mechanism of gdb amount to (overly conservative) defensive coding? If it were possible to get around the CRC check, would a mock rebuild of the SRPM of a package like python be close enough to oracle’s build to interpret the debug symbols in a coredump created on a system using their package?

Or is there really so much variability from one compile to the next, that the debug symbols would functionally differ too much?

4

Barry:

I have been experimenting with your suggestion to bump up the release number by one and it seems to work. In my case, I generated a core file using the python2.7 already installed by fedora 34. Then I modified the SRPM’s spec by bumping up the release number, and then used mock -r to rebuild from the SRPM.

Then I used rpm to upgrade to the binary packages I had just built, and I was able to analyze the core file (generated by running the earlier version of python). I never would have guessed that would work.

I thought sure the core file was at least as intimately connected to the binary package of the program that produced it as the debug infos package would be to that package. But it seems to be less so.

Thanks again for your help,

Chris

5

The core file does not contain version info, only filenames or .so etc.
Which is why what you did worked.