rocFFT: packaging non-library, non-executable binary files

Project Discussion
1

As I’m getting back into packaging rocFFT, I’m revisiting a packaging issue with non-executable binary files that I don’t have a clear answer to and probably requires a bit more discussion.

One feature of rocFFT is pre-compiled kernels where instead of compiling FFT kernels at runtime, those kernels are compiled at build time and stored in a sqlite database for later retrieval. The pre-compiled kernels are not required for rocFFT’s operation and they account for a significant portion of the build time but they can increase runtime performance.

  • The pre-compiled kernels are stored in a sqlite database
  • The pre-compiled kernel database is 1.9 GB for 5.7.1
  • The rocFFT code looks for the kernel database in one of three places (in order)
    1. a directory stored in the ROCFFT_RTC_SYS_CACHE_PATH environment variable
    2. alongside the rocfft library (for Fedora, that would be /usr/lib64)
    3. in the rocfft library subdir (/usr/lib64/rocfft) which is the default location for the database file and where my current package puts it

The relevant code from the 5.7.1 release is here. There have been no public changes to this code since the 5.7.1 release so far.

I’m not sure that the kernel db belongs in /usr/lib64 and something like /usr/libexec makes more sense to me. As I read the rocFFT code, there is no way to set the database location at compile time but if I’m missing something, please let me know. Technically, we could document the need to set an environment variable or work on setting a default at package install time but that doesn’t fell like a good solution to me.

As a reference, the debian package for rocFFT currently disables the kernel database at build time but that could change (reference github issue comment)

I have several questions for discussion:

  1. Should these kernels be pre-compiled and packaged?
  2. If we package the pre-compiled kernels, where should they live?
  3. If we package the pre-compiled kernels, should they be in the main rocFFT package, the devel sub-package or in a separate subpackage for just the kernel db?