Llama.cpp, Whisper.cpp packaging

I packaged the CPP forks of Meta’s Llama (text LLM) and Whisper (voice LLM) for COPR and would like to submit these for mainline repo if anybody is interested. They build and work well in Fedora depending on your release and architecture. COPR status here:
https://copr.fedorainfracloud.org/coprs/boeroboy/brynzai/monitor/

1 Like

The ai-ml-sig is likely interested - can you add that tag to this post @boeroboy ?

Note: these are ports, not forks, and Whisper is originally from OpenAI, not Meta

1 Like

Sure! And right sorry I meant OpenAI’s Llama and yes these are CPP rewrites. I’m a little curious how to offer the opencl and CUDA builds (separate packages) especially if CUDA linking causes license issues.

I suspect that any CUDA related builds would have to be done in rpmfusion due to license issues.

WRT your original question, If the models and weights have licenses which are acceptable according to Fedora packaging guidelines and there are no potential legal issues with the provenance of the weights, feel free to package them.

If you need a review after packaging, post it in this topic or have the reviews block the ML review tracker