After some discussion at Flock about LLM usage for packaging and how it is actually useful if you point it to ur packaging guidelines, I created an TMT test that can use Claude via Amazon Bedrock to perform LLM package reviews.
I’d like to experimentally hook it up to PR CI and/or forgejo action at packaging/package-review to see how it operates on the CI. However, to do that, I would need some budget and Amazon Bedrock credentials.
Who do I talk to about this? Council, Mindshare, FESCo, Infra, CI folks, my Red Hat management?
I suppose I could, but then I’d need that hardware. I asked the log-detective folks (@jpodivin) how they do this and they said they use AWS with GPU/CUDA from Fedora AWS budget but using Bedrock directly makes more sense.
This is the first time for me exploring an idea like this. I’d take anything that a) works b) yields useful results.
Yeah I highly recommend using Bedrock for this. Just make sure you put in a spend cap as it can easily eat up tens of thousands of dollars if you’re not careful. You should be able to use the Fedora AWS account for this. If that doesn’t pan out I can also see if I can sponsor an account from the Meta side.
Thanks, I’ll request a runner as well. Or use what we already have.
As far as I can tell, the “non-AI” resources are not much. It downloads a couple RPMs and logs and clones some small git repos. Wrt AI, it is ~$6 per review with sonnet, ~$1 with opus. If we hook it up just into packaging/package-review I don’t think it will ruin us. For starters, we can cap it up on $300 a month, which is an amount that should be familiar to Red Hat employees.
For regular dist-git PRs, I would pretty much prefer for this to be opt-in rather than automagic, as I can only imagine not everybody would want this and I don’t want to burn money and water willy-nilly.
I would recommend trying wider range of models. I’ve had good experience with DeepSeek and GLM line isn’t bad either. Both are substantially cheaper compared to models from Anthropic.