Bringing R packages to Fedora is an Herculean task, especially considering the rate at which [CRAN](https://cran.r-project.org) grows nowadays. This is an attempt to maintain an RPM repo for most of CRAN in an automated way using Fedora Copr, while ensuring compatibility with the packages already in the official repos.
Still under construction. Stay tuned.
Note that the long queue in Copr are not caused by Cran project. One user can block max 7 builders at the same time (out of 44). At the same time, if anyone submit big bulk of builds at once, it is highly recommended to use copr-cli build --background to lower your priorities and not block others.
Assuming that iucar/cran is not violating copr policies and that batches are sent correctly using --background to lower the noise of other users, question is: is it really needed to rebuild all CRAN repository? Is there a request behind this or is this building just for fun?
The CRAN builds are not submitted with --background. There are 0 builds with lower priority pending right now.
I’m also doing some “big” (a few hundred packages) batches of COPR builds regularly, and originally I used --background to do that, just to be a good “internet citizen”. But I had to stop using lower priority since all the CRAN builds started, or otherwise my builds would basically never get scheduled to run.
Sorry, guys, I wasn’t receiving notifications about these messages (does anyone know how to enable them?). Peak cancelled. As @msuchy explained, builders are distributed among users, so that a long queue for one user doesn’t hold all the resources. Anyway, I didn’t know about these “background” jobs (I think it should be better documented), so I’ll submit batches using the background flag from now on.
It would be great, though, if this wasn’t necessary. When the pending queue for a particular repo grows above some limit, builds could be automatically marked as background. The limit could depend on the number of builders available and the number of users submitting builds. Just an idea.
@sbonazzo: I was told that this was a fair use case when I started the project. I think that Copr should be able to deal with such projects (and the command line tools and APIs provided should be adequate to such projects; currently, they are not). At the very least, this project has served to identify some bugs and RFEs.
So is it really needed to rebuild all CRAN repository? I believe it is. CRAN provides binaries for Windows and MacOS, but not for Linux. This is why Debian/Ubuntu has had for years a similar initiative to maintain a non-official repo for most of CRAN. Fedora has only a few hundred R packages in the official repositories, and maintaining them is already a huge task. So this Copr repo is meant to fill this gap.
@iucar Thanks for switching to low priority builds! I’ve done the same for my batch jobs now
Also, judging by that I haven’t even noticed your 14k pending builds today, I’d say that submitting batch builds with lower priority definitely improves responsiveness for other users.
Hello @iucar, could you include the AL2023 chroots in your builds? I know that I am asking for you to include additional build time and that’s been a complication in the past, but I hope it’s not too much overhead.
chroots:
Hi, @davdunc, we are tracking this here. It wasn’s a matter of build time. There were too many dependencies missing that would result in many build failures, so we wanted to ensure that at least the spatial stack could be built successfully. Last time I checked, there was no news about the inclusion of such dependencies, but not sure if this is still true. We can continue the discussion on GitHub anyway. Thanks!
Thanks for the compiled packages. Speeds up package installs a lot, especially after R upgrades. In the future it would be great to make it work with pak as well.
I found it crucial to add the install location in the .Renviron so that it will end up where I want in the .libPaths(), i.e. after the user library. Without something like this, the installed packages cannot be used. In the terminal:
pak has a fundamentally different approach. But, anyway, we just provide the binaries here. Whether pak is able to use them or not is in pak’s court.
About the installation path, you shouldn’t need that. This is what I get with Fedora’s default R installation, no Renviron involved:
Hi @iucar , I am not sure this is where I am supposed to write but I wanted to draw your attention on a particular build: Package R-CRAN-s2 in iucar/cran
Given than s2 is heavily used in the popular package sf (and given that updating s2 creates visible change), it would be great if you could look into it. (I failed to find the logs describing what is wrong, but I suspect that a dependencies is missing, perhaps abseil-cpp-devel).