Improved build script for doc repos (with change detection!)

Hi folks,

The one thing I’ve been struggling with when writing docs is the ./build.sh and ./preview.sh bits. One has to re-run ./build.sh each time one makes a change to see the new preview. This is far from ideal.

So I went looking for a way to emulate “watch for changes and rebuild as required” that tools like pelican and jekyll allow us to do and came up with a solution using inotifywait.

Now, one can run./builder.sh -w and it’ll watch the modules folder for changes and automatically rebuild when required.

I didn’t see a point of having different scripts for build and preview, so I’ve also subsumed ./build.sh and ./preview.sh into this script.

I’m testing it out with the NeuroFedora docs here, and it seems to work well. Could folks please test it out, and if it does work, can we please replace ./build.sh and ./preview.sh with this in all our repos to make it slightly easier for people to work locally on docs?

The file is here:

https://pagure.io/neuro-sig/documentation/blob/feat/improved-build-script/f/builder.sh

PR here in the NeuroFedora docs repo, so please feel free to drop comments there if you prefer:

https://pagure.io/neuro-sig/documentation/pull-request/23

$ ./builder.sh -h
./builder.sh: Build and preview Fedora antora based documentation

Usage: ./builder.sh [-awbpkh]

-a: start preview, start watcher and rebuilder
-w: start watcher and rebuilder
-b: rebuild
-p: start_preview
-k: stop_preview
-h: print this usage text and exit
5 Likes

Hey, that’s very, very welcome! At the moment I’m stuck with the release of a Server VM, but as soon as I write on docs again I’ll try it.

2 Likes

This is awesome, @ankursinha! I wish we had an easy way to roll it out to all of the repos (or distribute it independently of the repos)

2 Likes

How about adding it to a separate repository and integrating it using git submodules?

If it is just a single file, git submodules seem like overkill. I’d rather just see a comment added to the top of the script that indicates where the latest version can be downloaded from.

1 Like

One way, that would work for doc writers on Fedora at least, would be to create a fedora-docwriter package which is similar to the fedora-packager package. It’ll:

  • include this script (with a better name!)
  • pull in the various packages that are required: (python3, podman, inotify-tools)

That way, the script could live in one place instead of a copy living in each doc repo? We could leave the build.sh, preview.sh scripts in repos for the moment and deprecate them as we go by adding some text to them like this:

replaced by fedora-docwriter script. On Fedora, install `fedora-docwriter`. For other systems, you can get the script here: URL
1 Like

Do we have/know any mac users who can please test the script to see how it works on macs?

I remember seeing a discussion on some doc-related issue on pagure about how best to distribute these scripts, but I can’t find it anymore. @bcotton any idea?
I think the idea of a dedicated RPM was mentioned, and I think it makes more sense now.

Edit: found it! Issue #5: How do we share template updates? - template - Pagure.io

2 Likes

Lots of changes to the script after Francois’s review folks, if you’d started using it already, please do use the latest version. I’ll leave the PR open for a bit for any more feedback. Then we can see if we want to move it to a different repo and make an rpm package etc.

1 Like

I will check out in the coming days if I find something on GitLab. It has several nice tools that go beyond the usual git functions (cherry-pick and such), maybe some of these tools can help/support us with the repos that have already moved to GitLab. Thanks @ankursinha !

1 Like

Did we find a way to roll it out? I’ve opened a PR for quick-docs here, in the meantime.

1 Like

You can open a PR for the Docs template too, it’ll be used for new projects at least.

2 Likes

Well, I think GitLab functions cannot help us here. We are bound to what Git in general offers us. We can push it manually into each repo. I think this is a question of only a few minutes anyway.

If we agree that a specific file (the link above does no longer work) should replace all build.sh & preview.sh in all repos, I can do the job manually on all GitLab repos that are maintained by the Docs team. Someone else might do it on pagure for the repos that are still there (or wait until they are moved to GitLab and let me know). We can inform the other teams and they can decide themselves if they want to use it.

And maybe we can additionally create a solution to centralize the update of the build/preview in future, such as the suggestion of @mateusrodcosta → however, we should be careful with putting too much efforts in this because I think updating these files takes place rarely, and we have not that many repos anyway. When we have everything on GitLab, each update of these files would be a matter of minutes for one person.

1 Like

PR opened:

I’ve got a simple spec here that will install the script as fedora-docs-builder to /usr/bin. that way, folks on Fedora systems just install this and use fedora-docs-builder .. instead of relying on whatever script is included in each repo.

If this sounds like a good idea, I can submit this as a new package for inclusion in Fedora:

Oh, and better names for the installed script and package name are welcome.

1 Like

HI,

This script is really nice, I have really been missing change detection.

However, I ran into a bug:
It does not trigger when I edit the file modules/ROOT/pages/Packaging_Tutorial_GNU_Hello.adoc.
Why? Well, the path matches the exclude regex .git* as defined in the script.
There is substring agi in there :grimacing:.

2 Likes

Hrm, that sounds like a bug. I’ll take a look now. Maybe I’ve used glob expressions there instead of regexes :thinking:

Fixed, PR:

https://pagure.io/fedora-docs/package-maintainer-docs/pull-request/89

Also updated the main PR.

While I was at it, I also added functionality to check if a new version of the script is available, and install it if required. Pushed this to the main templates PR only for now, since it won’t work until that is merged.
All it does is fetch the file from the templates repo, and check that has a higher version than the current script in use.

Just a question: Isn’t it a shell script? Then it should work on a Mac, too, or do you use Fedora-specific binaries? If it works on Mac, it would be nice to have it there as well.

1 Like

It should work on Mac too but I don’t have one so I don’t know if the command line tools/binaries the script relies on are available on Mac (or how one would go about installing them). If someone could test it out, that’d be great.

(The rpm, of course, is limited to Fedora users only.)