F40 Change Proposal: Switch bogofilter to use SQLite (Self-Contained)

Project Discussion Change Proposals
,
1

Switch bogofilter to use SQLite

This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.

:link: Summary

Switch bogofilter to use SQLite as its database engine, rather than Berkeley DB (libdb).

:link: Owner

:link: Detailed Description

Switch bogofilter to use SQLite as its database engine, rather than Berkeley DB (libdb). Another change (Changes/Libdb deprecated - Fedora Project Wiki) marked libdb as deprecated, and that change lists bogofilter as a dependency. Thus this change fixes another application to avoid the deprecated libdb.

:link: Benefit to Fedora[edit]

Fedora will have one less dependency on the deprecated libdb package. Additionally, other distributions have already migrated to SQLite, and this will allow sharing word lists with those distributions. For example, perhaps a workstation running Fedora generates wordlist.db before installing it on another computer running Alpine and acting as a server.

:link: Scope[edit]

  • Proposal owners:

Merge pull request PR#2: Resolves: #1788486 (Switch to SQLite database engine) - rpms/bogofilter - src.fedoraproject.org. This makes the database backend conditional, with SQLite being the default. Support for libdb can be conditionally compiled to create a migration tool capable of migrating existing libdb databases to SQLite.

  • Other developers: N/A (not needed for this Change)
  • Release engineering: N/A (not needed for this Change)
  • Policies and guidelines: N/A (not needed for this Change)
  • Trademark approval: N/A (not needed for this Change)
  • Alignment with Community Initiatives: Support Mark libdb as deprecated

:link: Upgrade/compatibility impact

Bogofilter can support only one database backend at a time, and thus a new SQLite bogofilter package will be unable to process libdb data. Thus the new package provides a migration script.

:link: How To Test

This test generates a word list and migrates it to work with the new SQLite backend. Install original bogofilter and add at least one word to its database, for example with:

echo abc | bogofilter --bogofilter-dir=/tmp/bftest/ --register-spam

Bogofilter will create the directory /tmp/bftest/, and it will contain a wordslist.db file. To verify the word had been added run:

bogoutil -d /tmp/bftest/wordlist.db

Install the updated bogofilter and migrate the existing libdb database with:

bogomigrate-berkeley /tmp/bftest/wordlist.db

This tool will print whether the migration succeeded. Verify the “abc” word is present in the newly created SQLite database with:

bogoutil -d /tmp/bftest/wordlist.db

:link: User Experience

:link: Dependencies

N/A (not needed for this Change)

:link: Contingency Plan

  • Contingency mechanism: N/A (not a System Wide Change)
  • Contingency deadline: N/A (not a System Wide Change)
  • Blocks release? No

:link: Documentation

N/A (not a System Wide Change)

:link: Release Notes

The bogofilter package switched its database engine from Berkeley DB (libdb) to SQLite because Fedora deprecated libdb. Users can migrate their word lists manually with bogomigrate-berkeley ~/.bogofiler/wordlist.db.

2

This change proposal has now been submitted to FESCo for voting and this topic thread will now be closed.

Ticket #3091

For details on the next phase of the process, visit our Changes Policy documentation.

3
4
5

Using my new-found powers for good, and after a good suggestion from a friend who has been involved in the Fedora project for a long, long time, ( :grin:) I will leave the topics open until their respective deadlines - see F40 schedule for exact dates.

I will use the date for the System-Wide changes to close topics related to this type of change, and same for changes tagged as Self-Contained.

Sorry for the noise and I hope this works out quite well for us :slight_smile:
Aoife

6

Bogofilter can support only one database backend at a time, and thus a new SQLite bogofilter package will be unable to process libdb data. Thus the new package provides a migration script.

OK, but what happens if the user doesn’t do the migration? Is there some clear message, data loss, etc?

Could we make it so that the conversion happens automatically?

7

Bogofilter claims an error about the database format in such case. The migration tool cannot be run automatically, that would require either changes in the upstream code, or write wrapper scripts to each bogofilter command and run the migration tool, if needed. The tool is compiled conditionally in the .spec file, thus it might not be always available.

In fact, I did think of migrating the database on the package update in users’ homes, but then I moved away from that idea, because it’s not good to touch users’ homes under their hands. I think I’ve been told so as well.

8

Yes, doing that during package install is explicitly forbidden by the packaging guidelines. See No Files or Directories Under /srv, /usr/local, or /home/$USER.