MariaDB:10.5 fails to start service

Ask Fedora
, , ,
1

For some time, maybe since the upgrade to Fedora 39 (KDE Workbench version) I have been getting errors while using dnf to upgrade the system. The upgrade complete, but there will be a message about not being able to connect to MariaDB 10.5. Since a reference to F38 was included, I figured this was an artifact of some sort, something that broke in the upgrade.

But today I tried invoking KMail, which failed with the complaint that the Akonadi service was disabled. Some research indicated that Akonadi relied on MariaDB, and further exploration showed that indeed MariaDB service was not running. Why?

MariaDB’s web site suggested several possible reasons for failure to start, and I tried a number of them. I have some more information but not an answer to the problem.

Invoking mariadb manually in a bash shell returns the following message:
“ERROR 2002 (HY000): Can’t connect to local MySQL server through socket ‘/var/lib/mysql/mysql.sock’ (2)” There is no such file in /var/lib/mysql visible to me, although this may be created by a process upon db startup - I can’t say.

I looked in the error log in /var/log/mysql and found a message that stated:
“Referenced but unset environment variable evaluates to an empty string: MYSQLD_OPTS, _WSREP_NEW_CLUSTER”. I was unable to find where that option file is, but MariaDB website suggested this might be a source of difficulty.

So, the bottom line is that I have a service that is failing to start for reasons unknown. If anyone is familiar with the problem i would welcome their suggestions. Thanks.

-CH-

2

Is mariadb.service running?
systemctl status mariadb.service should answer that.

That service must be running before the DB can be used by anything.
Even the mariadb command does not work without the service actively running.

$ systemctl status mariadb.service
● mariadb.service - MariaDB 10.5 database server
     Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; preset: disabled)
    Drop-In: /usr/lib/systemd/system/service.d
             └─10-timeout-abort.conf
     Active: active (running) since Sat 2024-04-13 10:58:07 CDT; 4 days ago
       Docs: man:mariadbd(8)
             https://mariadb.com/kb/en/library/systemd/
   Main PID: 2229 (mariadbd)
     Status: "Taking your SQL requests now..."
      Tasks: 8 (limit: 38320)
     Memory: 88.3M
        CPU: 15.178s
     CGroup: /system.slice/mariadb.service
             └─2229 /usr/libexec/mariadbd --basedir=/usr

Apr 13 10:58:07 eagle.home.domain systemd[1]: Starting mariadb.service - MariaDB 10.5 database server...
Apr 13 10:58:07 eagle.home.domain mariadb-prepare-db-dir[2188]: Database MariaDB is probably initialized in /var/lib/mysql already, nothing is done.
Apr 13 10:58:07 eagle.home.domain mariadb-prepare-db-dir[2188]: If this is not the case, make sure the /var/lib/mysql is empty before running mariadb-prepare-db-dir.
Apr 13 10:58:07 eagle.home.domain (mariadbd)[2229]: mariadb.service: Referenced but unset environment variable evaluates to an empty string: MYSQLD_OPTS, _WSREP_NEW_CLUSTER
Apr 13 10:58:07 eagle.home.domain systemd[1]: Started mariadb.service - MariaDB 10.5 database server.

My db server is functional even with that error string.

3

service is not running: see below:

chudson@syn-2603-6010-dc40-b500-09ae-3449-973e-1811:~$ systemctl status mariadb.service
× mariadb.service - MariaDB 10.5 database server
Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; preset: disabled)
Drop-In: /usr/lib/systemd/system/service.d
└─10-timeout-abort.conf
Active: failed (Result: exit-code) since Thu 2024-04-18 08:51:16 EDT; 47min ago
Docs: man:mariadbd(8)
systemd - MariaDB Knowledge Base
Process: 962 ExecStartPre=/usr/libexec/mariadb-check-socket (code=exited, status=0/SUCCESS)
Process: 994 ExecStartPre=/usr/libexec/mariadb-prepare-db-dir mariadb.service (code=exited, status=0/SUCCESS)
Process: 1047 ExecStart=/usr/libexec/mariadbd --basedir=/usr $MYSQLD_OPTS $_WSREP_NEW_CLUSTER (code=exited, status=1/FAILURE)
Main PID: 1047 (code=exited, status=1/FAILURE)
Status: “MariaDB server is down”
CPU: 349ms

Apr 18 08:51:15 fedora systemd[1]: Starting mariadb.service - MariaDB 10.5 database server…
Apr 18 08:51:15 fedora mariadb-prepare-db-dir[994]: Database MariaDB is probably initialized in /var/lib/mysql already, nothing is done.
Apr 18 08:51:15 fedora mariadb-prepare-db-dir[994]: If this is not the case, make sure the /var/lib/mysql is empty before running mariadb-prepare-db-dir.
Apr 18 08:51:15 fedora (mariadbd)[1047]: mariadb.service: Referenced but unset environment variable evaluates to an empty string: MYSQLD_OPTS, _WSREP_NEW_C>
Apr 18 08:51:16 fedora systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE
Apr 18 08:51:16 fedora systemd[1]: mariadb.service: Failed with result ‘exit-code’.
Apr 18 08:51:16 fedora systemd[1]: Failed to start mariadb.service - MariaDB 10.5 database server.
lines 1-21/21 (END)

4

This part explicitly tells you why it failed.
Did you check the status reported and take the suggested actions?

5

In /var/lib/mysql are the following entries: aria_log_control, ddl_recovery-backup.log, ibdata1, multi-master.info, mysql_upgrade_info, aria_log.00000001, ib_buffer_pool, ib_logfile0 (files) and sub-directories COOKBOOK, mysql and performance_schema.

From this I conclude that the mariadb database is already initialized and that I do not need to run mariadb-prepare-db-dir. However, the service is not running and fails when invoked manually.

6

Apparently the service is trying to run that command, as if it had not already been set up.

7

More research pointed to /etc/my.cnf.d/mariadb-server.cnf, which contains configuration including where to log errors:
/var/log/mariadb/mariadb.log. I consulted this file and found:

“2024-04-17 7:23:57 0 [ERROR] InnoDB: Unsupported redo log format. The redo log was created with MariaDB 10.9.4.
2024-04-17 7:23:57 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
2024-04-17 7:23:57 0 [Note] InnoDB: Starting shutdown…
2024-04-17 7:23:57 0 [ERROR] Plugin ‘InnoDB’ registration as a STORAGE ENGINE failed.
2024-04-17 7:23:57 0 [Note] Plugin ‘FEEDBACK’ is disabled.
2024-04-17 7:23:57 0 [ERROR] Unknown/unsupported storage engine: InnoDB
2024-04-17 7:23:57 0 [ERROR] Aborting”

The redo log format was unrecognized, apparently. I don’t know where the redo log is nor how it was created with 10.9, unless another app is creating it.

8

More research reveals that the redo log is created by InnoDB:
" The redo log is used by InnoDB during crash recovery and background flushing of transactions to the tablespaces. The redo log files have names like ib_logfileN , where N is an integer. From MariaDB 10.5, there is only one redo log, so the file will always be named ib_logfile0 . If the innodb_log_group_home_dir system variable is configured, then the redo log files will be created in that directory. Otherwise, they will be created in the directory defined by the datadir system variable."

I looked in /var/lib/mysql/ and found the file named ib_logfile0 which in its contents it listed 10.9. I renamed the file and invoked “systemctl start mariadb”. A status command revealed the service had started successfully and on checking the mysql subdirectory I found a new file ib_logfile0 had been created.

I still don’t know how the 10.9 version was created.

1 Like