For some time now, with any channel version of Google Chrome installed from the upstream (Google) repos, I’ve been experiencing an issue where it will crash immediately on startup ~80-90% of the time. Repeated attempts to launch it will eventually get it running, but it can sometimes take literally 10-15 tries before one “takes”. Everything seems fine once it does start, but the vast majority of launch attempts end in an immediate segfault and coredump.
This is different from the reports of “Chrome crashes a lot” that occur after startup (while browsing), or any of the reported “Chrome runs but renders/displays/browses incorrectly” issues.
I’m on a Fedora 40 system (but this dates back to F39, IIRC), across many Chrome versions, when running in GNOME Shell on Wayland with an Nvidia GT710 card using the nouveau drivers.
console output for failed launch
Sep 26 09:34:19 teevey systemd[3986]: Started app-gnome-google\x2dchrome-1294575.scope - Application launched by gnome-shell. Sep 26 09:34:20 teevey google-chrome.desktop[1294581]: [0926/093420.109622:ERROR:elf_dynamic_array_reader.h(64)] tag not found Sep 26 09:34:20 teevey google-chrome.desktop[1294581]: [0926/093420.120307:ERROR:directory_reader_posix.cc(43)] opendir /home/ferd/.config/google-chrome/Crash Reports/attachments/ad099065-dda1-4095-86d2-82e234f9d1ce: No such file or directory (2) Sep 26 09:34:20 teevey systemd-coredump[1294619]: [🡕] Process 1294575 (chrome) of user 1000 dumped core.Module libdconfsettings.so from rpm dconf-0.40.0-12.fc40.x86_64 Module libogg.so.0 from rpm libogg-1.3.5-8.fc40.x86_64 Module libvorbis.so.0 from rpm libvorbis-1.3.7-10.fc40.x86_64 Module libltdl.so.7 from rpm libtool-2.4.7-10.fc40.x86_64 Module libtdb.so.1 from rpm libtdb-1.4.10-1.fc40.x86_64 Module libvorbisfile.so.3 from rpm libvorbis-1.3.7-10.fc40.x86_64 Module libgthread-2.0.so.0 from rpm glib2-2.80.3-1.fc40.x86_64 Module libgvfscommon.so from rpm gvfs-1.54.2-1.fc40.x86_64 Module libgvfsdbus.so from rpm gvfs-1.54.2-1.fc40.x86_64 Module libX11-xcb.so.1 from rpm libX11-1.8.10-2.fc40.x86_64 Module libjson-glib-1.0.so.0 from rpm json-glib-1.8.0-3.fc40.x86_64 Module libjpeg.so.62 from rpm libjpeg-turbo-3.0.2-1.fc40.x86_64 Module libXinerama.so.1 from rpm libXinerama-1.1.5-6.fc40.x86_64 Module libXcursor.so.1 from rpm libXcursor-1.2.1-7.fc40.x86_64 Module libwayland-egl.so.1 from rpm wayland-1.23.0-2.fc40.x86_64 Module libwayland-cursor.so.0 from rpm wayland-1.23.0-2.fc40.x86_64 Module libwayland-client.so.0 from rpm wayland-1.23.0-2.fc40.x86_64 Module libtracker-sparql-3.0.so.0 from rpm tracker-3.7.3-1.fc40.x86_64 Module libcloudproviders.so.0 from rpm libcloudproviders-0.3.5-3.fc40.x86_64 Module libepoxy.so.0 from rpm libepoxy-1.5.10-6.fc40.x86_64 Module libgdk_pixbuf-2.0.so.0 from rpm gdk-pixbuf2-2.42.10-8.fc40.x86_64 Module libcairo-gobject.so.2 from rpm cairo-1.18.0-3.fc40.x86_64 Module libpangoft2-1.0.so.0 from rpm pango-1.54.0-1.fc40.x86_64 Module libpangocairo-1.0.so.0 from rpm pango-1.54.0-1.fc40.x86_64 Module libgdk-3.so.0 from rpm gtk3-3.24.43-1.fc40.x86_64 Module libgtk-3.so.0 from rpm gtk3-3.24.43-1.fc40.x86_64 Module libbrotlicommon.so.1 from rpm brotli-1.1.0-3.fc40.x86_64 Module libbrotlidec.so.1 from rpm brotli-1.1.0-3.fc40.x86_64 Module libbz2.so.1 from rpm bzip2-1.0.8-18.fc40.x86_64 Module libxml2.so.2 from rpm libxml2-2.12.8-1.fc40.x86_64 Module libgraphite2.so.3 from rpm graphite2-1.3.14-15.fc40.x86_64 Module libdatrie.so.1 from rpm libdatrie-0.2.13-9.fc40.x86_64 Module libblkid.so.1 from rpm util-linux-2.40.1-1.fc40.x86_64 Module libgmp.so.10 from rpm gmp-6.2.1-8.fc40.x86_64 Module libhogweed.so.6 from rpm nettle-3.9.1-6.fc40.x86_64 Module libnettle.so.8 from rpm nettle-3.9.1-6.fc40.x86_64 Module libtasn1.so.6 from rpm libtasn1-4.19.0-6.fc40.x86_64 Module libunistring.so.5 from rpm libunistring-1.1-7.fc40.x86_64 Module libidn2.so.0 from rpm libidn2-2.3.7-1.fc40.x86_64 Module libp11-kit.so.0 from rpm p11-kit-0.25.5-1.fc40.x86_64 Module libcrypto.so.3 from rpm openssl-3.2.2-3.fc40.x86_64 Module libkeyutils.so.1 from rpm keyutils-1.6.3-3.fc40.x86_64 Module libkrb5support.so.0 from rpm krb5-1.21.3-1.fc40.x86_64 Module libcom_err.so.2 from rpm e2fsprogs-1.47.0-5.fc40.x86_64 Module libk5crypto.so.3 from rpm krb5-1.21.3-1.fc40.x86_64 Module libkrb5.so.3 from rpm krb5-1.21.3-1.fc40.x86_64 Module libzstd.so.1 from rpm zstd-1.5.6-1.fc40.x86_64 Module liblzma.so.5 from rpm xz-5.4.6-3.fc40.x86_64 Module liblz4.so.1 from rpm lz4-1.9.4-6.fc40.x86_64 Module libcap.so.2 from rpm libcap-2.69-8.fc40.x86_64 Module libpixman-1.so.0 from rpm pixman-0.43.4-1.fc40.x86_64 Module libxcb-shm.so.0 from rpm libxcb-1.17.0-1.fc40.x86_64 Module libxcb-render.so.0 from rpm libxcb-1.17.0-1.fc40.x86_64 Module libfreetype.so.6 from rpm freetype-2.13.2-5.fc40.x86_64 Module libfontconfig.so.1 from rpm fontconfig-2.15.0-6.fc40.x86_64 Module libpng16.so.16 from rpm libpng-1.6.40-3.fc40.x86_64 Module libharfbuzz.so.0 from rpm harfbuzz-8.5.0-1.fc40.x86_64 Module libthai.so.0 from rpm libthai-0.1.29-8.fc40.x86_64 Module libfribidi.so.0 from rpm fribidi-1.0.14-2.fc40.x86_64 Module libxcb-randr.so.0 from rpm libxcb-1.17.0-1.fc40.x86_64 Module libwayland-server.so.0 from rpm wayland-1.23.0-2.fc40.x86_64 Module libXrender.so.1 from rpm libXrender-0.9.11-6.fc40.x86_64 Module libXi.so.6 from rpm libXi-1.8.2-1.fc40.x86_64 Module libXau.so.6 from rpm libXau-1.0.11-6.fc40.x86_64 Module libselinux.so.1 from rpm libselinux-3.7-5.fc40.x86_64 Module libmount.so.1 from rpm util-linux-2.40.1-1.fc40.x86_64 Module libgmodule-2.0.so.0 from rpm glib2-2.80.3-1.fc40.x86_64 Module libz.so.1 from rpm zlib-ng-2.1.7-2.fc40.x86_64 Module libgnutls.so.30 from rpm gnutls-3.8.6-1.fc40.x86_64 Module libavahi-client.so.3 from rpm avahi-0.8-26.fc40.x86_64 Module libavahi-common.so.3 from rpm avahi-0.8-26.fc40.x86_64 Module libgssapi_krb5.so.2 from rpm krb5-1.21.3-1.fc40.x86_64 Module libsystemd.so.0 from rpm systemd-255.12-1.fc40.x86_64 Module libplds4.so from rpm nss-3.104.0-1.fc40.x86_64 Module libplc4.so from rpm nss-3.104.0-1.fc40.x86_64 Module libpcre2-8.so.0 from rpm pcre2-10.44-1.fc40.x86_64 Module libffi.so.8 from rpm libffi-3.4.4-7.fc40.x86_64 Module libasound.so.2 from rpm alsa-lib-1.2.12-1.fc40.x86_64 Module libudev.so.1 from rpm systemd-255.12-1.fc40.x86_64 Module libcairo.so.2 from rpm cairo-1.18.0-3.fc40.x86_64 Module libpango-1.0.so.0 from rpm pango-1.54.0-1.fc40.x86_64 Module libgbm.so.1 from rpm mesa-24.1.7-1.fc40.x86_64 Module libXrandr.so.2 from rpm libXrandr-1.5.4-3.fc40.x86_64 Module libXfixes.so.3 from rpm libXfixes-6.0.1-3.fc40.x86_64 Module libXext.so.6 from rpm libXext-1.3.6-1.fc40.x86_64 Module libXdamage.so.1 from rpm libXdamage-1.1.6-3.fc40.x86_64 Module libXcomposite.so.1 from rpm libXcomposite-0.4.6-3.fc40.x86_64 Module libX11.so.6 from rpm libX11-1.8.10-2.fc40.x86_64 Module libatspi.so.0 from rpm at-spi2-core-2.52.0-1.fc40.x86_64 Module libxkbcommon.so.0 from rpm libxkbcommon-1.6.0-2.fc40.x86_64 Module libxcb.so.1 from rpm libxcb-1.17.0-1.fc40.x86_64 Module libexpat.so.1 from rpm expat-2.6.3-1.fc40.x86_64 Module libdrm.so.2 from rpm libdrm-2.4.123-1.fc40.x86_64 Module libgio-2.0.so.0 from rpm glib2-2.80.3-1.fc40.x86_64 Module libcups.so.2 from rpm cups-2.4.10-6.fc40.x86_64 Module libatk-bridge-2.0.so.0 from rpm at-spi2-core-2.52.0-1.fc40.x86_64 Module libatk-1.0.so.0 from rpm at-spi2-core-2.52.0-1.fc40.x86_64 Module libdbus-1.so.3 from rpm dbus-1.14.10-3.fc40.x86_64 Module libnspr4.so from rpm nss-3.104.0-1.fc40.x86_64 Module libglib-2.0.so.0 from rpm glib2-2.80.3-1.fc40.x86_64 Module libgobject-2.0.so.0 from rpm glib2-2.80.3-1.fc40.x86_64 Stack trace of thread 1294599: #0 0x00005654026935d3 n/a (chrome + 0x9f1d5d3) #1 0x0000565400f4ca7b n/a (chrome + 0x87d6a7b) #2 0x0000565402698cbf n/a (chrome + 0x9f22cbf) #3 0x00005654022dd3ad n/a (chrome + 0x9b673ad) #4 0x000056540269857b __libc_malloc (chrome + 0x9f2257b) #5 0x00007f1bd9785b6f __strdup (libc.so.6 + 0xaeb6f) #6 0x00005653fcf72fcc n/a (chrome + 0x47fcfcc) #7 0x00005653fcf733fd n/a (chrome + 0x47fd3fd) #8 0x00005653fcf7358b n/a (chrome + 0x47fd58b) #9 0x00005653fd16aca6 n/a (chrome + 0x49f4ca6) #10 0x00005653fd177830 n/a (chrome + 0x4a01830) #11 0x00005653fd16dbd9 n/a (chrome + 0x49f7bd9) #12 0x00005653fb9f3ea3 n/a (chrome + 0x327dea3) #13 0x00005653fe492b09 n/a (chrome + 0x5d1cb09) #14 0x00005653fe4927e2 n/a (chrome + 0x5d1c7e2) #15 0x00005653fb3fd591 n/a (chrome + 0x2c87591) #16 0x00005653fb405ec1 n/a (chrome + 0x2c8fec1) #17 0x00005653fb403560 n/a (chrome + 0x2c8d560) #18 0x00005653fb402c9d n/a (chrome + 0x2c8cc9d) #19 0x00005653fc505e8b n/a (chrome + 0x3d8fe8b) #20 0x00005653fd14d1f2 n/a (chrome + 0x49d71f2) #21 0x00007f1bd976e6d7 start_thread (libc.so.6 + 0x976d7) #22 0x00007f1bd97f260c __clone3 (libc.so.6 + 0x11b60c) Stack trace of thread 1294575: #0 0x00005653fef43830 n/a (chrome + 0x67cd830) #1 0x00005653fe6cc278 n/a (chrome + 0x5f56278) #2 0x00005653fe6cbfe5 n/a (chrome + 0x5f55fe5) #3 0x00005653fedf2833 n/a (chrome + 0x667c833) #4 0x00005653fd095516 n/a (chrome + 0x491f516) #5 0x00005653fe49f8e8 n/a (chrome + 0x5d298e8) #6 0x00005653fe49eed5 n/a (chrome + 0x5d28ed5) #7 0x00005653fe684a55 n/a (chrome + 0x5f0ea55) #8 0x00005653fda7e7ea n/a (chrome + 0x53087ea) #9 0x00005653fdc8e737 n/a (chrome + 0x5518737) #10 0x00005653fdc8e5dd n/a (chrome + 0x55185dd) #11 0x00005653fdc8f45b n/a (chrome + 0x551945b) #12 0x00005653fd4100d8 n/a (chrome + 0x4c9a0d8) #13 0x0000565401dd7317 n/a (chrome + 0x9661317) #14 0x00005653fbea38b4 n/a (chrome + 0x372d8b4) #15 0x00005653fd804e3f ChromeMain (chrome + 0x508ee3f) #16 0x00007f1bd9701088 __libc_start_call_main (libc.so.6 + 0x2a088) #17 0x00007f1bd970114b __libc_start_main@@GLIBC_2.34 (libc.so.6 + 0x2a14b) #18 0x00005653feff3dda _start (chrome + 0x687ddda) Stack trace of thread 1294596: #0 0x00007f1bd976ada9 __futex_abstimed_wait_common (libc.so.6 + 0x93da9) #1 0x00007f1bd976db62 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x96b62) #2 0x00005653fb54a35e n/a (chrome + 0x2dd435e) #3 0x00005653fe420499 n/a (chrome + 0x5caa499) #4 0x0000565402669754 n/a (chrome + 0x9ef3754) #5 0x00005653fd14d1f2 n/a (chrome + 0x49d71f2) #6 0x00007f1bd976e6d7 start_thread (libc.so.6 + 0x976d7) #7 0x00007f1bd97f260c __clone3 (libc.so.6 + 0x11b60c) Stack trace of thread 1294589: #0 0x00007f1bd97e487d __poll (libc.so.6 + 0x10d87d) #1 0x00005653fd6cd747 n/a (chrome + 0x4f57747) #2 0x00005653fd14d1f2 n/a (chrome + 0x49d71f2) #3 0x00007f1bd976e6d7 start_thread (libc.so.6 + 0x976d7) #4 0x00007f1bd97f260c __clone3 (libc.so.6 + 0x11b60c) Stack trace of thread 1294595: #0 0x00007f1bd97dae13 wait4 (libc.so.6 + 0x103e13) #1 0x00005653fded40ab n/a (chrome + 0x575e0ab) #2 0x00005653fded3d8d n/a (chrome + 0x575dd8d) #3 0x00005653fd14d1f2 n/a (chrome + 0x49d71f2) #4 0x00007f1bd976e6d7 start_thread (libc.so.6 + 0x976d7) #5 0x00007f1bd97f260c __clone3 (libc.so.6 + 0x11b60c) Stack trace of thread 1294600: #0 0x00007f1bd976ada9 __futex_abstimed_wait_common (libc.so.6 + 0x93da9) #1 0x00007f1bd976db62 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x96b62) #2 0x00005653fb403e1c n/a (chrome + 0x2c8de1c) #3 0x00005653fb402c9d n/a (chrome + 0x2c8cc9d) #4 0x00005653fc505e8b n/a (chrome + 0x3d8fe8b) #5 0x00005653fd14d1f2 n/a (chrome + 0x49d71f2) #6 0x00007f1bd976e6d7 start_thread (libc.so.6 + 0x976d7) #7 0x00007f1bd97f260c __clone3 (libc.so.6 + 0x11b60c) Stack trace of thread 1294601: #0 0x00007f1bd976ada9 __futex_abstimed_wait_common (libc.so.6 + 0x93da9) #1 0x00007f1bd976db62 pthread_cond_timedwait@@GLIBC_2.3.2 (libc.so.6 + 0x96b62) #2 0x00005653fb403e1c n/a (chrome + 0x2c8de1c) #3 0x00005653fc505f5d n/a (chrome + 0x3d8ff5d) #4 0x00005653fc505ec7 n/a (chrome + 0x3d8fec7) #5 0x00005653fd14d1f2 n/a (chrome + 0x49d71f2) #6 0x00007f1bd976e6d7 start_thread (libc.so.6 + 0x976d7) #7 0x00007f1bd97f260c __clone3 (libc.so.6 + 0x11b60c) Stack trace of thread 1294605: #0 0x00007f1bd97f03dd syscall (libc.so.6 + 0x1193dd) #1 0x00007f1bda4ecccd g_cond_wait (libglib-2.0.so.0 + 0xbbccd) #2 0x00007f1bda45861b g_async_queue_pop_intern_unlocked (libglib-2.0.so.0 + 0x2761b) #3 0x00007f1bda4bfa03 g_thread_pool_spawn_thread (libglib-2.0.so.0 + 0x8ea03) #4 0x00007f1bda4be813 g_thread_proxy (libglib-2.0.so.0 + 0x8d813) #5 0x00007f1bd976e6d7 start_thread (libc.so.6 + 0x976d7) #6 0x00007f1bd97f260c __clone3 (libc.so.6 + 0x11b60c)
HOWEVER, I’ve discovered that if I run Chrome from the command line and pass it the argument --ozone-platform=wayland, it will launch correctly seemingly every time. It’s still plenty pissed off about my GPU and its failings, but it runs and browses pretty much without incident. So, it appears that it may be the ozone platform’s wayland/x11 autodetection that’s falling apart somewhere.
$ google-chrome --ozone-platform=wayland
[1294769:1294769:0926/093445.473905:ERROR:object_proxy.cc(576)] Failed to call method: org.freedesktop.ScreenSaver.GetActive: object_path= /org/freedesktop/ScreenSaver: org.freedesktop.DBus.Error.NotSupported: This method is not part of the idle inhibition specification: https://specifications.freedesktop.org/idle-inhibit-spec/latest/
[1294814:1294814:0926/093445.775614:ERROR:gbm_wrapper.cc(79)] Failed to get fd for plane.: No such file or directory (2)
[1294814:1294814:0926/093445.786133:ERROR:gbm_pixmap_wayland.cc(82)] Cannot create bo with format= RGBA_8888 and usage=SCANOUT
[1294814:1294814:0926/093445.786836:ERROR:gbm_wrapper.cc(79)] Failed to get fd for plane.: No such file or directory (2)
[1294814:1294814:0926/093445.788557:ERROR:gbm_wrapper.cc(79)] Failed to get fd for plane.: No such file or directory (2)
[1294814:1294814:0926/093445.791149:ERROR:gbm_pixmap_wayland.cc(82)] Cannot create bo with format= RGBA_8888 and usage=GPU_READ
[1294814:1294814:0926/093445.791246:ERROR:shared_image_factory.cc(899)] CreateSharedImage: could not create backing.
[1294814:1294814:0926/093445.794235:ERROR:shared_image_manager.cc(248)] SharedImageManager::ProduceSkia: Trying to Produce a Skia representation from a non-existent mailbox.
[1294814:1294814:0926/093445.794529:ERROR:gpu_service_impl.cc(1159)] Exiting GPU process because some drivers can't recover from errors. GPU process will restart shortly.
[1294769:1294769:0926/093445.876585:ERROR:gpu_process_host.cc(980)] GPU process exited unexpectedly: exit_code=8704
[1295102:1295102:0926/093447.007523:ERROR:shared_image_manager.cc(248)] SharedImageManager::ProduceSkia: Trying to Produce a Skia representation from a non-existent mailbox.
[1295102:1295102:0926/093447.007761:ERROR:gpu_service_impl.cc(1159)] Exiting GPU process because some drivers can't recover from errors. GPU process will restart shortly.
[1295017:1:0926/093447.053944:ERROR:command_buffer_proxy_impl.cc(324)] GPU state invalid after WaitForGetOffsetInRange.
[1294892:10:0926/093447.068817:ERROR:command_buffer_proxy_impl.cc(131)] ContextResult::kTransientFailure: Failed to send GpuControl.CreateCommandBuffer.
[1294769:1294769:0926/093447.176868:ERROR:gpu_process_host.cc(980)] GPU process exited unexpectedly: exit_code=8704
[1294904:1:0926/093447.327768:ERROR:command_buffer_proxy_impl.cc(131)] ContextResult::kTransientFailure: Failed to send GpuControl.CreateCommandBuffer.
[1295165:1295165:0926/093448.228591:ERROR:gbm_wrapper.cc(79)] Failed to get fd for plane.: No such file or directory (2)
[1295165:1295165:0926/093448.234207:ERROR:gbm_pixmap_wayland.cc(82)] Cannot create bo with format= RGBA_8888 and usage=SCANOUT
[1295165:1295165:0926/093448.234335:ERROR:gbm_wrapper.cc(79)] Failed to get fd for plane.: No such file or directory (2)
[1295165:1295165:0926/093448.237320:ERROR:gbm_pixmap_wayland.cc(82)] Cannot create bo with format= RGBA_8888 and usage=GPU_READ
[1295165:1295165:0926/093448.237405:ERROR:shared_image_factory.cc(899)] CreateSharedImage: could not create backing.
[1295165:1295165:0926/093448.237872:ERROR:shared_image_manager.cc(248)] SharedImageManager::ProduceSkia: Trying to Produce a Skia representation from a non-existent mailbox.
[1295165:1295165:0926/093448.238022:ERROR:gpu_service_impl.cc(1159)] Exiting GPU process because some drivers can't recover from errors. GPU process will restart shortly.
Created TensorFlow Lite XNNPACK delegate for CPU.
[1294769:1294769:0926/093448.466956:ERROR:gpu_process_host.cc(980)] GPU process exited unexpectedly: exit_code=8704
[1294939:7:0926/093448.911191:ERROR:command_buffer_proxy_impl.cc(131)] ContextResult::kTransientFailure: Failed to send GpuControl.CreateCommandBuffer.
[1294948:7:0926/093448.912626:ERROR:command_buffer_proxy_impl.cc(131)] ContextResult::kTransientFailure: Failed to send GpuControl.CreateCommandBuffer.
[1295224:1295224:0926/093449.139951:ERROR:gl_utils.cc(426)] [.WebGL-0x279c000f8600]GL Driver Message (OpenGL, Performance, GL_CLOSE_PATH_NV, High): GPU stall due to ReadPixels
[1295224:1295224:0926/093449.227988:ERROR:gl_utils.cc(426)] [.WebGL-0x279c000f8600]GL Driver Message (OpenGL, Performance, GL_CLOSE_PATH_NV, High): GPU stall due to ReadPixels
Fontconfig error: Cannot load default config file: No such file: (null)
Fontconfig error: "memory", line 3: out of memory; cannot add directory fonts
[1295224:1295224:0926/093449.342174:ERROR:gl_utils.cc(426)] [.WebGL-0x279c000f8600]GL Driver Message (OpenGL, Performance, GL_CLOSE_PATH_NV, High): GPU stall due to ReadPixels
[1294930:7:0926/093449.589462:ERROR:command_buffer_proxy_impl.cc(131)] ContextResult::kTransientFailure: Failed to send GpuControl.CreateCommandBuffer.
[1294930:7:0926/093449.610216:ERROR:command_buffer_proxy_impl.cc(131)] ContextResult::kTransientFailure: Failed to send GpuControl.CreateCommandBuffer.
Attempting to use a delegate that only supports static-sized tensors with a graph that has dynamic-sized tensors (tensor#58 is a dynamic-sized tensor).
Fontconfig error: Cannot load default config file: No such file: (null)
Fontconfig error: "memory", line 3: out of memory; cannot add directory fonts
(I deduplicated that output some, it’s actually much more verbose and repetitious normally.)
This is mostly just an FYI for anyone else who may be seeing the same problem, as it’s a not-uncommon one to see reported on various sites (regardless of distro).