I have a Fedora IoT machine that I use as a reverse proxy. For months, Fedora 41 + Nginx served that role well, proxying data to the right webservers. When Fedora IoT moved to Fedora 42, one of my servers (InvoiceNinja + Debian + Nginx) started to exhibit issues.
InvoiceNinja users could no longer view client PDFs or login outside of the LAN. They could view the site login page and client landing pages. For users on LAN everything functions just fine. Using the magic of Fedora IoT, I rebooted the device into the previous Fedora 41 ostree branch and everything works again. What changed in Fedora 42 that might cause this? I believe the Nginx versions is the same between both: “nginx version: nginx/1.26.3”
Also I believe it is just InvoiceNinja, other webservers like Nextcloud and Bookstack seem to be fine.
Here is the proxy config file that works in Fedora 41, but not in Fedora 42.
server {
# if ($host = REDACTED.SERVER.NAME) {
# return 301 https://$host$request_uri;
# } # managed by Certbot
listen 80;
server_name REDACTED.SERVER.NAME;
# return 404; # managed by Certbot
location / {
proxy_pass http://XXX.XXX.X.XX;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
server {
listen 443 ssl;
server_name REDACTED.SERVER.NAME;
location / {
proxy_pass https://XXX.XXX.X.XX;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
ssl_certificate /etc/letsencrypt/live/REDACTED.SERVER.NAME/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/REDACTED.SERVER.NAME/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
Also here is the main nginx.con file:
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log notice;
pid /run/nginx.pid;
events {
worker_connections 1024;
}
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
keepalive_timeout 65;
types_hash_max_size 4096;
client_max_body_size 12000M;
include /etc/nginx/mime.types;
default_type application/octet-stream;
include /etc/nginx/conf.d/*.conf;
server {
listen 80;
listen [::]:80;
server_name _;
root /usr/share/nginx/html;
# Load configuration files for the default server block.
include /etc/nginx/default.d/*.conf;
}
}
Edit: If any other information would be helpful, please feel free to let me know.
@miabbott Thank you for the response. I follow your advice and looked at the diff between the commits. Lots of changes, but nothing jumped out to me as a culprit.
Is there anything in the nginx logs? Or logs on the webservers that would indicate the issue? -miabbott
I had looked over my logs initially, but took a glance again today and there it was.
According to this very old post, It is a permission error on “/var/lib/nginx/tmp/proxy”. I checked the folder with ls -l /var/lib/nginx/tmp/proxy/ and it showed all users and groups to be 980:980. I ran the following, as my webuser is nginx, chown -R nginx:root /var/lib/nginx/tmp/proxy && systemctl restart nginx and the now PDFs load again on my InvoiceNinja server. Not sure why the ownership changed in F42 and properly reverts when I rolled back to F41. Over the next few days I will continue to look for other regressions, but it seems back to normal. Thank you very much for the help and also for getting me to read the logs more carefully .
.