Running Ollama Locally with Podman on Fedora Linux

News & Announcements Fedora Magazine
6

I use a Quadlet to manage my Ollama container (on AMD GPU hardware and ROCm).

Put this in ~/.config/containers/systemd/ollama.container.

[Unit]
Description=Ollama Container with ROCm Support

[Container]
ContainerName=ollama
Image=docker.io/ollama/ollama:rocm
AutoUpdate=registry
AddDevice=/dev/kfd
AddDevice=/dev/dri
Volume=ollama:/root/.ollama
Environment="OLLAMA_HOST=0.0.0.0"
Environment="OLLAMA_FLASH_ATTENTION=1"
Environment="OLLAMA_KEEP_ALIVE=30m"
PublishPort=11434:11434

[Service]
Restart=always
TimeoutStartSec=300

[Install]
WantedBy=default.target

Then run this.

$ systemctl --user daemon-reload
$ systemctl --user start ollama.service
1 Like