Setting Up Jellyfin with Hardware Transcoding

Setting Up Jellyfin with Hardware Transcoding

Jellyfin is the media server this site recommends as a first self-hosting project. It is self-contained, useful immediately, and failing at it costs you an evening rather than your data.

The part people get wrong is transcoding.

Deploying it

services:
  jellyfin:
    image: jellyfin/jellyfin:latest
    user: "1000:1000"
    group_add:
      - "989"            # the 'render' group GID on the host
    devices:
      - /dev/dri:/dev/dri
    volumes:
      - ./config:/config
      - ./cache:/cache
      - /srv/media:/media:ro
    ports:
      - "127.0.0.1:8096:8096"
    restart: unless-stopped

Points worth noting:

/srv/media:/media:ro mounts your library read-only. Jellyfin does not need to modify your files, and a compromised container cannot then delete them.

127.0.0.1:8096 binds to loopback only. Our container security guide covers why publishing a port otherwise bypasses your firewall entirely.

/dev/dri passes the GPU in. Without it there is no hardware transcoding, regardless of what the settings page says.

group_add is the part that catches people. The container user must be in the host’s render group to open the device:

getent group render
# render:x:989:
ls -l /dev/dri/renderD128
# crw-rw---- 1 root render 226, 128 ... /dev/dri/renderD128

Use the numeric GID from the host, not the name, because the group may not exist inside the container.

Transcoding, and why it matters

Three things can happen when a client requests a file.

Direct play. The file is sent unchanged. Costs nothing.

Direct stream. The container is repackaged, streams untouched. Cheap.

Transcode. The video is decoded and re-encoded in real time. Very expensive in software.

Transcoding happens when the client cannot handle the original: an older TV that does not do HEVC, a browser without the codec, a phone on a slow connection needing a lower bitrate, or burned-in subtitles.

A single 4K HEVC software transcode will saturate most CPUs. Two will fail.

Enabling hardware acceleration

Dashboard, then Playback, then Transcoding.

Intel Quick Sync

The best option for this job, and it does not need a dedicated card. Intel’s encoder block is purpose-built and frequently outperforms consumer GPUs for transcoding specifically.

Select VAAPI, device /dev/dri/renderD128.

sudo apt install intel-media-va-driver-non-free vainfo
vainfo | grep -E 'VAProfileH264|VAProfileHEVC'

vainfo on the host tells you which codecs the hardware supports before you go looking for configuration problems. The non-free driver package is the one with the full codec set, and Debian users need non-free enabled.

AMD

VAAPI as well, same device path, with Mesa providing the driver.

sudo apt install mesa-va-drivers

NVIDIA

Select NVENC. Needs the proprietary driver and the NVIDIA container toolkit:

    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

Consumer NVIDIA cards historically limited concurrent NVENC sessions in the driver. That limit has been relaxed on recent drivers and is worth checking for your card if you expect many simultaneous streams.

Verifying it works

The settings page claiming hardware acceleration is enabled means nothing on its own.

# while something is transcoding
sudo intel_gpu_top        # Intel
nvidia-smi                # NVIDIA, look for an encoder session
radeontop                 # AMD

docker logs jellyfin 2>&1 | grep -i -E 'vaapi|nvenc|hwaccel'
htop                      # CPU should be low, not pinned

If CPU is at 100 percent, it is not using the GPU. Check the device mapping and the group first; those account for most failures.

Which codecs to enable

Enable decoding for what your library contains, typically H.264 and HEVC. Enable hardware encoding as well, or it decodes on the GPU and encodes on the CPU, which defeats the point.

Leave tone mapping on if you have HDR content and clients that cannot display it. It is GPU-accelerated on capable hardware and expensive without.

Set transcode throttling on, which pauses transcoding when the client has buffered enough rather than racing ahead.

Point the transcode cache at fast storage, or better, at RAM:

    tmpfs:
      - /cache/transcodes:size=8g

Transcoding writes continuously to that path. On an SSD that is real write wear, and on a spinning disk it is a bottleneck.

Organising the library

This is where metadata matching succeeds or fails, and it has nothing to do with Jellyfin.

/srv/media/
├── Movies/
│   ├── Arrival (2016)/
│   │   ├── Arrival (2016) - 2160p.mkv
│   │   └── Arrival (2016) - 2160p.en.srt
│   └── Blade Runner 2049 (2017)/
│       └── Blade Runner 2049 (2017) - 1080p.mkv
└── Shows/
    └── Severance/
        └── Season 01/
            ├── Severance - S01E01 - Good News About Hell.mkv
            └── Severance - S01E02 - Half Loop.mkv

Title and year in the folder name. That is what the matcher uses.

SxxExx for episodes. Anything else and Jellyfin guesses.

External subtitles named after the file with a language code, so .en.srt sits beside the video.

A file called movie_final_2.mkv in a flat directory will not match, and no amount of configuration fixes that. Renaming is the fix, and tools like filebot automate it.

Access

Do not expose port 8096 directly.

The best answer is WireGuard and never opening anything, which our self-hosting guide argues for at length.

If you are sharing with family who will not install a VPN client, put it behind a reverse proxy with TLS from Let’s Encrypt, and consider forward authentication in front. One factor is reasonable for a media server; requiring a TOTP code to watch a film is friction people will route around.

Whatever you do, complete the setup wizard immediately after first start. An unconfigured Jellyfin lets the first visitor create the admin account.

Clients

Native apps exist for Android, iOS, Android TV, Roku, and the web client works in any browser. Kodi integrates through a plugin.

The one worth knowing about is Jellyfin Media Player on desktop, which uses mpv and handles far more formats directly than a browser can, meaning more direct play and less transcoding.

That is the general principle: the best transcoding configuration is the one that rarely runs. A client that plays your files natively costs the server nothing, and a library encoded in formats your devices actually support is worth more than any amount of GPU tuning.

Frequently Asked Questions

Why is my CPU at 100 percent when someone watches a film?

Jellyfin is transcoding in software. That happens when the client cannot play the original format, so the server decodes and re-encodes in real time, which is extremely expensive on a CPU. Enabling hardware transcoding moves that work to the GPU where it costs very little.

What is the difference between transcoding and direct play?

Direct play sends the file unchanged and costs almost nothing. Transcoding converts video, audio, or the container on the fly because the client cannot handle the original. Direct stream is in between, repackaging the container while leaving the streams alone.

Do I need a dedicated GPU for hardware transcoding?

No. Intel integrated graphics with Quick Sync are excellent for this and frequently outperform dedicated cards for transcoding specifically, because the encoder block is purpose-built. Any reasonably recent Intel CPU with integrated graphics will handle several simultaneous streams.

Why does my media not match the right metadata?

Almost always naming or folder structure. Jellyfin matches against online databases using the filename and directory, so a file named download_final2.mkv in a flat folder has nothing to work with. Following the documented naming convention resolves the overwhelming majority of matching problems.

Should I expose Jellyfin to the internet?

Prefer a VPN. If you need to share with people who will not install a VPN client, put it behind a reverse proxy with TLS and consider forward authentication in front. Never expose the setup wizard, and make sure the first thing you do after install is create an admin account with a real password.

What does the /dev/dri device do in the container configuration?

It passes the GPU render node into the container so Jellyfin can use it for hardware encoding and decoding. Without that device mapping the container has no access to the GPU at all, which is the most common reason hardware transcoding appears enabled but never engages.