Docker Model Runner unable to start on remote Mac with M4 chip

[blackblather]

Hi,

I'm trying to run docker on my remote machine (mac with M4 chip) like so:
docker -H ssh://blackblather:192.168.1.151 model list

But I am facing the following error:

latest: Pulling from docker/model-runner
608c81a99d0f: Download complete
723ca77ac84e: Download complete
Digest: sha256:887f073f3a753e647df2c17fcc83487b1356d3ceab113832bc18a9d75123bf33
Status: Downloaded newer image for docker/model-runner:latest
Successfully pulled docker/model-runner:latest
Starting model runner container docker-model-runner...
unable to initialize standalone model runner: unable to initialize standalone model runner container: failed to start container docker-model-runner: Error response from daemon: ports are not available: exposing port TCP 172.17.0.1:12434 -> 127.0.0.1:0: listen tcp4 172.17.0.1:12434: bind: can't assign requested address

I think this issue is not related to the SSH connection per se, but rather to running DMR on a remote machine.
I say this because I tried using a "direct TCP connection" to the remote server, like this:

• socat TCP-LISTEN:2376,reuseaddr,fork UNIX-CLIENT:/var/run/docker.sock (on the remote machine)
• docker -H 192.168.1.151:2376 model list (on the client machine)

and the same behavior happens.

It pulls the docker/model-runner docker image on the remote machine, but cannot start it.
I did notice in the DockerHub link for this image it mentions: These images aren't currently intended for direct use by end-users.

However, I did try to run this image directly in the mac to see what happens, using docker run -p 12434:12434 docker/model-runner, and it prints:

time="2026-02-11T21:10:27Z" level=info msg="Successfully initialized store" component=model-manager
time="2026-02-11T21:10:27Z" level=info msg="LLAMA_SERVER_PATH: /app/bin"
time="2026-02-11T21:10:27Z" level=info msg="Metrics endpoint enabled at /metrics"
time="2026-02-11T21:10:27Z" level=info msg="Listening on TCP port 12434" 👈
time="2026-02-11T21:10:27Z" level=warning msg="Backend installation failed for sglang: python3 not found in PATH"
time="2026-02-11T21:10:27Z" level=warning msg="Backend installation failed for diffusers: python3 not found in PATH"
time="2026-02-11T21:10:27Z" level=warning msg="Backend installation failed for vllm: vLLM binary not found"
time="2026-02-11T21:10:27Z" level=info msg="failed to ensure latest llama.cpp: llama.cpp auto-updated is disabled\n"
time="2026-02-11T21:10:27Z" level=info msg="installed llama-server with gpuSupport=false" 👈
time="2026-02-11T21:10:27Z" level=warning msg="Backend installation failed for mlx: MLX is only available on macOS ARM64"

I still only have surface level understanding on technical inner workings of docker, so I have some follow-up questions.

Why does this happen:

• docker -H ssh://blackblather:192.168.1.151 model list (from the client machine)
  • 🛑 docker complains that port 12434 is occupied
• docker run -p 12434:12434 docker/model-runner (from the remote machine)
  • ✅ docker doesn't complain anymore (wasn't this port "supposed" to be occupied?)

Lastly, AFAIK, the docker daemon in Mac OS runs in a VM created using the Apple Virtualization Framework (AVF), but Apple doesn't provide GPU access to VMs created using AVF

That's why when using Docker Desktop, Docker Model Runner (DMR) will run directly on the host.

If my understanding is correct:

• Why is it trying to download DMR as a container running in a VM, if that VM has no GPU access?
  • (as the logs above seem to suggest: installed llama-server with gpuSupport=false)

Thank you for your help 👍