Configurable listen address for network access

[gvagenas]

It would be nice to allow the setup the address the server will run "LLAMA_LISTEN_ADDRESS".
Right now it defaults to localhost.

[erusev]

What would be your use case for something like this? I would like to understand the problem it solves.

[gvagenas]

Hi @erusev.
By having the option to start llama-server on 0.0.0.0 I could point other tools running on different machines on my network to use the API.
For example, I have OpenWebUI running on a old and slow machine in the network, and I would like to be able use LLamabarn/llama-server API from OpenWebUI. Currently Llamabarn starts llama-server on localhost and there is no option to bind to 0.0.0.0.
Thanks :)

[ggerganov]

@gvagenas You can achieve the same by running this command:

ssh -NgL 0.0.0.0:2276:127.0.0.1:2276 $USER@localhost

This will tunnel all incoming requests on any network interface to the LlamaBarn localhost server, effectively doing the same as binding to 0.0.0.0.

[gvagenas]

Thanks @ggerganov this is very useful indeed but it would be great and more convenient if Llamabarn could support this setting.
Out of curiosity and trying to understand, is there a specific reason that the Llamabarn binds only to localhost?

Thanks for your support guys.

[erusev]

Caution

Don't use this unless you understand network security risks!

Just released 0.12.0 with an experimental option to expose llama-server to network by binding to 0.0.0.0.

To enable it:

defaults write app.llamabarn.LlamaBarn exposeToNetwork -bool true

[bhubbb]

It would be nicer if we could pass a bool for expose or a list for the IP we want to bind to:

defaults write app.llamabarn.LlamaBarn bindToNetwork 10.0.0.1

[erusev]

@bhubbb why a list (and not just a single IP)?

[bhubbb]

It can be a single. That would cover 99% of cases.