CDNShark Documentation

Private LLM Gateway

← Back to Documentation

Create a Gateway and Install the Connector

Before you start, make sure your model server is running on the machine. For example, run ollama list to check Ollama is up and has at least one model.

1. Create the gateway

  1. In your dashboard, open LLM Gateway and click Create Gateway.
  2. Give it a name, such as the machine it runs on.
  3. Pick your model server. The host and port fill in for you:
    Model serverDefault address
    Ollamalocalhost:11434
    LM Studiolocalhost:1234
    vLLMlocalhost:8000
    llama.cpp serverlocalhost:8080
    Choose Custom if yours uses another port. Use localhost when the connector runs on the same machine as the model, or the model machine's LAN address, such as 192.168.1.20, when it runs on another one.
  4. Click Create Gateway.

2. Copy your API key and install command

The next page shows two things, once only:

  • Your API key, starting with sk-llm-. We only keep a fingerprint of it, so it can't be shown again. If you lose it, issue a new one (see API Keys and Security).
  • The install command, with a one-time token. It expires after 15 minutes and works once. If it runs out, click New install command.

3. Install the connector

Linux or macOS: run the command from the page on the machine next to your model:

curl -fsSL https://cdnshark.com/install/tunnel.sh | sudo bash -s -- --token=YOUR_TOKEN

On Linux it installs a systemd service that starts on boot and restarts on its own. On macOS it only installs the connector, so start it with cdnshark-tunnel run. To keep it running after a reboot, set it up as a launchd service, as described in Run the Connector as a Service on Linux and macOS.

Windows: open the Windows tab on the gateway page and follow the PowerShell steps.

It is the same connector as Private Origin Tunnel. The machine only needs outbound internet access, including outbound UDP.

4. Check it worked

Within a minute the gateway page shows the connector as Online and the gateway as Active. Then test it from any computer, using your gateway address and key:

curl https://YOUR-ID.llm.cdnshark.com/v1/models \
  -H "Authorization: Bearer YOUR_KEY"

You should see your models listed. Next: Connect Your Apps to Your Gateway.