Private LLM Gateway
Run a language model on your own machine and use it from anywhere. One secure HTTPS endpoint with an API key, responses that stream token by token, and no public IP or open port on your side.
Works with Ollama, LM Studio, vLLM and llama.cpp, and with any app or tool that speaks the OpenAI API.
Why a Private LLM Gateway
A model running on your own GPU or Mac is private and has no per-token bill, but it only answers on your local network. Exposing it safely means a public IP, port forwarding, TLS certificates and an authentication layer. The gateway does all of that for you.
Your laptop on the road, your phone, a second office: call the model at home or in the server room through one HTTPS address, as if it were a cloud API. Nothing to reconfigure when your connection or IP changes.
Ship AI features backed by your own hardware. The endpoint speaks the OpenAI API, so the official SDKs work by changing the base URL. Responses stream token by token, errors come back in the format SDKs expect, and there is no per-token fee from us.
Prompts and answers are processed by the model on hardware you control, not by a third-party AI provider. Give teams and internal tools one governed endpoint with a revocable key, instead of copies of a model on every laptop.
A small connector next to your model dials out to CDNShark. Nothing connects in, so the machine never appears on the internet and there is no port for anyone to scan.
Paste one URL into Cline, Open WebUI, LangChain or any OpenAI-compatible client. Ollama-native and OpenAI-style tools both work, with or without /v1 on the end.
Every request needs your API key, and our key is removed before the request reaches your server. Issue a new key and the old one stops working within seconds; suspend the gateway with one click.
How It Works
Create a gateway
Pick your model server (Ollama, LM Studio, vLLM, llama.cpp). You get an API key and a one-line install command.
Install the connector
Run it on the machine next to your model. It dials out to us; nothing dials in.
Get your endpoint
Your model is live at https://<id>.llm.cdnshark.com, with free TLS.
Call it from anywhere
Use the URL and key in your app, SDK or tool. Requests reach the nearest edge and stream back.
One URL, Every Tool
curl -N https://YOUR-ID.llm.cdnshark.com/v1/chat/completions \
-H "Authorization: Bearer $LLM_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"llama3.2","stream":true,
"messages":[{"role":"user","content":"Hello!"}]}'import OpenAI from "openai";
const ai = new OpenAI({
baseURL: "https://YOUR-ID.llm.cdnshark.com",
apiKey: process.env.LLM_KEY,
});
const stream = await ai.chat.completions.create({
model: "llama3.2", stream: true,
messages: [{ role: "user", content: "Hello!" }],
});Also works as the server URL in Cline (VS Code), Open WebUI and other Ollama-compatible tools.
A Good Fit For
Use the model on your workstation from your laptop anywhere, including AI coding assistants like Cline.
Back a web or mobile product with your own model through your backend, with a key you can rotate at any time.
One shared, private model endpoint for chatbots, document tools and automations, without sending company data to an AI vendor.
Try open-weights models against real apps and demos before deciding where to run them in production.
It's your model on your hardware. We don't host models, and we don't store your prompts or responses: requests are encrypted in transit and passed straight through. Speed depends on your machine and its upload bandwidth, so a small home connection or an older GPU will feel like it.
Each gateway publishes one model server; create another gateway for a second one. The connector needs outbound UDP, which most networks allow (a TCP fallback for strict firewalls is in development). Don't put the API key inside a mobile app, where it can be extracted: call the gateway from your own backend.
Simple Pricing
LLM gateways are included with selected plans, and extra gateways can be added from Resource Add-ons. Traffic counts as ordinary CDN bandwidth on your plan. No per-token or per-request fees.
Your model, reachable from anywhere. Only by you.
Create a gateway and connect your model in a few minutes.
Get Started Publishing a website instead?