It's not impossible. Websites will ask for an iris scan to identify if you are a human as a means of auth. They will be provided by Apple/Google and governed by local law. Those will be integrated in your phone. There will be a global database of all human iris to fight ai abuse since ai can't fake the creation of a baby. Passkeys and email/passwords will be a thing of the past soon.
Personally, I absolutely use it locally. I’m always trying different editors and tech. It saves me from entering a multitude of API keys into each different software, in addition to the other reasons supplied for being able to specify your own limits and avoid surprise charges if you want.
When I want to try a new editor, vs code plugin or software, I only have to point it at my litellm proxy and immediately have access to all of my providers and models I’ve configured, no extra setup. It’s like a locally hosted openrouter that doesn’t charge you for routing. I can just select a different provider as easy as choosing the model in the software; switching from “openai/gpt-4o” to “groq/moonshotai/kimi-k2-instruct”, for example.
You can use litellm or OpenAI protocols which makes it compatible with most software. Add on ollama proxy and you can proxy ollama requests from software that doesn’t support specifying OpenAI’s base address but that does support ollama (a not uncommon situation). That combo covers most software.
So yes, to me it is absolutely worth running locally and as easy as editing a config file and starting a docker (or a shell script to open a venv and start litellm, if you prefer).
The only drawbacks I’ve found so far is that not all providers accurately respond with their model information so you sometimes have to configure models/pricing/limits manually in the config (about 5 lines of text that be copy/pasted and edited). All the SOTA models are pre-configured and kept relatively up to date, but one can expect updates to lag behind real pricing changes.
The UI is necessary if you want to set up api key/billing restrictions which requires a db, but that is rather trivial with docker as well.
Its a server-side proxy, so instead of ie. OpenAI url you point your AI tool to url of LiteLLM proxy, and use its virtual keys with budget limits or LLM models restrictions etc... - the features LLM providers will not give you, because it might save you money ;)
I mean that's the kind of website I would have been done by Flash and I would played around for ages on. Now it's just stupid scroll jacking and poor HTML.
What a bunch of crap. Everyone who lived these era remembers the horrors of optimizing for a bunch of different browsers. Ajax was barely understood. PHP was all over the place. No frameworks. No Stack overflow. No Vibe Coding. Nah. Nowadays i just have to prompt Claude "Hey, write me a html site for html hobbyists and upload it somewhere on the internet". voila!
It wasn't so much "horror" as "this designer wants rounded corners". AJAX was well-understood but not yet standardized or widely supported, which is down to slow standardization bodies and browser developers like Microsoft and nowadays Google doing their own thing.
Anyway, show us your HTML site, document how you built it - that's the kind of thing this page is advocating for.
