I kept hitting the same problems with LLMs in production:
- OpenAI goes down → my app breaks
- I’m using expensive models for simple tasks
- No visibility into what I’m spending
- PII leaking to external APIs So I built Sentinel - an open-source gateway that handles all of this. What it does:
- Automatic failover (OpenAI down? Switch to Anthropic)
- Cost tracking (see exactly what you’re spending)
- PII redaction (strip sensitive data before it leaves your network)
- Smart caching (save money on repeated queries)
- OpenAI-compatible API (just change your base URL) Tech:
- Built in Rust for performance
- Sub-millisecond overhead
- 9 LLM providers supported
- SQLite for logging, DashMap for caching GitHub: https://github.com/fbk2111/Sentinel I’m looking for:
- Feedback on the architecture
- Bug reports (if you try it)
- Ideas for what’s missing Built this for myself, but figured others might have the same pain points. submitted by /u/SchemeVivid4175
Originally posted by u/SchemeVivid4175 on r/ArtificialInteligence
You must log in or # to comment.