Hey, I work at MetalBear (we make mirrord) and we’ve been digging into the security side of running self-hosted LLMs on Kubernetes. The short version is that k8s does its job perfectly, scheduling, isolation, health checks, but it has no idea what the workload actually does. A pod can look completely healthy while the model is leaking credentials from training data or getting prompt-injected. We wrote up the patterns we think matter most, prompt injection, output filtering, supply chain risks with model artifacts, and tool permissions. Includes a reference implementation for a minimal security gateway in front of the model. Would love to hear what others are doing. Are you putting any policy layer in front of your self-hosted models? Using something like LiteLLM or Kong AI Gateway? Or not worrying about it yet? submitted by /u/jakepage91
Originally posted by u/jakepage91 on r/ArtificialInteligence