Something I’ve been thinking about lately is how AI teams decide when to use different models in production. When you’re building with LLMs, it’s easy to focus on getting the best output from a model during development. But once real users arrive, the trade-offs get more complicated: A more capable model might be unnecessary for straightforward requests. A cheaper model might not perform as well on your specific use case. Switching providers means dealing with different APIs, rate limits, errors, and failure modes. Having a fallback is useful, but it doesn’t guarantee the alternative will deliver comparable results. I work in the LLM infrastructure space, so I’m particularly interested in how teams handle this in practice. For those running AI features in production: Do you stick with one primary model, or use multiple models? How do you decide which model to use for a given request? What’s been hardest to manage: cost, output quality, latency, or reliability? I’d especially love to hear from people who’ve moved beyond prototypes and are dealing with real production traffic. What has actually worked for you? submitted by /u/SkillKiller3010
Originally posted by u/SkillKiller3010 on r/ArtificialInteligence
You must log in or # to comment.