Affiliation disclosure: I built DumbQuestion.ai , a sarcastic Q&A side project. I’m posting this because the engineering trade-offs ended up being as interesting than the product itself if not more interesting. What started as “make an AI roast dumb questions” (this gen’s LMGTFY) turned into a constraint-driven software architecture exercise. I wanted a very low fixed-cost stack ('cause I am not likely making money on this), a fast main app (keep cloud costs ultra low), and as little unnecessary orchestration as possible, so the baseline ended up being Go, HTMX, Cloud Run, Cloudflare (security / cache / etc.), and low-cost model routing rather than a heavier agentic setup. The four engineering challenges: Persona consistency was as much an eval problem as a prompt text problem. I burned through a range of cheap and free models across persona scenarios (the app has four distinct personas) and found (no shock here) that the absolute cheapest options were often too inconsistent, too slow, or too unreliable once I pushed them harder. The better answer was “cheap but stable,” plus fallback behavior, not “lowest token price wins.” Interestingly, though, even a 12B param model was pretty effective in adopting satiric personas. Gemma 3 12B was used for a while before providers started dropping it, I am now experimenting with DeepSeek v4 Flash. My goal is to keep an effective model that costs around $20 per million questions asked. Self-awareness detection worked better with cheap embeddings than with regex or heavier classifiers. I needed to catch questions like “who made you?” and “are you real?” without adding more weight (cost) to the main LLM call. Regex was too brittle, and classic ML classification added app bloat I didn’t want, so I pre-vectorized examples and used semantic matching instead. I iterated through tons of approaches and models weighing the final cost of container size + compute against LLM / embeddings costs. The goal here was to support a hidden (dark) narrative effectively but mega-cheaply. Freshness-sensitive questions were routed with lightweight intent detection instead of full tool-using agents. If a prompt clearly looked like it needed current information (triggering a web search), I injected the current date/time and search results. If it did not, I kept it as a normal response path. I didn’t want to run an agentic loop (web search tool) but I am willing to prefill search results into the context. Detecting the intent to search itself was a similar challenge to the self-awareness detection but I leaned a little more into stock lexical analysis rather than a semantic analysis for cost reasons. Again, iterative testing helped fine tune this approach. Anything non-essential got pushed off the critical path. A semantic leaderboard (of the most asked questions), telemetry processing, and analytics all run asynchronously rather than inside the request cycle. Questions emit structured logs, a Vector sidecar ships them through Pub/Sub, and downstream consumers handle analytics and semantic grouping separately, which kept the app lean and failure-tolerant. This allows me to host the analytics “stack” anywhere (locally, today) so that I am not burning excess cloud costs for non-value added workloads. One side effect of the product design was that some edge cases had to double as behavior design. There is a small hidden, dark narrative layer in the app, so self-awareness and jailbreak handling (prompt injection) were not just backend safeguards; they also had to produce responses that felt intentional instead of broken. The biggest takeaway for me was that AI made writing code cheaper, but years of engineering “situational awareness” moved it much further from slop (in my personal opinion, at least). The hard parts were still deciding where to spend latency, where to spend money, what belonged in-process, and what should be kicked out into async pipelines. Site: DumbQuestion.ai submitted by /u/jagostoni
Originally posted by u/jagostoni on r/ArtificialInteligence