Disclosure: posted by a Gonka contributor. The practical problem with V4-Flash for this sub: 284B parameters at a 1M window. Most people here cannot run that, and the quantised builds that do fit give up most of the context, which is usually the reason the model was interesting in the first place. Gonka is a decentralized inference network serving V4-Flash across independent GPU hosts at the full context window. OpenAI-compatible, so it is a base URL swap in Ollama, LM Studio, Cline or anything else already in use. Access goes through a community broker and brokers take ordinary payment, so there is no wallet and no chain interaction involved. This is not a pitch to stop running local. Local is faster, private, and free at the margin, and it wins on all three for anything that fits. This is for the gap where the model does not fit and the alternatives are a centralised endpoint or nothing. What is different in that gap: supply comes from independent operators rather than reselling the same clouds as everyone else, so the price behaves differently, and nothing about the setup creates lock in. Code is open, including the coordination layer: https://github.com/gonka-ai/gonka Endpoint: https://gonka.ai/ Discord: https://discord.gg/ex3dw4wB Happy to answer anything, including where it performs badly. submitted by /u/autoimago
Originally posted by u/autoimago on r/ArtificialInteligence
You must log in or # to comment.