Ive been trying to deploy LLMs and figured out I can create flags and based on that the inference is optimized for it. For eg you can run on flex mode that gives you api discounts and completes your response at off-peak time, or faster speeds, etc. This is for my upcoming project Astorias ai when we go out of beta. Would it be something you would use in your SaaS or integrate in your coding workflows ? Ill be honest here, Im doing market research for my YC application. Your feedbacks would help :) submitted by /u/me_broke
Originally posted by u/me_broke on r/ArtificialInteligence
You must log in or # to comment.