Why aren’t LLMs designed with built in context window trackers or word/token counting capabilities? Either function would help users to make more effective use of them. I am constantly having to remind my A.I. assistant of things discussed earlier, or using fact injection in my prompts to ensure accurate output from it. Having a context window tracker would be the best solution, seeing as the assistant frequently does web searches to collect facts before generating output, taking up huge amounts of the context window. A context window tracker could allow you to see where the facts impacting the output came from, or how much of the window was used by the a.i. researching and outputting as well as the human prompting to give the user an idea of how much of the window is left to utilize before the a.i. starts forgetting things and making things up as a result. You would also presumably be able to see what content is about to be forgotten by the a.i. if it’s towards the back end of the context window so you would know which facts need to be refreshed or summarized to avoid being forgotten. Adding a word/token counter would be less effective but could still achieve similar results. submitted by /u/Dangerous_Teaching82
Originally posted by u/Dangerous_Teaching82 on r/ArtificialInteligence
You must log in or # to comment.