Just a Student trying to build some projects and this one caught my attention. An automated system for detecting statistically significant behavioural drift in deployed LLM/API services by repeatedly evaluating fixed probe prompts and comparing current responses against historical behavioural baselines. The over all approach of the project is that the Raw responses is Scored based on Semantic and Behavioural Features and then Drift Detection Score is done by comparing to previous model answers, Predefined tests, Change Point and Anomaly detection If the Score is unusual than the historic record then we can investigate the Model and know the exact time and anomaly that caused the changes. Is this approach Worth Moving forward with and Should I include anything else. any suggestion is welcomed. submitted by /u/Particular-Roof4257
Originally posted by u/Particular-Roof4257 on r/ArtificialInteligence