Where I work, the folks are either creating internal API test generation tools or trying to buy one. But I feel it is all madness because no one other than a person who knows the entire architecture and design can find the best edge cases. I was trying to find something to evaluate the tool that these folks are claiming to be the best in finding bugs. Came across this, seems helpful. If you are on the same boat, you can evaluate using this dataset on huggingface: https://huggingface.co/datasets/kusho-ai/api-eval-20 From what I understand, it’s designed to evaluate whether an agent can really find bugs in APIs given just a schema and sample payload which seems to be closer to how these tools claim to work. submitted by /u/zoismom
Originally posted by u/zoismom on r/ArtificialInteligence