How to measure whether an AI assistant gets it right
A test with real questions before putting the assistant in front of anyone. The score did not change, and that was the most useful measurement of the project.
I taught my AI assistant to doubt its own search before answering. I measured it, and the score did not change at all.
That measurement was the most useful thing I did in the project.
The project
My portfolio has a chat that answers questions about my career. Before answering, it searches my résumé for the passages related to the question. This design has a name: RAG.
Each passage gets a score, which is how similar it is to the question. High score, probably the right passage.
What I changed
When the score lands in the middle, neither good nor bad, the assistant now asks itself: "does what I found really answer this?". If it does not, it rewrites the question in other words and searches again.
How I tested it
I picked 24 real questions and ran them twice, with this doubt turned on and off. To grade them, I used a second AI as the teacher, comparing each answer with my résumé.
19 right answers out of 24 both times. A tie.
What only showed up question by question
The doubt almost never kicked in. I had guessed the score at which it would start. In practice, it ran on 3 of the 24 questions, and on none of the 5 the chat got wrong. I looked at the real scores, adjusted, and it started running on 14.
One error went away, and stays away every time I run it. A question about my current experience was answered with a job from years ago. Now it gets the right answer.
And something worse than the original error showed up. When it could not find the answer in the résumé, the assistant went searching the internet and wrote, in the first person, about places where I never worked. I blocked that path. Today, when it does not know, it says it does not know.
What I learned
With 24 questions, the score proves almost nothing: one right answer more or less is luck. What a test this size shows is the way the system fails. That is exactly what showed up.
It is the same care that goes into every assistant Weft delivers: test with real questions before putting it in front of anyone, and again after every change.
Originally published in Portuguese on LinkedIn, where the conversation continues in the comments.
