Know what it got wrong, and what it cost
An assistant nobody measures slowly gets worse as your documents change. SupaCortex shows you the questions it could not answer, the answers people disliked, and every credit it spent getting there.
- Knowledge-gap reports
- Thumbs on every answer
- Credits by operation and day
How does SupaCortex measure answer quality?
Every answer SupaCortex gives - in the widget, team chat, the API or the playground - is recorded with its status, the passages it used, whether it passed the citation checks, how long it took and the credits it cost. People rate answers with a thumbs up or down. The Insights view turns this into an answer rate, the most-asked unanswered questions and the latest thumbs-down answers with their sources, per project or across the workspace. Reply drafts add acceptance, edit and reopen rates from the help desk.
- Answer rate, unanswered questions and the reason each went unanswered.
- Recent thumbs-down answers shown with the question, the answer and the citations.
- Widget conversation logs per project, with ratings and system calls.
- Reply-draft acceptance, edit and reopen rates by category.
- AI credits by operation, by day, and as a full ledger.
- Alerts at 80% and 100% of the month's credits.
Where to look
Insights
Answered against unanswered over any period up to 90 days, a daily chart, the most-asked unanswered questions, and why each one failed.
Thumbs-down review
The latest disliked answers with the question, the answer, the note if one was left, and the sources it cited - so you can see whether the document or the answer was wrong.
Conversations
Every widget session per project: who it was (public or signed in), the transcript, citations, ratings and calls such as "looked up Order status (200, 120 ms)".
Draft metrics
For reply drafts: how many were sent unedited, edited, discarded or reopened - by category, over 7, 30 or 90 days.
Mining summary
For Q&A mined from past tickets: pending, approved and rejected counts, the approval rate, and what each run skipped and why.
AI credits
Balance split into plan and pack credits, the last 30 days by day, usage by operation, the price list and every movement in a ledger.
A fixed price per operation, not a token bill
Each plan includes a monthly allowance. You can see what anything will cost before you switch it on.
- Credits are taken before the AI call and returned if it fails.
- Plan credits are used first; bought packs never expire.
- At zero, answering pauses politely and indexing waits in the queue - nothing is lost.
| Operation | Credits |
|---|---|
| An answer (widget, team chat, API, playground) | 1 |
| A chat turn that looks something up or acts in your systems | 2 |
| A reply draft for a help desk ticket, including a regenerate | 3 |
| One past conversation analysed by ticket mining | 2 |
| Indexing: every 25 chunks of uploaded or synced content | 1 |
| Search (/v1/search and the playground) | 0 |
Built to be checked, not trusted
Every answer leaves a record
When someone asks "why did it say that?", the answer should be on file - not reconstructed from memory.
- The passages used, their rank, score and trust tier, and the citation each became.
- The answer status, whether it was grounded, the model, the latency and the credits charged.
- Failures too: outages and out-of-credit refusals are recorded, not hidden.
- Questions are stored redacted, so the gap list is useful without holding personal data.
A loop that closes
Measuring is only useful if it changes the documents. Each report points at something you can fix.
- An unanswered question with no evidence: write or upload the missing page.
- Evidence found but the answer declined: the page probably needs a clearer section.
- A thumbs-down with a citation: open the cited passage and check whether it is out of date.
- Feeds bring the fix in on their next run; uploads are indexed as soon as they land.
Our guide to measuring answer quality covers golden question sets and the numbers worth watching.
Related
Questions people ask before they start
Find out which ten questions your docs cannot answer yet.
Book a 30-minute demo on your own documents. We set up a 14-day trial workspace for you - no credit card, no setup fee.
Questions first? Email info@greatdevelopers.com or call +91 8585953085.
ChatGPT
Claude
Perplexity
Grok