置信度門控路由
把置信度當作第二條軸。答案告訴你該做什麼;置信度告訴你該不該做。
TypeSafe 最強大的特性之一,是置信度。只要你有意識地把決策門控在置信度上,就能構建出既可靠又安全的系統。
示例:語音銀行指令
設想你要做一個語音銀行介面,讓使用者能口頭操作自己的賬戶。解讀使用者意圖時,你當然希望始終有說得過去的置信度,但有些操作比另一些風險更高,因而需要更高的置信度閾值。
%%{init: {"fontFamily": "Inter, sans-serif", "flowchart": {"rankSpacing": 35, "wrappingWidth": 300, "subGraphTitleMargin": {"top": 12, "bottom": 36}}}}%%
flowchart LR
command["voice banking command"]
subgraph req["TypeSafe evaluates<br/>the question"]
intent["<b>Choice:</b> intent"]
end
command -- "one request<br/>command + intent<br/>question" --> req
req -- "one response<br/>intent answer +<br/>confidence" --> gate{"<b>confidence high enough?</b><br/>your code"}
gate -- "below 0.6<br/>or other intent" --> human["send to a support agent"]
gate -- "check_balance<br/>at least 0.6" --> balance["show the balance"]
gate -- "approve_transfer<br/>0.6 to 0.85" --> confirm["ask the user to confirm"]
gate -- "approve_transfer<br/>above 0.85" --> approve["approve the transfer"]
第 1 步:判斷使用者意圖
{
"intent": {
"type": "choice",
"instructions": "What action is the user requesting?",
"criteria": {
"check_balance": "Check the balance of an account",
"approve_transfer": "Approve the pending transfer request",
"other": "Something else"
}
}
}第 2 步:置信度門控路由
action = response.answers["intent"]
# Below 0.6 confidence on any action, route to a human
if action.confidence < 0.6:
route_to_support_agent(account_id)
elif action.choice == "check_balance":
# Low stakes. 0.6 confidence is sufficient.
show_balance(account_id)
elif action.choice == "approve_transfer":
if action.confidence > 0.85:
# High stakes, but high confidence. Safe to act automatically.
approve_transfer(account_id)
else:
# High stakes, moderate confidence. Verify intent first.
ask_user_to_confirm("Just to confirm: you would like to approve this transfer, is that correct?")
else:
route_to_support_agent(account_id)
0.6 這條底線,兜住的是模型確實拿不準的部分。底線之上,每種操作各自有閾值,取決於分類錯了還要照做會有什麼後果。以 0.6 的置信度去查餘額沒問題,因為最壞的情況不過是使用者聽一遍餘額播報。但批准一筆轉賬需要很高的置信度(>0.85),否則系統就該請使用者先確認一下。
想更詳細地瞭解在系統裡該如何看待置信度,見置信度。