{"id":2313,"date":"2022-05-06T14:29:15","date_gmt":"2022-05-06T17:29:15","guid":{"rendered":"https:\/\/icc.fcen.uba.ar\/?p=2313"},"modified":"2022-05-06T14:42:13","modified_gmt":"2022-05-06T17:42:13","slug":"out-of-a-hundred-trials-how-many-errors-does-your-speaker-verifier-make","status":"publish","type":"post","link":"https:\/\/icc.fcen.uba.ar\/en\/out-of-a-hundred-trials-how-many-errors-does-your-speaker-verifier-make\/","title":{"rendered":"Out of a hundred trials, how many errors does your speaker verifier make?"},"content":{"rendered":"<div class=\"fusion-fullwidth fullwidth-box fusion-builder-row-1 fusion-flex-container nonhundred-percent-fullwidth non-hundred-percent-height-scrolling\" style=\"--awb-border-radius-top-left:0px;--awb-border-radius-top-right:0px;--awb-border-radius-bottom-right:0px;--awb-border-radius-bottom-left:0px;--awb-flex-wrap:wrap;\" ><div class=\"fusion-builder-row fusion-row fusion-flex-align-items-flex-start fusion-flex-content-wrap\" style=\"max-width:1144px;margin-left: calc(-4% \/ 2 );margin-right: calc(-4% \/ 2 );\"><div class=\"fusion-layout-column fusion_builder_column fusion-builder-column-0 fusion_builder_column_1_1 1_1 fusion-flex-column\" style=\"--awb-bg-size:cover;--awb-width-large:100%;--awb-margin-top-large:0px;--awb-spacing-right-large:1.92%;--awb-margin-bottom-large:20px;--awb-spacing-left-large:1.92%;--awb-width-medium:100%;--awb-order-medium:0;--awb-spacing-right-medium:1.92%;--awb-spacing-left-medium:1.92%;--awb-width-small:100%;--awb-order-small:0;--awb-spacing-right-small:1.92%;--awb-spacing-left-small:1.92%;\"><div class=\"fusion-column-wrapper fusion-column-has-shadow fusion-flex-justify-content-flex-start fusion-content-layout-column\"><div class=\"fusion-text fusion-text-1\"><p>Authors: Niko Br\u00fcmmer, Luciana Ferrer, Albert Swart.<\/p>\n<p>Abstract: Out of a hundred trials, how many errors does your speaker verifier make? For the user this is an important, practical question, but researchers and vendors typically sidestep it and supply instead the conditional error-rates that are given by the ROC\/DET curve. We posit that the user&#8217;s question is answered by the Bayes error-rate. We present a tutorial to show how\u00a0to compute the error-rate that results when making Bayes decisions with calibrated likelihood ratios, supplied by\u00a0the verifier, and an hypothesis prior, supplied by the user. For perfect calibration, the Bayes error-rate is\u00a0upper bounded by min(EER,P,1-P), where EER is the equal-error-rate and P, 1-P are the prior probabilities\u00a0of the competing hypotheses. The EER represents the accuracy of the verifier, while min(P,1-P) represents\u00a0the hardness of the classification problem. We further show how the Bayes error-rate can be computed also for\u00a0non-perfect calibration and how to generalize from error-rate to expected cost. We offer some criticism of\u00a0decisions made by direct score thresholding. Finally, we demonstrate by analyzing error-rates of the recently\u00a0published DCA-PLDA speaker verifier.<\/p>\n<p>More information:\u00a0<a href=\"https:\/\/www.isca-speech.org\/archive\/pdfs\/interspeech_2021\/brummer21_interspeech.pdf\" target=\"_blank\" rel=\"noopener noreferrer\">https:\/\/www.isca-speech.org\/archive\/pdfs\/interspeech_2021\/brummer21_interspeech.pdf<\/a><\/p>\n<\/div><\/div><\/div><\/div><\/div>\n","protected":false},"excerpt":{"rendered":"","protected":false},"author":9,"featured_media":2314,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[98],"tags":[],"class_list":["post-2313","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-papers"],"_links":{"self":[{"href":"https:\/\/icc.fcen.uba.ar\/en\/wp-json\/wp\/v2\/posts\/2313","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/icc.fcen.uba.ar\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/icc.fcen.uba.ar\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/icc.fcen.uba.ar\/en\/wp-json\/wp\/v2\/users\/9"}],"replies":[{"embeddable":true,"href":"https:\/\/icc.fcen.uba.ar\/en\/wp-json\/wp\/v2\/comments?post=2313"}],"version-history":[{"count":2,"href":"https:\/\/icc.fcen.uba.ar\/en\/wp-json\/wp\/v2\/posts\/2313\/revisions"}],"predecessor-version":[{"id":2324,"href":"https:\/\/icc.fcen.uba.ar\/en\/wp-json\/wp\/v2\/posts\/2313\/revisions\/2324"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/icc.fcen.uba.ar\/en\/wp-json\/wp\/v2\/media\/2314"}],"wp:attachment":[{"href":"https:\/\/icc.fcen.uba.ar\/en\/wp-json\/wp\/v2\/media?parent=2313"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/icc.fcen.uba.ar\/en\/wp-json\/wp\/v2\/categories?post=2313"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/icc.fcen.uba.ar\/en\/wp-json\/wp\/v2\/tags?post=2313"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}