command line (run on 2004 Oct 29 at 14:13:46): ../src/md-eval-v19a.pl -1 -v -e -d -D -m -af -c 0.0 -T 0.0 -u sd_test6.uem -r sd_test6.ref.rttm -s sd_test6.sys.rttm Time-based metadata alignment Metadata evaluation parameters: time-optimized metadata mapping max gap between matching metadata events = 0.0 sec max extent to match for SU's = 0.5 sec Speaker Diarization evaluation parameters: The max time to extend no-score zones for NON-LEX exclusions is 0.5 sec The no-score collar at SPEAKER boundaries is 0.0 sec Exclusion zones for evaluation and scoring are: -----MetaData----- -----SpkrData----- exclusion set name: DEFAULT DEFAULT DEFAULT DEFAULT token type/subtype no-eval no-score no-eval no-score (UEM) X X LEXEME/un-lex X NON-LEX/breath X NON-LEX/cough X NON-LEX/laugh X NON-LEX/lipsmack X NON-LEX/other X NON-LEX/sneeze X NOSCORE/ X X X X NO_RT_METADATA/ X SU/unannotated X Word alignment and scoring details for channel 1 of file sd_test6 ref del ins sub REF: token type tbeg tend speaker SYS: token type tbeg tend speaker 1 - - 0 firstword lex 10.00 11.00 spkr1r firstword lex 10.00 11.00 spkr1s 1 - - 0 t.'s lex 11.00 12.00 spkr1r t.'s lex 11.00 12.00 spkr1s 1 - - 0 thirdword lex 12.00 13.00 spkr1r thirdword lex 12.00 13.00 spkr1s 1 - - 0 fourthword lex 13.00 14.00 spkr1r fourthword lex 13.00 14.00 spkr1s 1 - - 0 fifthword lex 14.00 15.00 spkr1r fifthword lex 14.00 15.00 spkr1s 1 - - 0 sixthword lex 15.00 16.00 spkr1r sixthword lex 15.00 16.00 spkr1s 1 - - 0 seventhword lex 16.00 17.00 spkr2r seventhword lex 16.00 17.00 spkr2s 1 - - 0 eighthword lex 17.00 18.00 spkr2r eighthword lex 17.00 18.00 spkr2s 1 - - 0 ninthword lex 18.00 19.00 spkr2r ninthword lex 18.00 19.00 spkr2s 1 - - 0 tenthword lex 19.00 20.00 spkr3r tenthword lex 19.00 20.00 spkr3s 1 - - 0 eleventhword lex 20.00 21.00 spkr3r eleventhword lex 20.00 21.00 spkr3s SU alignment and scoring details for channel 1 of file sd_test6 ref del ins sub REF: token type tbeg tend speaker SYS: token type tbeg tend speaker 1 - - 0 statement SU 10.00 12.00 spkr1r statement SU 10.00 12.00 spkr1s 1 - - 0 statement SU 12.00 14.00 spkr1r statement SU 12.00 14.00 spkr1s 1 - - 0 question SU 14.00 16.00 spkr1r question SU 14.00 16.00 spkr1s 1 - - 0 backchannel SU 16.00 19.00 spkr2r backchannel SU 16.00 19.00 spkr2s 1 - - 0 incomplete SU 19.00 21.00 spkr3r incomplete SU 19.00 21.00 spkr3s 'spkr1r' => 'spkr1s' 6.00 secs matched to 'spkr1s' 'spkr2r' => 'spkr2s' 3.00 secs matched to 'spkr2s' 'spkr3r' => 'spkr3s' 2.00 secs matched to 'spkr3s' beg/dur/end = 10.000/ 2.000/ 12.000; REF = (spkr1r); SYS = (spkr1s) beg/dur/end = 12.000/ 2.000/ 14.000; REF = (spkr1r); SYS = (spkr1s) beg/dur/end = 14.000/ 2.000/ 16.000; REF = (spkr1r); SYS = (spkr1s) beg/dur/end = 16.000/ 3.000/ 19.000; REF = (spkr2r); SYS = (spkr2s) beg/dur/end = 19.000/ 2.000/ 21.000; REF = (spkr3r); SYS = (spkr3s) Chronological display of sys data aligned with ref data for file 'sd_test6', channel '1' ----------------------- reference ----------------------- | mapped | --------------------- system output --------------------- --type-- -subtyp- -----word/spkr----- -tbeg- -tend- | ref_ID | --type-- -subtyp- -----word/spkr----- -tbeg- -tend- beg SEGMENT spkr1r 10.00 | | beg SPEAKER child spkr1r 10.00 | | 10.00 | |beg SPEAKER child spkr1s=>spkr1r 10.00 beg SU statemen spkr1r 10.00 | SU1 |beg SU statemen spkr1s=>spkr1r 10.00 LEXEME lex FIRSTWORD 10.00 11.00 | LX1 | LEXEME lex FIRSTWORD 10.00 11.00 LEXEME alpha T.'S 11.00 12.00 | LX2 | LEXEME alpha T.'S 11.00 12.00 end SU statemen spkr1r 12.00 | SU1 |end SU statemen spkr1s=>spkr1r 12.00 12.00 | |beg SPEAKER child spkr1s=>spkr1r 12.00 beg SU statemen spkr1r 12.00 | SU2 |beg SU statemen spkr1s=>spkr1r 12.00 LEXEME acronym THIRDWORD 12.00 13.00 | LX3 | LEXEME acronym THIRDWORD 12.00 13.00 LEXEME interjec FOURTHWORD 13.00 14.00 | LX4 | LEXEME interjec FOURTHWORD 13.00 14.00 end SU statemen spkr1r 14.00 | SU2 |end SU statemen spkr1s=>spkr1r 14.00 14.00 | |end SPEAKER child spkr1s=>spkr1r 14.00 beg SU question spkr1r 14.00 | SU3 |beg SU question spkr1s=>spkr1r 14.00 LEXEME properno FIFTHWORD 14.00 15.00 | LX5 | LEXEME properno FIFTHWORD 14.00 15.00 LEXEME other SIXTHWORD 15.00 16.00 | LX6 | LEXEME other SIXTHWORD 15.00 16.00 end SU question spkr1r 16.00 | SU3 |end SU question spkr1s=>spkr1r 16.00 end SPEAKER child spkr1r 16.00 | | 16.00 | |end SPEAKER child spkr1s=>spkr1r 16.00 end SEGMENT spkr1r 16.00 | | beg SEGMENT spkr2r 16.00 | | beg SPEAKER unknown spkr2r 16.00 | | 16.00 | |beg SPEAKER unknown spkr2s=>spkr2r 16.00 beg SU backchan spkr2r 16.00 | SU4 |beg SU backchan spkr2s=>spkr2r 16.00 LEXEME lex SEVENTHWORD 16.00 17.00 | LX7 | LEXEME lex SEVENTHWORD 16.00 17.00 LEXEME lex EIGHTHWORD 17.00 18.00 | LX8 | LEXEME lex EIGHTHWORD 17.00 18.00 LEXEME lex NINTHWORD 18.00 19.00 | LX9 | LEXEME lex NINTHWORD 18.00 19.00 end SU backchan spkr2r 19.00 | SU4 |end SU backchan spkr2s=>spkr2r 19.00 end SPEAKER unknown spkr2r 19.00 | | 19.00 | |end SPEAKER unknown spkr2s=>spkr2r 19.00 end SEGMENT spkr2r 19.00 | | beg SEGMENT spkr3r 19.00 | | beg SPEAKER adult_fe spkr3r 19.00 | | 19.00 | |beg SPEAKER adult_fe spkr3s=>spkr3r 19.00 beg SU incomple spkr3r 19.00 | SU5 |beg SU incomple spkr3s=>spkr3r 19.00 LEXEME lex TENTHWORD 19.00 20.00 | LX10 | LEXEME lex TENTHWORD 19.00 20.00 LEXEME lex ELEVENTHWORD 20.00 21.00 | LX11 | LEXEME lex ELEVENTHWORD 20.00 21.00 end SU incomple spkr3r 21.00 | SU5 |end SU incomple spkr3s=>spkr3r 21.00 end SPEAKER adult_fe spkr3r 22.00 | | 22.00 | |end SPEAKER adult_fe spkr3s=>spkr3r 22.00 end SEGMENT spkr3r 22.00 | | *** Performance analysis for SUs *** overall error SCORE = 0.00% SU (exact) end detection statistics -- in terms of reference words Nref Ndel Nins Nsub %Del %Ins %Sub %D+I %Tot ALL 5 0 0 0 0.00 0.00 0.00 0.00 0.00 SU detection statistics -- in terms of # of SUs Nref Ndel Nins Nsub %Del %Ins %Sub %D+I %Tot ALL 5 0 0 0 0.00 0.00 0.00 0.00 0.00 f=sd_test6 5 0 0 0 0.00 0.00 0.00 0.00 0.00 SU detection confusion matrix -- in terms of # of SUs ALL - ref\sys backchan incomple question statemen {Miss} backchannel 1 0 0 0 0 incomplete 0 1 0 0 0 question 0 0 1 0 0 statement 0 0 0 2 0 {FA} 0 0 0 0 SU word offset statistics for ALL data word offsets: <-3 -3 -2 -1 0 1 2 3 >3 BEG: 0 - - - 5 - - - 0 END: 0 - - - 5 - - - 0 *** Performance analysis for Speaker Diarization for f=sd_test6 *** EVAL TIME = 11.00 secs EVAL SPEECH = 11.00 secs (100.0 percent of evaluated time) SCORED TIME = 11.00 secs (100.0 percent of evaluated time) SCORED SPEECH = 11.00 secs (100.0 percent of scored time) EVAL WORDS = 11 SCORED WORDS = 11 (100.0 percent of evaluated words) --------------------------------------------- MISSED SPEECH = 0.00 secs ( 0.0 percent of scored time) FALARM SPEECH = 0.00 secs ( 0.0 percent of scored time) MISSED WORDS = 0 ( 0.0 percent of scored words) --------------------------------------------- SCORED SPEAKER TIME = 11.00 secs (100.0 percent of scored speech) MISSED SPEAKER TIME = 0.00 secs ( 0.0 percent of scored speaker time) FALARM SPEAKER TIME = 0.00 secs ( 0.0 percent of scored speaker time) SPEAKER ERROR TIME = 0.00 secs ( 0.0 percent of scored speaker time) SPEAKER ERROR WORDS = 0 ( 0.0 percent of scored speaker words) --------------------------------------------- OVERALL SPEAKER DIARIZATION ERROR = 0.00 percent of scored speaker time `(f=sd_test6) --------------------------------------------- Speaker type confusion matrix -- speaker weighted REF\SYS (count) adult_female child unknown MISS adult_female 1 / 33.3% 0 / 0.0% 0 / 0.0% 0 / 0.0% child 0 / 0.0% 1 / 33.3% 0 / 0.0% 0 / 0.0% unknown 0 / 0.0% 0 / 0.0% 1 / 33.3% 0 / 0.0% FALSE ALARM 0 / 0.0% 0 / 0.0% 0 / 0.0% --------------------------------------------- Speaker type confusion matrix -- time weighted REF\SYS (seconds) adult_female child unknown MISS adult_female 2.00 / 18.2% 0.00 / 0.0% 0.00 / 0.0% 0.00 / 0.0% child 0.00 / 0.0% 6.00 / 54.5% 0.00 / 0.0% 0.00 / 0.0% unknown 0.00 / 0.0% 0.00 / 0.0% 3.00 / 27.3% 0.00 / 0.0% FALSE ALARM 0.00 / 0.0% 0.00 / 0.0% 0.00 / 0.0% --------------------------------------------- *** Performance analysis for Speaker Diarization for ALL *** EVAL TIME = 11.00 secs EVAL SPEECH = 11.00 secs (100.0 percent of evaluated time) SCORED TIME = 11.00 secs (100.0 percent of evaluated time) SCORED SPEECH = 11.00 secs (100.0 percent of scored time) EVAL WORDS = 11 SCORED WORDS = 11 (100.0 percent of evaluated words) --------------------------------------------- MISSED SPEECH = 0.00 secs ( 0.0 percent of scored time) FALARM SPEECH = 0.00 secs ( 0.0 percent of scored time) MISSED WORDS = 0 ( 0.0 percent of scored words) --------------------------------------------- SCORED SPEAKER TIME = 11.00 secs (100.0 percent of scored speech) MISSED SPEAKER TIME = 0.00 secs ( 0.0 percent of scored speaker time) FALARM SPEAKER TIME = 0.00 secs ( 0.0 percent of scored speaker time) SPEAKER ERROR TIME = 0.00 secs ( 0.0 percent of scored speaker time) SPEAKER ERROR WORDS = 0 ( 0.0 percent of scored speaker words) --------------------------------------------- OVERALL SPEAKER DIARIZATION ERROR = 0.00 percent of scored speaker time `(ALL) --------------------------------------------- Speaker type confusion matrix -- speaker weighted REF\SYS (count) adult_female child unknown MISS adult_female 1 / 33.3% 0 / 0.0% 0 / 0.0% 0 / 0.0% child 0 / 0.0% 1 / 33.3% 0 / 0.0% 0 / 0.0% unknown 0 / 0.0% 0 / 0.0% 1 / 33.3% 0 / 0.0% FALSE ALARM 0 / 0.0% 0 / 0.0% 0 / 0.0% --------------------------------------------- Speaker type confusion matrix -- time weighted REF\SYS (seconds) adult_female child unknown MISS adult_female 2.00 / 18.2% 0.00 / 0.0% 0.00 / 0.0% 0.00 / 0.0% child 0.00 / 0.0% 6.00 / 54.5% 0.00 / 0.0% 0.00 / 0.0% unknown 0.00 / 0.0% 0.00 / 0.0% 3.00 / 27.3% 0.00 / 0.0% FALSE ALARM 0.00 / 0.0% 0.00 / 0.0% 0.00 / 0.0% ---------------------------------------------