command line (run on 2004 Oct 29 at 14:17:46): ../src/md-eval-v19a.pl -af -e -D -d -W -w -t 1.0 -l 2 -u md_test10.uem -r md_test10.ref.rttm -s md_test10.sys.rttm Word-based metadata alignment, max gap between matching words = 1.0 sec Metadata evaluation parameters: word-optimized metadata mapping max gap between matching metadata events = 0.1 words max extent to match for SU's = 2 words Speaker Diarization evaluation parameters: The max time to extend no-score zones for NON-LEX exclusions is 0.5 sec The no-score collar at SPEAKER boundaries is 0 sec Exclusion zones for evaluation and scoring are: -----MetaData----- -----SpkrData----- exclusion set name: DEFAULT DEFAULT DEFAULT DEFAULT token type/subtype no-eval no-score no-eval no-score (UEM) X X LEXEME/un-lex X NON-LEX/breath X NON-LEX/cough X NON-LEX/laugh X NON-LEX/lipsmack X NON-LEX/other X NON-LEX/sneeze X NOSCORE/ X X X X NO_RT_METADATA/ X SU/unannotated X Word alignment and scoring details for channel 1 of file md_test10 ref del ins sub REF: token type tbeg tend speaker SYS: token type Rtbeg Rtend tbeg tend sys-speaker 1 - - 0 firstword lex 10.00 11.95 spkr1r firstword lex ( 10.00 11.95) 10.00 10.95 spkr1s 1 - - 0 i lex 11.95 13.90 spkr1r i lex ( 11.95 13.90) 10.95 11.90 spkr1s 1 - - 0 secondword fp 14.00 14.95 spkr1r secondword fp ( 14.00 14.95) 12.00 13.45 spkr1s 1 - - 0 uh fp 14.95 15.90 spkr1r uh fp ( 14.95 15.90) 13.45 14.90 spkr1s 1 - - 0 thirdword lex 16.00 17.95 spkr1r thirdword lex ( 16.00 17.95) 15.00 16.95 spkr1s 1 - - 0 will lex 17.95 19.90 spkr1r will lex ( 17.95 19.90) 16.95 18.90 spkr1s 1 - - 0 fourthword lex 20.00 20.95 spkr1r fourthword lex ( 20.00 20.95) 19.00 19.95 spkr1s 1 - - 0 drive lex 20.95 21.90 spkr1r drive lex ( 20.95 21.90) 19.95 20.90 spkr1s 1 - - 0 fifthword lex 22.00 23.95 spkr1r fifthword lex ( 22.00 23.95) 21.00 23.45 spkr1s 1 - - 0 fly lex 23.95 25.90 spkr1r fly lex ( 23.95 25.90) 23.45 25.90 spkr1s EDIT alignment and scoring details for channel 1 of file md_test10 ref del ins sub REF: token type tbeg tend speaker SYS: token type Rtbeg Rtend tbeg tend sys-speaker 1 - - 0 revision EDIT 20.00 21.90 spkr1r revision EDIT ( 20.00 21.90) 19.00 20.90 spkr1s FILLER alignment and scoring details for channel 1 of file md_test10 ref del ins sub REF: token type tbeg tend speaker SYS: token type Rtbeg Rtend tbeg tend sys-speaker 1 - - 0 filled_pause FILLER 14.00 16.00 spkr1r filled_pause FILLER ( 14.00 16.00) 12.00 15.00 spkr1s IP alignment and scoring details for channel 1 of file md_test10 ref del ins sub REF: token type tbeg tend speaker SYS: token type Rtbeg Rtend tbeg tend sys-speaker 1 - - 0 filler IP 14.00 14.00 spkr1r filler IP ( 14.00 14.00) 12.00 12.00 spkr1s 1 - - 0 edit IP 21.90 21.90 spkr1r edit IP ( 21.90 21.90) 20.90 20.90 spkr1s Chronological display of sys data aligned with ref data for file 'md_test10', channel '1' ----------------------- reference ----------------------- | mapped | --------------------- system output --------------------- --type-- -subtyp- -----word/spkr----- -tbeg- -tend- | ref_ID | --type-- -subtyp- -----word/spkr----- -tbeg- -tend- beg SPEAKER adult_fe spkr1r 10.00 | | 10.00 | |beg SPEAKER adult_fe spkr1s=>spkr1r 10.00 LEXEME lex FIRSTWORD 10.00 11.95 | LX1 | LEXEME lex FIRSTWORD 10.00 10.95 LEXEME lex I 11.95 13.90 | LX2 | LEXEME lex I 10.95 11.90 beg FILLER filled_p spkr1r 14.00 | FL1 |beg FILLER filled_p spkr1s=>spkr1r 12.00 IP filler spkr1r 14.00 | IP1 | IP filler spkr1s=>spkr1r 12.00 LEXEME fp SECONDWORD 14.00 14.95 | LX3 | LEXEME fp SECONDWORD 12.00 13.45 LEXEME fp UH 14.95 15.90 | LX4 | LEXEME fp UH 13.45 14.90 end FILLER filled_p spkr1r 16.00 | FL1 |end FILLER filled_p spkr1s=>spkr1r 15.00 LEXEME lex THIRDWORD 16.00 17.95 | LX5 | LEXEME lex THIRDWORD 15.00 16.95 LEXEME lex WILL 17.95 19.90 | LX6 | LEXEME lex WILL 16.95 18.90 beg EDIT revision spkr1r 20.00 | ED1 |beg EDIT revision spkr1s=>spkr1r 19.00 LEXEME lex FOURTHWORD 20.00 20.95 | LX7 | LEXEME lex FOURTHWORD 19.00 19.95 LEXEME lex DRIVE 20.95 21.90 | LX8 | LEXEME lex DRIVE 19.95 20.90 end EDIT revision spkr1r 21.90 | ED1 |end EDIT revision spkr1s=>spkr1r 20.90 IP edit spkr1r 21.90 | IP2 | IP edit spkr1s=>spkr1r 20.90 LEXEME lex FIFTHWORD 22.00 23.95 | LX9 | LEXEME lex FIFTHWORD 21.00 23.45 LEXEME lex FLY 23.95 25.90 | LX10 | LEXEME lex FLY 23.45 25.90 end SPEAKER adult_fe spkr1r 28.00 | | 28.00 | |end SPEAKER adult_fe spkr1s=>spkr1r 28.00 *** Performance analysis for EDITs *** overall error SCORE = 0.00% EDIT word coverage statistics -- in terms of reference words Nref Ndel Nins Nsub %Del %Ins %Sub %D+I %Tot ALL 2 0 0 0 0.00 0.00 0.00 0.00 0.00 EDIT detection statistics -- in terms of # of EDITs Nref Ndel Nins Nsub %Del %Ins %Sub %D+I %Tot ALL 1 0 0 0 0.00 0.00 0.00 0.00 0.00 f=md_test10 1 0 0 0 0.00 0.00 0.00 0.00 0.00 EDIT detection confusion matrix -- in terms of # of EDITs ALL - ref\sys revision {Miss} revision 1 0 {FA} 0 EDIT word offset statistics for ALL data word offsets: <-3 -3 -2 -1 0 1 2 3 >3 BEG: 0 - - - 1 - - - 0 END: 0 - - - 1 - - - 0 *** Performance analysis for FILLERs *** overall error SCORE = 0.00% FILLER word coverage statistics -- in terms of reference words Nref Ndel Nins Nsub %Del %Ins %Sub %D+I %Tot ALL 2 0 0 0 0.00 0.00 0.00 0.00 0.00 FILLER detection statistics -- in terms of # of FILLERs Nref Ndel Nins Nsub %Del %Ins %Sub %D+I %Tot ALL 1 0 0 0 0.00 0.00 0.00 0.00 0.00 f=md_test10 1 0 0 0 0.00 0.00 0.00 0.00 0.00 FILLER detection confusion matrix -- in terms of # of FILLERs ALL - ref\sys filled_p {Miss} filled_pause 1 0 {FA} 0 FILLER word offset statistics for ALL data word offsets: <-3 -3 -2 -1 0 1 2 3 >3 BEG: 0 - - - 1 - - - 0 END: 0 - - - 1 - - - 0 *** Performance analysis for IPs *** overall error SCORE = 0.00% IP (exact) detection statistics -- in terms of reference words Nref Ndel Nins Nsub %Del %Ins %Sub %D+I %Tot ALL 2 0 0 0 0.00 0.00 0.00 0.00 0.00 IP detection statistics -- in terms of # of IPs Nref Ndel Nins Nsub %Del %Ins %Sub %D+I %Tot ALL 2 0 0 0 0.00 0.00 0.00 0.00 0.00 f=md_test10 2 0 0 0 0.00 0.00 0.00 0.00 0.00 IP detection confusion matrix -- in terms of # of IPs ALL - ref\sys edit filler {Miss} edit 1 0 0 filler 0 1 0 {FA} 0 0 IP word offset statistics for ALL data word offsets: <-3 -3 -2 -1 0 1 2 3 >3 BEG: 0 - - - 2 - - - 0 END: 0 - - - 2 - - - 0 *** Performance analysis for Speaker Diarization for f=md_test10 *** EVAL TIME = 16.00 secs EVAL SPEECH = 16.00 secs (100.0 percent of evaluated time) SCORED TIME = 16.00 secs (100.0 percent of evaluated time) SCORED SPEECH = 16.00 secs (100.0 percent of scored time) EVAL WORDS = 10 SCORED WORDS = 10 (100.0 percent of evaluated words) --------------------------------------------- MISSED SPEECH = 0.00 secs ( 0.0 percent of scored time) FALARM SPEECH = 0.00 secs ( 0.0 percent of scored time) MISSED WORDS = 0 ( 0.0 percent of scored words) --------------------------------------------- SCORED SPEAKER TIME = 16.00 secs (100.0 percent of scored speech) MISSED SPEAKER TIME = 0.00 secs ( 0.0 percent of scored speaker time) FALARM SPEAKER TIME = 0.00 secs ( 0.0 percent of scored speaker time) SPEAKER ERROR TIME = 0.00 secs ( 0.0 percent of scored speaker time) SPEAKER ERROR WORDS = 0 ( 0.0 percent of scored speaker words) --------------------------------------------- OVERALL SPEAKER DIARIZATION ERROR = 0.00 percent of scored speaker time `(f=md_test10) --------------------------------------------- Speaker type confusion matrix -- speaker weighted REF\SYS (count) adult_female MISS adult_female 1 / 100.0% 0 / 0.0% FALSE ALARM 0 / 0.0% --------------------------------------------- Speaker type confusion matrix -- time weighted REF\SYS (seconds) adult_female MISS adult_female 16.00 / 100.0% 0.00 / 0.0% FALSE ALARM 0.00 / 0.0% --------------------------------------------- *** Performance analysis for Speaker Diarization for ALL *** EVAL TIME = 16.00 secs EVAL SPEECH = 16.00 secs (100.0 percent of evaluated time) SCORED TIME = 16.00 secs (100.0 percent of evaluated time) SCORED SPEECH = 16.00 secs (100.0 percent of scored time) EVAL WORDS = 10 SCORED WORDS = 10 (100.0 percent of evaluated words) --------------------------------------------- MISSED SPEECH = 0.00 secs ( 0.0 percent of scored time) FALARM SPEECH = 0.00 secs ( 0.0 percent of scored time) MISSED WORDS = 0 ( 0.0 percent of scored words) --------------------------------------------- SCORED SPEAKER TIME = 16.00 secs (100.0 percent of scored speech) MISSED SPEAKER TIME = 0.00 secs ( 0.0 percent of scored speaker time) FALARM SPEAKER TIME = 0.00 secs ( 0.0 percent of scored speaker time) SPEAKER ERROR TIME = 0.00 secs ( 0.0 percent of scored speaker time) SPEAKER ERROR WORDS = 0 ( 0.0 percent of scored speaker words) --------------------------------------------- OVERALL SPEAKER DIARIZATION ERROR = 0.00 percent of scored speaker time `(ALL) --------------------------------------------- Speaker type confusion matrix -- speaker weighted REF\SYS (count) adult_female MISS adult_female 1 / 100.0% 0 / 0.0% FALSE ALARM 0 / 0.0% --------------------------------------------- Speaker type confusion matrix -- time weighted REF\SYS (seconds) adult_female MISS adult_female 16.00 / 100.0% 0.00 / 0.0% FALSE ALARM 0.00 / 0.0% ---------------------------------------------