Blame view
tools/sctk-2.4.10/src/md-eval/test/md_test12.output.saved
19.4 KB
8dcb6dfcb
first commit
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 |
command line (run on 2004 Oct 29 at 14:17:47): ../src/md-eval-v19a.pl -af -e -D -d -W -w -t 1.0 -l 2 -u md_test12.uem -r md_test12.ref.rttm -s md_test12.sys.rttm
Word-based metadata alignment, max gap between matching words = 1.0 sec
Metadata evaluation parameters:
word-optimized metadata mapping
max gap between matching metadata events = 0.1 words
max extent to match for SU's = 2 words
Speaker Diarization evaluation parameters:
The max time to extend no-score zones for NON-LEX exclusions is 0.5 sec
The no-score collar at SPEAKER boundaries is 0 sec
Exclusion zones for evaluation and scoring are:
-----MetaData----- -----SpkrData-----
exclusion set name: DEFAULT DEFAULT DEFAULT DEFAULT
token type/subtype no-eval no-score no-eval no-score
(UEM) X X
LEXEME/un-lex X
NON-LEX/breath X
NON-LEX/cough X
NON-LEX/laugh X
NON-LEX/lipsmack X
NON-LEX/other X
NON-LEX/sneeze X
NOSCORE/<na> X X X X
NO_RT_METADATA/<na> X
SU/unannotated X
Word alignment and scoring details for channel 1 of file md_test12
ref del ins sub REF: token type tbeg tend speaker SYS: token type Rtbeg Rtend tbeg tend sys-speaker
1 - - 0 wordzero lex 9.00 10.00 spkr1r wordzero lex ( 9.00 10.00) 9.00 10.00 spkr1s
1 - - 0 firstword lex 10.00 11.00 spkr1r firstword lex ( 10.00 11.00) 10.00 11.00 spkr1s
1 - - 0 secondword lex 11.00 12.00 spkr1r secondword lex ( 11.00 12.00) 11.00 12.00 spkr1s
1 - - 0 thirdword lex 12.00 13.00 spkr1r thirdword lex ( 12.00 13.00) 12.00 13.00 spkr1s
1 - - 0 fourthword lex 13.00 14.00 spkr1r fourthword lex ( 13.00 14.00) 13.00 14.00 spkr1s
1 - - 0 fifthword lex 14.00 15.00 spkr1r fifthword lex ( 14.00 15.00) 14.00 15.00 spkr1s
1 - - 0 sixthword lex 15.00 16.00 spkr1r sixthword lex ( 15.00 16.00) 15.00 16.00 spkr1s
1 - - 0 seventhword lex 16.00 17.00 spkr1r seventhword lex ( 16.00 17.00) 16.00 17.00 spkr1s
1 - - 0 eighthword lex 17.00 18.00 spkr1r eighthword lex ( 17.00 18.00) 17.00 18.00 spkr1s
1 - - 0 ninthword lex 18.00 19.00 spkr1r ninthword lex ( 18.00 19.00) 18.00 19.00 spkr1s
1 - - 0 tenthword lex 19.00 20.00 spkr1r tenthword lex ( 19.00 20.00) 19.00 20.00 spkr1s
1 - - 0 eleventhword lex 20.00 21.00 spkr1r eleventhword lex ( 20.00 21.00) 20.00 21.00 spkr1s
1 - - 0 twelfthword lex 21.00 22.00 spkr1r twelfthword lex ( 21.00 22.00) 21.00 22.00 spkr1s
1 - - 0 thirteenthword lex 22.00 23.00 spkr1r thirteenthword lex ( 22.00 23.00) 22.00 23.00 spkr1s
1 - - 0 fourteenthword lex 23.00 24.00 spkr1r fourteenthword lex ( 23.00 24.00) 23.00 24.00 spkr1s
1 - - 0 fifteenthword lex 24.00 25.00 spkr1r fifteenthword lex ( 24.00 25.00) 24.00 25.00 spkr1s
1 - - 0 sixteenthword lex 25.00 26.00 spkr1r sixteenthword lex ( 25.00 26.00) 25.00 26.00 spkr1s
1 - - 0 seventeenthword lex 26.00 27.00 spkr1r seventeenthword lex ( 26.00 27.00) 26.00 27.00 spkr1s
1 - - 0 eighteenthword lex 27.00 28.00 spkr1r eighteenthword lex ( 27.00 28.00) 27.00 28.00 spkr1s
1 - - 0 nineteenthword lex 28.00 29.00 spkr1r nineteenthword lex ( 28.00 29.00) 28.00 29.00 spkr1s
1 - - 0 twentiethword lex 29.00 30.00 spkr1r twentiethword lex ( 29.00 30.00) 29.00 30.00 spkr1s
EDIT alignment and scoring details for channel 1 of file md_test12
ref del ins sub REF: token type tbeg tend speaker SYS: token type Rtbeg Rtend tbeg tend sys-speaker
1 - - 0 revision EDIT 12.00 15.00 spkr1r revision EDIT ( 11.00 14.00) 11.00 14.00 spkr1s
1 - - 0 revision EDIT 18.00 21.00 spkr1r revision EDIT ( 16.00 19.00) 16.00 19.00 spkr1s
0 - 1 - --- --- --- --- --- restart EDIT ( 21.00 22.00) 21.00 22.00 spkr1s
1 - - 0 revision EDIT 24.00 27.00 spkr1r revision EDIT ( 25.00 26.00) 25.00 26.00 spkr1s
IP alignment and scoring details for channel 1 of file md_test12
ref del ins sub REF: token type tbeg tend speaker SYS: token type Rtbeg Rtend tbeg tend sys-speaker
1 1 - - edit IP 15.00 15.00 spkr1r --- --- ( --- --- ) --- --- ---
1 1 - - edit IP 21.00 21.00 spkr1r --- --- ( --- --- ) --- --- ---
1 1 - - edit IP 27.00 27.00 spkr1r --- --- ( --- --- ) --- --- ---
0 - 1 - --- --- --- --- --- edit IP ( 14.00 14.00) 14.00 14.00 spkr1s
0 - 1 - --- --- --- --- --- edit IP ( 19.00 19.00) 19.00 19.00 spkr1s
0 - 1 - --- --- --- --- --- edit IP ( 22.00 22.00) 22.00 22.00 spkr1s
0 - 1 - --- --- --- --- --- edit IP ( 26.00 26.00) 26.00 26.00 spkr1s
SU alignment and scoring details for channel 1 of file md_test12
ref del ins sub REF: token type tbeg tend speaker SYS: token type Rtbeg Rtend tbeg tend sys-speaker
1 - - 0 statement SU 9.00 30.00 spkr1r statement SU ( 9.00 30.00) 9.00 30.00 spkr1s
Chronological display of sys data aligned with ref data for file 'md_test12', channel '1'
----------------------- reference ----------------------- | mapped | --------------------- system output ---------------------
--type-- -subtyp- -----word/spkr----- -tbeg- -tend- | ref_ID | --type-- -subtyp- -----word/spkr----- -tbeg- -tend-
beg SEGMENT <na> spkr1r 9.00 | |
beg SPEAKER adult_fe spkr1r 9.00 | |
9.00 | |beg SPEAKER adult_fe spkr1s=>spkr1r 9.00
beg SU statemen spkr1r 9.00 | SU1 |beg SU statemen spkr1s=>spkr1r 9.00
LEXEME lex WORDZERO 9.00 10.00 | LX1 | LEXEME lex WORDZERO 9.00 10.00
LEXEME lex FIRSTWORD 10.00 11.00 | LX2 | LEXEME lex FIRSTWORD 10.00 11.00
11.00 | ED1 |beg EDIT revision spkr1s=>spkr1r 11.00 dw=-1
LEXEME lex SECONDWORD 11.00 12.00 | LX3 | LEXEME lex SECONDWORD 11.00 12.00
beg EDIT revision spkr1r 12.00 | ED1 |
LEXEME lex THIRDWORD 12.00 13.00 | LX4 | LEXEME lex THIRDWORD 12.00 13.00
LEXEME lex FOURTHWORD 13.00 14.00 | LX5 | LEXEME lex FOURTHWORD 13.00 14.00
14.00 | ED1 |end EDIT revision spkr1s=>spkr1r 14.00 dw=-1
14.00 | **FA** | IP edit spkr1s=>spkr1r 14.00
LEXEME lex FIFTHWORD 14.00 15.00 | LX6 | LEXEME lex FIFTHWORD 14.00 15.00
end EDIT revision spkr1r 15.00 | ED1 |
IP edit spkr1r 15.00 | *Miss* |
LEXEME lex SIXTHWORD 15.00 16.00 | LX7 | LEXEME lex SIXTHWORD 15.00 16.00
16.00 | ED2 |beg EDIT revision spkr1s=>spkr1r 16.00 dw=-2
LEXEME lex SEVENTHWORD 16.00 17.00 | LX8 | LEXEME lex SEVENTHWORD 16.00 17.00
LEXEME lex EIGHTHWORD 17.00 18.00 | LX9 | LEXEME lex EIGHTHWORD 17.00 18.00
beg EDIT revision spkr1r 18.00 | ED2 |
LEXEME lex NINTHWORD 18.00 19.00 | LX10 | LEXEME lex NINTHWORD 18.00 19.00
19.00 | ED2 |end EDIT revision spkr1s=>spkr1r 19.00 dw=-2
19.00 | **FA** | IP edit spkr1s=>spkr1r 19.00
LEXEME lex TENTHWORD 19.00 20.00 | LX11 | LEXEME lex TENTHWORD 19.00 20.00
LEXEME lex ELEVENTHWORD 20.00 21.00 | LX12 | LEXEME lex ELEVENTHWORD 20.00 21.00
end EDIT revision spkr1r 21.00 | ED2 |
21.00 | **FA** |beg EDIT restart spkr1s=>spkr1r 21.00
IP edit spkr1r 21.00 | *Miss* |
LEXEME lex TWELFTHWORD 21.00 22.00 | LX13 | LEXEME lex TWELFTHWORD 21.00 22.00
22.00 | **FA** |end EDIT restart spkr1s=>spkr1r 22.00
22.00 | **FA** | IP edit spkr1s=>spkr1r 22.00
LEXEME lex THIRTEENTHWORD 22.00 23.00 | LX14 | LEXEME lex THIRTEENTHWORD 22.00 23.00
LEXEME lex FOURTEENTHWORD 23.00 24.00 | LX15 | LEXEME lex FOURTEENTHWORD 23.00 24.00
beg EDIT revision spkr1r 24.00 | ED3 |
LEXEME lex FIFTEENTHWORD 24.00 25.00 | LX16 | LEXEME lex FIFTEENTHWORD 24.00 25.00
25.00 | ED3 |beg EDIT revision spkr1s=>spkr1r 25.00 dw=1
LEXEME lex SIXTEENTHWORD 25.00 26.00 | LX17 | LEXEME lex SIXTEENTHWORD 25.00 26.00
26.00 | ED3 |end EDIT revision spkr1s=>spkr1r 26.00 dw=-1
26.00 | **FA** | IP edit spkr1s=>spkr1r 26.00
LEXEME lex SEVENTEENTHWORD 26.00 27.00 | LX18 | LEXEME lex SEVENTEENTHWORD 26.00 27.00
end EDIT revision spkr1r 27.00 | ED3 |
IP edit spkr1r 27.00 | *Miss* |
LEXEME lex EIGHTEENTHWORD 27.00 28.00 | LX19 | LEXEME lex EIGHTEENTHWORD 27.00 28.00
end SPEAKER adult_fe spkr1r 28.00 | |
28.00 | |end SPEAKER adult_fe spkr1s=>spkr1r 28.00
end SEGMENT <na> spkr1r 28.00 | |
LEXEME lex NINETEENTHWORD 28.00 29.00 | LX20 | LEXEME lex NINETEENTHWORD 28.00 29.00
LEXEME lex TWENTIETHWORD 29.00 30.00 | LX21 | LEXEME lex TWENTIETHWORD 29.00 30.00
end SU statemen spkr1r 30.00 | SU1 |end SU statemen spkr1s=>spkr1r 30.00
*** Performance analysis for EDITs *** overall error SCORE = 100.00%
EDIT word coverage statistics -- in terms of reference words
Nref Ndel Nins Nsub %Del %Ins %Sub %D+I %Tot
ALL 9 5 4 0 55.56 44.44 0.00 100.00 100.00
EDIT detection statistics -- in terms of # of EDITs
Nref Ndel Nins Nsub %Del %Ins %Sub %D+I %Tot
ALL 3 0 1 0 0.00 33.33 0.00 33.33 33.33
f=md_test12 3 0 1 0 0.00 33.33 0.00 33.33 33.33
EDIT detection confusion matrix -- in terms of # of EDITs
ALL - ref\sys restart revision {Miss}
restart 0 0 0
revision 0 3 0
{FA} 1 0
EDIT word offset statistics for ALL data
word offsets: <-3 -3 -2 -1 0 1 2 3 >3
BEG: 0 - 1 1 - 1 - - 0
END: 0 - 1 2 - - - - 0
*** Performance analysis for IPs *** overall error SCORE = 233.33%
IP (exact) detection statistics -- in terms of reference words
Nref Ndel Nins Nsub %Del %Ins %Sub %D+I %Tot
ALL 3 3 4 0 100.00 133.33 0.00 233.33 233.33
IP detection statistics -- in terms of # of IPs
Nref Ndel Nins Nsub %Del %Ins %Sub %D+I %Tot
ALL 3 3 4 0 100.00 133.33 0.00 233.33 233.33
f=md_test12 3 3 4 0 100.00 133.33 0.00 233.33 233.33
IP detection confusion matrix -- in terms of # of IPs
ALL - ref\sys edit {Miss}
edit 0 3
{FA} 4
*** Performance analysis for SUs *** overall error SCORE = 0.00%
SU (exact) end detection statistics -- in terms of reference words
Nref Ndel Nins Nsub %Del %Ins %Sub %D+I %Tot
ALL 1 0 0 0 0.00 0.00 0.00 0.00 0.00
SU detection statistics -- in terms of # of SUs
Nref Ndel Nins Nsub %Del %Ins %Sub %D+I %Tot
ALL 1 0 0 0 0.00 0.00 0.00 0.00 0.00
f=md_test12 1 0 0 0 0.00 0.00 0.00 0.00 0.00
SU detection confusion matrix -- in terms of # of SUs
ALL - ref\sys statemen {Miss}
statement 1 0
{FA} 0
SU word offset statistics for ALL data
word offsets: <-3 -3 -2 -1 0 1 2 3 >3
BEG: 0 - - - 1 - - - 0
END: 0 - - - 1 - - - 0
*** Performance analysis for Speaker Diarization for f=md_test12 ***
EVAL TIME = 21.00 secs
EVAL SPEECH = 19.00 secs ( 90.5 percent of evaluated time)
SCORED TIME = 21.00 secs (100.0 percent of evaluated time)
SCORED SPEECH = 19.00 secs ( 90.5 percent of scored time)
EVAL WORDS = 21
SCORED WORDS = 21 (100.0 percent of evaluated words)
---------------------------------------------
MISSED SPEECH = 0.00 secs ( 0.0 percent of scored time)
FALARM SPEECH = 0.00 secs ( 0.0 percent of scored time)
MISSED WORDS = 2 ( 9.5 percent of scored words)
---------------------------------------------
SCORED SPEAKER TIME = 19.00 secs (100.0 percent of scored speech)
MISSED SPEAKER TIME = 0.00 secs ( 0.0 percent of scored speaker time)
FALARM SPEAKER TIME = 0.00 secs ( 0.0 percent of scored speaker time)
SPEAKER ERROR TIME = 0.00 secs ( 0.0 percent of scored speaker time)
SPEAKER ERROR WORDS = 0 ( 0.0 percent of scored speaker words)
---------------------------------------------
OVERALL SPEAKER DIARIZATION ERROR = 0.00 percent of scored speaker time `(f=md_test12)
---------------------------------------------
Speaker type confusion matrix -- speaker weighted
REF\SYS (count) adult_female MISS
adult_female 1 / 100.0% 0 / 0.0%
FALSE ALARM 0 / 0.0%
---------------------------------------------
Speaker type confusion matrix -- time weighted
REF\SYS (seconds) adult_female MISS
adult_female 19.00 / 100.0% 0.00 / 0.0%
FALSE ALARM 0.00 / 0.0%
---------------------------------------------
*** Performance analysis for Speaker Diarization for ALL ***
EVAL TIME = 21.00 secs
EVAL SPEECH = 19.00 secs ( 90.5 percent of evaluated time)
SCORED TIME = 21.00 secs (100.0 percent of evaluated time)
SCORED SPEECH = 19.00 secs ( 90.5 percent of scored time)
EVAL WORDS = 21
SCORED WORDS = 21 (100.0 percent of evaluated words)
---------------------------------------------
MISSED SPEECH = 0.00 secs ( 0.0 percent of scored time)
FALARM SPEECH = 0.00 secs ( 0.0 percent of scored time)
MISSED WORDS = 2 ( 9.5 percent of scored words)
---------------------------------------------
SCORED SPEAKER TIME = 19.00 secs (100.0 percent of scored speech)
MISSED SPEAKER TIME = 0.00 secs ( 0.0 percent of scored speaker time)
FALARM SPEAKER TIME = 0.00 secs ( 0.0 percent of scored speaker time)
SPEAKER ERROR TIME = 0.00 secs ( 0.0 percent of scored speaker time)
SPEAKER ERROR WORDS = 0 ( 0.0 percent of scored speaker words)
---------------------------------------------
OVERALL SPEAKER DIARIZATION ERROR = 0.00 percent of scored speaker time `(ALL)
---------------------------------------------
Speaker type confusion matrix -- speaker weighted
REF\SYS (count) adult_female MISS
adult_female 1 / 100.0% 0 / 0.0%
FALSE ALARM 0 / 0.0%
---------------------------------------------
Speaker type confusion matrix -- time weighted
REF\SYS (seconds) adult_female MISS
adult_female 19.00 / 100.0% 0.00 / 0.0%
FALSE ALARM 0.00 / 0.0%
---------------------------------------------
|