Blame view
tools/sctk-2.4.10/src/md-eval/test/sd_test6.output.saved
15.1 KB
8dcb6dfcb
first commit
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 |
command line (run on 2004 Oct 29 at 14:13:46): ../src/md-eval-v19a.pl -1 -v -e -d -D -m -af -c 0.0 -T 0.0 -u sd_test6.uem -r sd_test6.ref.rttm -s sd_test6.sys.rttm
Time-based metadata alignment
Metadata evaluation parameters:
time-optimized metadata mapping
max gap between matching metadata events = 0.0 sec
max extent to match for SU's = 0.5 sec
Speaker Diarization evaluation parameters:
The max time to extend no-score zones for NON-LEX exclusions is 0.5 sec
The no-score collar at SPEAKER boundaries is 0.0 sec
Exclusion zones for evaluation and scoring are:
-----MetaData----- -----SpkrData-----
exclusion set name: DEFAULT DEFAULT DEFAULT DEFAULT
token type/subtype no-eval no-score no-eval no-score
(UEM) X X
LEXEME/un-lex X
NON-LEX/breath X
NON-LEX/cough X
NON-LEX/laugh X
NON-LEX/lipsmack X
NON-LEX/other X
NON-LEX/sneeze X
NOSCORE/<na> X X X X
NO_RT_METADATA/<na> X
SU/unannotated X
Word alignment and scoring details for channel 1 of file sd_test6
ref del ins sub REF: token type tbeg tend speaker SYS: token type tbeg tend speaker
1 - - 0 firstword lex 10.00 11.00 spkr1r firstword lex 10.00 11.00 spkr1s
1 - - 0 t.'s lex 11.00 12.00 spkr1r t.'s lex 11.00 12.00 spkr1s
1 - - 0 thirdword lex 12.00 13.00 spkr1r thirdword lex 12.00 13.00 spkr1s
1 - - 0 fourthword lex 13.00 14.00 spkr1r fourthword lex 13.00 14.00 spkr1s
1 - - 0 fifthword lex 14.00 15.00 spkr1r fifthword lex 14.00 15.00 spkr1s
1 - - 0 sixthword lex 15.00 16.00 spkr1r sixthword lex 15.00 16.00 spkr1s
1 - - 0 seventhword lex 16.00 17.00 spkr2r seventhword lex 16.00 17.00 spkr2s
1 - - 0 eighthword lex 17.00 18.00 spkr2r eighthword lex 17.00 18.00 spkr2s
1 - - 0 ninthword lex 18.00 19.00 spkr2r ninthword lex 18.00 19.00 spkr2s
1 - - 0 tenthword lex 19.00 20.00 spkr3r tenthword lex 19.00 20.00 spkr3s
1 - - 0 eleventhword lex 20.00 21.00 spkr3r eleventhword lex 20.00 21.00 spkr3s
SU alignment and scoring details for channel 1 of file sd_test6
ref del ins sub REF: token type tbeg tend speaker SYS: token type tbeg tend speaker
1 - - 0 statement SU 10.00 12.00 spkr1r statement SU 10.00 12.00 spkr1s
1 - - 0 statement SU 12.00 14.00 spkr1r statement SU 12.00 14.00 spkr1s
1 - - 0 question SU 14.00 16.00 spkr1r question SU 14.00 16.00 spkr1s
1 - - 0 backchannel SU 16.00 19.00 spkr2r backchannel SU 16.00 19.00 spkr2s
1 - - 0 incomplete SU 19.00 21.00 spkr3r incomplete SU 19.00 21.00 spkr3s
'spkr1r' => 'spkr1s'
6.00 secs matched to 'spkr1s'
'spkr2r' => 'spkr2s'
3.00 secs matched to 'spkr2s'
'spkr3r' => 'spkr3s'
2.00 secs matched to 'spkr3s'
beg/dur/end = 10.000/ 2.000/ 12.000; REF = (spkr1r); SYS = (spkr1s)
beg/dur/end = 12.000/ 2.000/ 14.000; REF = (spkr1r); SYS = (spkr1s)
beg/dur/end = 14.000/ 2.000/ 16.000; REF = (spkr1r); SYS = (spkr1s)
beg/dur/end = 16.000/ 3.000/ 19.000; REF = (spkr2r); SYS = (spkr2s)
beg/dur/end = 19.000/ 2.000/ 21.000; REF = (spkr3r); SYS = (spkr3s)
Chronological display of sys data aligned with ref data for file 'sd_test6', channel '1'
----------------------- reference ----------------------- | mapped | --------------------- system output ---------------------
--type-- -subtyp- -----word/spkr----- -tbeg- -tend- | ref_ID | --type-- -subtyp- -----word/spkr----- -tbeg- -tend-
beg SEGMENT <na> spkr1r 10.00 | |
beg SPEAKER child spkr1r 10.00 | |
10.00 | |beg SPEAKER child spkr1s=>spkr1r 10.00
beg SU statemen spkr1r 10.00 | SU1 |beg SU statemen spkr1s=>spkr1r 10.00
LEXEME lex FIRSTWORD 10.00 11.00 | LX1 | LEXEME lex FIRSTWORD 10.00 11.00
LEXEME alpha T.'S 11.00 12.00 | LX2 | LEXEME alpha T.'S 11.00 12.00
end SU statemen spkr1r 12.00 | SU1 |end SU statemen spkr1s=>spkr1r 12.00
12.00 | |beg SPEAKER child spkr1s=>spkr1r 12.00
beg SU statemen spkr1r 12.00 | SU2 |beg SU statemen spkr1s=>spkr1r 12.00
LEXEME acronym THIRDWORD 12.00 13.00 | LX3 | LEXEME acronym THIRDWORD 12.00 13.00
LEXEME interjec FOURTHWORD 13.00 14.00 | LX4 | LEXEME interjec FOURTHWORD 13.00 14.00
end SU statemen spkr1r 14.00 | SU2 |end SU statemen spkr1s=>spkr1r 14.00
14.00 | |end SPEAKER child spkr1s=>spkr1r 14.00
beg SU question spkr1r 14.00 | SU3 |beg SU question spkr1s=>spkr1r 14.00
LEXEME properno FIFTHWORD 14.00 15.00 | LX5 | LEXEME properno FIFTHWORD 14.00 15.00
LEXEME other SIXTHWORD 15.00 16.00 | LX6 | LEXEME other SIXTHWORD 15.00 16.00
end SU question spkr1r 16.00 | SU3 |end SU question spkr1s=>spkr1r 16.00
end SPEAKER child spkr1r 16.00 | |
16.00 | |end SPEAKER child spkr1s=>spkr1r 16.00
end SEGMENT <na> spkr1r 16.00 | |
beg SEGMENT <na> spkr2r 16.00 | |
beg SPEAKER unknown spkr2r 16.00 | |
16.00 | |beg SPEAKER unknown spkr2s=>spkr2r 16.00
beg SU backchan spkr2r 16.00 | SU4 |beg SU backchan spkr2s=>spkr2r 16.00
LEXEME lex SEVENTHWORD 16.00 17.00 | LX7 | LEXEME lex SEVENTHWORD 16.00 17.00
LEXEME lex EIGHTHWORD 17.00 18.00 | LX8 | LEXEME lex EIGHTHWORD 17.00 18.00
LEXEME lex NINTHWORD 18.00 19.00 | LX9 | LEXEME lex NINTHWORD 18.00 19.00
end SU backchan spkr2r 19.00 | SU4 |end SU backchan spkr2s=>spkr2r 19.00
end SPEAKER unknown spkr2r 19.00 | |
19.00 | |end SPEAKER unknown spkr2s=>spkr2r 19.00
end SEGMENT <na> spkr2r 19.00 | |
beg SEGMENT <na> spkr3r 19.00 | |
beg SPEAKER adult_fe spkr3r 19.00 | |
19.00 | |beg SPEAKER adult_fe spkr3s=>spkr3r 19.00
beg SU incomple spkr3r 19.00 | SU5 |beg SU incomple spkr3s=>spkr3r 19.00
LEXEME lex TENTHWORD 19.00 20.00 | LX10 | LEXEME lex TENTHWORD 19.00 20.00
LEXEME lex ELEVENTHWORD 20.00 21.00 | LX11 | LEXEME lex ELEVENTHWORD 20.00 21.00
end SU incomple spkr3r 21.00 | SU5 |end SU incomple spkr3s=>spkr3r 21.00
end SPEAKER adult_fe spkr3r 22.00 | |
22.00 | |end SPEAKER adult_fe spkr3s=>spkr3r 22.00
end SEGMENT <na> spkr3r 22.00 | |
*** Performance analysis for SUs *** overall error SCORE = 0.00%
SU (exact) end detection statistics -- in terms of reference words
Nref Ndel Nins Nsub %Del %Ins %Sub %D+I %Tot
ALL 5 0 0 0 0.00 0.00 0.00 0.00 0.00
SU detection statistics -- in terms of # of SUs
Nref Ndel Nins Nsub %Del %Ins %Sub %D+I %Tot
ALL 5 0 0 0 0.00 0.00 0.00 0.00 0.00
f=sd_test6 5 0 0 0 0.00 0.00 0.00 0.00 0.00
SU detection confusion matrix -- in terms of # of SUs
ALL - ref\sys backchan incomple question statemen {Miss}
backchannel 1 0 0 0 0
incomplete 0 1 0 0 0
question 0 0 1 0 0
statement 0 0 0 2 0
{FA} 0 0 0 0
SU word offset statistics for ALL data
word offsets: <-3 -3 -2 -1 0 1 2 3 >3
BEG: 0 - - - 5 - - - 0
END: 0 - - - 5 - - - 0
*** Performance analysis for Speaker Diarization for f=sd_test6 ***
EVAL TIME = 11.00 secs
EVAL SPEECH = 11.00 secs (100.0 percent of evaluated time)
SCORED TIME = 11.00 secs (100.0 percent of evaluated time)
SCORED SPEECH = 11.00 secs (100.0 percent of scored time)
EVAL WORDS = 11
SCORED WORDS = 11 (100.0 percent of evaluated words)
---------------------------------------------
MISSED SPEECH = 0.00 secs ( 0.0 percent of scored time)
FALARM SPEECH = 0.00 secs ( 0.0 percent of scored time)
MISSED WORDS = 0 ( 0.0 percent of scored words)
---------------------------------------------
SCORED SPEAKER TIME = 11.00 secs (100.0 percent of scored speech)
MISSED SPEAKER TIME = 0.00 secs ( 0.0 percent of scored speaker time)
FALARM SPEAKER TIME = 0.00 secs ( 0.0 percent of scored speaker time)
SPEAKER ERROR TIME = 0.00 secs ( 0.0 percent of scored speaker time)
SPEAKER ERROR WORDS = 0 ( 0.0 percent of scored speaker words)
---------------------------------------------
OVERALL SPEAKER DIARIZATION ERROR = 0.00 percent of scored speaker time `(f=sd_test6)
---------------------------------------------
Speaker type confusion matrix -- speaker weighted
REF\SYS (count) adult_female child unknown MISS
adult_female 1 / 33.3% 0 / 0.0% 0 / 0.0% 0 / 0.0%
child 0 / 0.0% 1 / 33.3% 0 / 0.0% 0 / 0.0%
unknown 0 / 0.0% 0 / 0.0% 1 / 33.3% 0 / 0.0%
FALSE ALARM 0 / 0.0% 0 / 0.0% 0 / 0.0%
---------------------------------------------
Speaker type confusion matrix -- time weighted
REF\SYS (seconds) adult_female child unknown MISS
adult_female 2.00 / 18.2% 0.00 / 0.0% 0.00 / 0.0% 0.00 / 0.0%
child 0.00 / 0.0% 6.00 / 54.5% 0.00 / 0.0% 0.00 / 0.0%
unknown 0.00 / 0.0% 0.00 / 0.0% 3.00 / 27.3% 0.00 / 0.0%
FALSE ALARM 0.00 / 0.0% 0.00 / 0.0% 0.00 / 0.0%
---------------------------------------------
*** Performance analysis for Speaker Diarization for ALL ***
EVAL TIME = 11.00 secs
EVAL SPEECH = 11.00 secs (100.0 percent of evaluated time)
SCORED TIME = 11.00 secs (100.0 percent of evaluated time)
SCORED SPEECH = 11.00 secs (100.0 percent of scored time)
EVAL WORDS = 11
SCORED WORDS = 11 (100.0 percent of evaluated words)
---------------------------------------------
MISSED SPEECH = 0.00 secs ( 0.0 percent of scored time)
FALARM SPEECH = 0.00 secs ( 0.0 percent of scored time)
MISSED WORDS = 0 ( 0.0 percent of scored words)
---------------------------------------------
SCORED SPEAKER TIME = 11.00 secs (100.0 percent of scored speech)
MISSED SPEAKER TIME = 0.00 secs ( 0.0 percent of scored speaker time)
FALARM SPEAKER TIME = 0.00 secs ( 0.0 percent of scored speaker time)
SPEAKER ERROR TIME = 0.00 secs ( 0.0 percent of scored speaker time)
SPEAKER ERROR WORDS = 0 ( 0.0 percent of scored speaker words)
---------------------------------------------
OVERALL SPEAKER DIARIZATION ERROR = 0.00 percent of scored speaker time `(ALL)
---------------------------------------------
Speaker type confusion matrix -- speaker weighted
REF\SYS (count) adult_female child unknown MISS
adult_female 1 / 33.3% 0 / 0.0% 0 / 0.0% 0 / 0.0%
child 0 / 0.0% 1 / 33.3% 0 / 0.0% 0 / 0.0%
unknown 0 / 0.0% 0 / 0.0% 1 / 33.3% 0 / 0.0%
FALSE ALARM 0 / 0.0% 0 / 0.0% 0 / 0.0%
---------------------------------------------
Speaker type confusion matrix -- time weighted
REF\SYS (seconds) adult_female child unknown MISS
adult_female 2.00 / 18.2% 0.00 / 0.0% 0.00 / 0.0% 0.00 / 0.0%
child 0.00 / 0.0% 6.00 / 54.5% 0.00 / 0.0% 0.00 / 0.0%
unknown 0.00 / 0.0% 0.00 / 0.0% 3.00 / 27.3% 0.00 / 0.0%
FALSE ALARM 0.00 / 0.0% 0.00 / 0.0% 0.00 / 0.0%
---------------------------------------------
|