md_test4.output.saved
20.1 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
command line (run on 2004 Oct 29 at 14:18:14): ../src/md-eval-v19a.pl -af -e -D -d -W -w -t 1.0 -l 2 -u md_test4.uem -r md_test4.ref.rttm -s md_test4.sys.rttm
Word-based metadata alignment, max gap between matching words = 1.0 sec
Metadata evaluation parameters:
word-optimized metadata mapping
max gap between matching metadata events = 0.1 words
max extent to match for SU's = 2 words
Speaker Diarization evaluation parameters:
The max time to extend no-score zones for NON-LEX exclusions is 0.5 sec
The no-score collar at SPEAKER boundaries is 0 sec
Exclusion zones for evaluation and scoring are:
-----MetaData----- -----SpkrData-----
exclusion set name: DEFAULT DEFAULT DEFAULT DEFAULT
token type/subtype no-eval no-score no-eval no-score
(UEM) X X
LEXEME/un-lex X
NON-LEX/breath X
NON-LEX/cough X
NON-LEX/laugh X
NON-LEX/lipsmack X
NON-LEX/other X
NON-LEX/sneeze X
NOSCORE/<na> X X X X
NO_RT_METADATA/<na> X
SU/unannotated X
Word alignment and scoring details for channel 1 of file md_test4
ref del ins sub REF: token type tbeg tend speaker SYS: token type Rtbeg Rtend tbeg tend sys-speaker
1 - - 0 firstword lex 10.00 11.00 spkr1r firstword lex ( 10.00 11.00) 10.00 11.00 spkr1s
1 - - 0 t.'s lex 11.00 12.00 spkr1r t.'s lex ( 11.00 12.00) 11.00 12.00 spkr1s
1 - - 0 thirdword lex 12.00 13.00 spkr1r thirdword lex ( 12.00 13.00) 12.00 13.00 spkr1s
1 - - 0 fourthword lex 13.00 14.00 spkr1r fourthword lex ( 13.00 14.00) 13.00 14.00 spkr1s
1 - - 0 fifthword lex 14.00 15.00 spkr1r fifthword lex ( 14.00 15.00) 14.00 15.00 spkr1s
1 - - 0 sixthword lex 15.00 16.00 spkr1r sixthword lex ( 15.00 16.00) 15.00 16.00 spkr1s
1 - - 0 seventhword lex 16.00 17.00 name="<nar>" seventhword lex ( 16.00 17.00) 16.00 17.00 name="<nas>"
1 - - 0 eighthword lex 17.00 18.00 name="<nar>" eighthword lex ( 17.00 18.00) 17.00 18.00 name="<nas>"
1 1 - - ninthword lex 18.00 19.00 name="<nar>" --- --- ( --- --- ) --- --- ---
1 - - 0 tenthword lex 19.00 20.00 spkr=<sp3r> tenthword lex ( 19.00 20.00) 19.00 20.00 spkr=<sp3s>
1 - - 0 eleventhword fp 20.00 21.00 spkr=<sp3r> eleventhword fp ( 20.00 21.00) 20.00 21.00 spkr=<sp3s>
1 - - 0 twelfthword lex 21.00 22.00 spkr=<sp3r> twelfthword lex ( 21.00 22.00) 21.00 22.00 spkr=<sp3s>
1 1 - - thirteenthword lex 22.00 23.00 spkr=<sp3r> --- --- ( --- --- ) --- --- ---
1 1 - - fourteenthword lex 23.00 24.00 spkr=<sp3r> --- --- ( --- --- ) --- --- ---
1 - - 0 fifteenthword lex 24.00 25.00 spkr=<sp3r> fifteenthword lex ( 24.00 25.00) 24.00 25.00 spkr=<sp3s>
1 - - 0 sixteenthword fp 25.00 26.00 spkr=<sp3r> sixteenthword fp ( 25.00 26.00) 25.00 26.00 spkr=<sp3s>
1 - - 0 seventeenthword lex 26.00 27.00 spkr=<sp3r> seventeenthword lex ( 26.00 27.00) 26.00 27.00 spkr=<sp3s>
1 1 - - eighteenthword lex 27.00 28.00 spkr=<sp3r> --- --- ( --- --- ) --- --- ---
1 1 - - nineteenthword lex 28.00 29.00 spkr=<sp3r> --- --- ( --- --- ) --- --- ---
1 - - 0 twentiethword lex 29.00 30.00 spkr=<sp3r> twentiethword lex ( 29.00 30.00) 29.00 30.00 spkr=<sp3s>
FILLER alignment and scoring details for channel 1 of file md_test4
ref del ins sub REF: token type tbeg tend speaker SYS: token type Rtbeg Rtend tbeg tend sys-speaker
1 - - 0 filled_pause FILLER 20.00 21.00 spkr=<sp3r> filled_pause FILLER ( 20.00 21.00) 20.00 21.00 spkr=<sp3s>
1 - - 0 filled_pause FILLER 25.00 26.00 spkr=<sp3r> filled_pause FILLER ( 25.00 26.00) 25.00 26.00 spkr=<sp3s>
IP alignment and scoring details for channel 1 of file md_test4
ref del ins sub REF: token type tbeg tend speaker SYS: token type Rtbeg Rtend tbeg tend sys-speaker
1 - - 0 filler IP 20.00 20.00 spkr=<sp3r> filler IP ( 20.00 20.00) 20.00 20.00 spkr=<sp3s>
1 - - 0 filler IP 25.00 25.00 spkr=<sp3r> filler IP ( 25.00 25.00) 25.00 25.00 spkr=<sp3s>
SU alignment and scoring details for channel 1 of file md_test4
ref del ins sub REF: token type tbeg tend speaker SYS: token type Rtbeg Rtend tbeg tend sys-speaker
1 - - 0 statement SU 10.00 13.00 spkr1r statement SU ( 10.00 13.00) 10.00 13.00 spkr1s
1 - - 0 question SU 13.00 16.00 spkr1r question SU ( 13.00 16.00) 13.00 16.00 spkr1s
1 - - 0 backchannel SU 16.00 19.00 name="<nar>" backchannel SU ( 16.00 19.00) 16.00 19.00 name="<nas>"
1 - - 0 question SU 19.00 24.00 spkr=<sp3r> question SU ( 19.00 22.00) 19.00 22.00 spkr=<sp3s>
1 - - 0 question SU 24.00 29.00 spkr=<sp3r> question SU ( 24.00 27.00) 24.00 27.00 spkr=<sp3s>
1 - - 0 question SU 29.00 30.00 spkr=<sp3r> question SU ( 29.00 30.00) 29.00 30.00 spkr=<sp3s>
Chronological display of sys data aligned with ref data for file 'md_test4', channel '1'
----------------------- reference ----------------------- | mapped | --------------------- system output ---------------------
--type-- -subtyp- -----word/spkr----- -tbeg- -tend- | ref_ID | --type-- -subtyp- -----word/spkr----- -tbeg- -tend-
beg SEGMENT <na> spkr1r 10.00 | |
beg SPEAKER child spkr1r 10.00 | |
10.00 | |beg SPEAKER child spkr1s=>spkr1r 10.00
beg SU statemen spkr1r 10.00 | SU1 |beg SU statemen spkr1s=>spkr1r 10.00
LEXEME lex FIRSTWORD 10.00 11.00 | LX1 | LEXEME lex FIRSTWORD 10.00 11.00
LEXEME alpha T.'S 11.00 12.00 | LX2 | LEXEME alpha T.'S 11.00 12.00
LEXEME acronym THIRDWORD 12.00 13.00 | LX3 | LEXEME acronym THIRDWORD 12.00 13.00
end SU statemen spkr1r 13.00 | SU1 |end SU statemen spkr1s=>spkr1r 13.00
beg SU question spkr1r 13.00 | SU2 |beg SU question spkr1s=>spkr1r 13.00
LEXEME interjec FOURTHWORD 13.00 14.00 | LX4 | LEXEME interjec FOURTHWORD 13.00 14.00
LEXEME properno FIFTHWORD 14.00 15.00 | LX5 | LEXEME properno FIFTHWORD 14.00 15.00
LEXEME other SIXTHWORD 15.00 16.00 | LX6 | LEXEME other SIXTHWORD 15.00 16.00
end SU question spkr1r 16.00 | SU2 |end SU question spkr1s=>spkr1r 16.00
end SPEAKER child spkr1r 16.00 | |
16.00 | |end SPEAKER child spkr1s=>spkr1r 16.00
end SEGMENT <na> spkr1r 16.00 | |
beg SEGMENT <na> name="<nar>" 16.00 | |
beg SPEAKER unknown name="<nar>" 16.00 | |
16.00 | |beg SPEAKER unknown name="<nas>"=>name= 16.00
beg SU backchan name="<nar>" 16.00 | SU3 |beg SU backchan name="<nas>"=>name= 16.00
LEXEME lex SEVENTHWORD 16.00 17.00 | LX7 | LEXEME lex SEVENTHWORD 16.00 17.00
LEXEME lex EIGHTHWORD 17.00 18.00 | LX8 | LEXEME lex EIGHTHWORD 17.00 18.00
LEXEME lex NINTHWORD 18.00 19.00 | |
end SU backchan name="<nar>" 19.00 | SU3 |end SU backchan name="<nas>"=>name= 19.00
end SPEAKER unknown name="<nar>" 19.00 | |
19.00 | |end SPEAKER unknown name="<nas>"=>name= 19.00
end SEGMENT <na> name="<nar>" 19.00 | |
beg SEGMENT <na> spkr=<sp3r> 19.00 | |
beg SPEAKER unknown spkr=<sp3r> 19.00 | |
19.00 | |beg SPEAKER unknown spkr=<sp3s>=>spkr=< 19.00
beg SU question spkr=<sp3r> 19.00 | SU4 |beg SU question spkr=<sp3s>=>spkr=< 19.00
LEXEME lex TENTHWORD 19.00 20.00 | LX10 | LEXEME lex TENTHWORD 19.00 20.00
beg FILLER filled_p spkr=<sp3r> 20.00 | FL1 |beg FILLER filled_p spkr=<sp3s>=>spkr=< 20.00
IP filler spkr=<sp3r> 20.00 | IP1 | IP filler spkr=<sp3s>=>spkr=< 20.00
LEXEME fp ELEVENTHWORD 20.00 21.00 | LX11 | LEXEME fp ELEVENTHWORD 20.00 21.00
end FILLER filled_p spkr=<sp3r> 21.00 | FL1 |end FILLER filled_p spkr=<sp3s>=>spkr=< 21.00
LEXEME lex TWELFTHWORD 21.00 22.00 | LX12 | LEXEME lex TWELFTHWORD 21.00 22.00
end SU question spkr=<sp3r> 24.00 | SU4 |end SU question spkr=<sp3s>=>spkr=< 22.00
beg SU question spkr=<sp3r> 24.00 | SU5 |beg SU question spkr=<sp3s>=>spkr=< 24.00
LEXEME lex FIFTEENTHWORD 24.00 25.00 | LX13 | LEXEME lex FIFTEENTHWORD 24.00 25.00
beg FILLER filled_p spkr=<sp3r> 25.00 | FL2 |beg FILLER filled_p spkr=<sp3s>=>spkr=< 25.00
IP filler spkr=<sp3r> 25.00 | IP2 | IP filler spkr=<sp3s>=>spkr=< 25.00
LEXEME fp SIXTEENTHWORD 25.00 26.00 | LX14 | LEXEME fp SIXTEENTHWORD 25.00 26.00
end FILLER filled_p spkr=<sp3r> 26.00 | FL2 |end FILLER filled_p spkr=<sp3s>=>spkr=< 26.00
LEXEME lex SEVENTEENTHWORD 26.00 27.00 | LX15 | LEXEME lex SEVENTEENTHWORD 26.00 27.00
27.00 | SU5 |end SU question spkr=<sp3s>=>spkr=< 27.00 dw=-1
LEXEME lex EIGHTEENTHWORD 27.00 28.00 | |
end SU question spkr=<sp3r> 29.00 | SU5 |
beg SU question spkr=<sp3r> 29.00 | SU6 |beg SU question spkr=<sp3s>=>spkr=< 29.00
LEXEME lex TWENTIETHWORD 29.00 30.00 | LX17 | LEXEME lex TWENTIETHWORD 29.00 30.00
end SU question spkr=<sp3r> 30.00 | SU6 |end SU question spkr=<sp3s>=>spkr=< 30.00
end SPEAKER unknown spkr=<sp3r> 30.00 | |
30.00 | |end SPEAKER unknown spkr=<sp3s>=>spkr=< 30.00
end SEGMENT <na> spkr=<sp3r> 30.00 | |
*** Performance analysis for FILLERs *** overall error SCORE = 0.00%
FILLER word coverage statistics -- in terms of reference words
Nref Ndel Nins Nsub %Del %Ins %Sub %D+I %Tot
ALL 2 0 0 0 0.00 0.00 0.00 0.00 0.00
FILLER detection statistics -- in terms of # of FILLERs
Nref Ndel Nins Nsub %Del %Ins %Sub %D+I %Tot
ALL 2 0 0 0 0.00 0.00 0.00 0.00 0.00
f=md_test4 2 0 0 0 0.00 0.00 0.00 0.00 0.00
FILLER detection confusion matrix -- in terms of # of FILLERs
ALL - ref\sys filled_p {Miss}
filled_pause 2 0
{FA} 0
FILLER word offset statistics for ALL data
word offsets: <-3 -3 -2 -1 0 1 2 3 >3
BEG: 0 - - - 2 - - - 0
END: 0 - - - 2 - - - 0
*** Performance analysis for IPs *** overall error SCORE = 0.00%
IP (exact) detection statistics -- in terms of reference words
Nref Ndel Nins Nsub %Del %Ins %Sub %D+I %Tot
ALL 2 0 0 0 0.00 0.00 0.00 0.00 0.00
IP detection statistics -- in terms of # of IPs
Nref Ndel Nins Nsub %Del %Ins %Sub %D+I %Tot
ALL 2 0 0 0 0.00 0.00 0.00 0.00 0.00
f=md_test4 2 0 0 0 0.00 0.00 0.00 0.00 0.00
IP detection confusion matrix -- in terms of # of IPs
ALL - ref\sys filler {Miss}
filler 2 0
{FA} 0
IP word offset statistics for ALL data
word offsets: <-3 -3 -2 -1 0 1 2 3 >3
BEG: 0 - - - 2 - - - 0
END: 0 - - - 2 - - - 0
*** Performance analysis for SUs *** overall error SCORE = 66.67%
SU (exact) end detection statistics -- in terms of reference words
Nref Ndel Nins Nsub %Del %Ins %Sub %D+I %Tot
ALL 6 2 2 0 33.33 33.33 0.00 66.67 66.67
SU detection statistics -- in terms of # of SUs
Nref Ndel Nins Nsub %Del %Ins %Sub %D+I %Tot
ALL 6 0 0 0 0.00 0.00 0.00 0.00 0.00
f=md_test4 6 0 0 0 0.00 0.00 0.00 0.00 0.00
SU detection confusion matrix -- in terms of # of SUs
ALL - ref\sys backchan question statemen {Miss}
backchannel 1 0 0 0
question 0 4 0 0
statement 0 0 1 0
{FA} 0 0 0
SU word offset statistics for ALL data
word offsets: <-3 -3 -2 -1 0 1 2 3 >3
BEG: 0 - - - 6 - - - 0
END: 0 - - 1 5 - - - 0
*** Performance analysis for Speaker Diarization for f=md_test4 ***
EVAL TIME = 16.99 secs
EVAL SPEECH = 16.99 secs (100.0 percent of evaluated time)
SCORED TIME = 16.99 secs (100.0 percent of evaluated time)
SCORED SPEECH = 16.99 secs (100.0 percent of scored time)
EVAL WORDS = 17
SCORED WORDS = 17 (100.0 percent of evaluated words)
---------------------------------------------
MISSED SPEECH = 0.00 secs ( 0.0 percent of scored time)
FALARM SPEECH = 0.00 secs ( 0.0 percent of scored time)
MISSED WORDS = 0 ( 0.0 percent of scored words)
---------------------------------------------
SCORED SPEAKER TIME = 16.99 secs (100.0 percent of scored speech)
MISSED SPEAKER TIME = 0.00 secs ( 0.0 percent of scored speaker time)
FALARM SPEAKER TIME = 0.00 secs ( 0.0 percent of scored speaker time)
SPEAKER ERROR TIME = 0.00 secs ( 0.0 percent of scored speaker time)
SPEAKER ERROR WORDS = 0 ( 0.0 percent of scored speaker words)
---------------------------------------------
OVERALL SPEAKER DIARIZATION ERROR = 0.00 percent of scored speaker time `(f=md_test4)
---------------------------------------------
Speaker type confusion matrix -- speaker weighted
REF\SYS (count) child unknown MISS
child 1 / 33.3% 0 / 0.0% 0 / 0.0%
unknown 0 / 0.0% 2 / 66.7% 0 / 0.0%
FALSE ALARM 0 / 0.0% 0 / 0.0%
---------------------------------------------
Speaker type confusion matrix -- time weighted
REF\SYS (seconds) child unknown MISS
child 6.00 / 35.3% 0.00 / 0.0% 0.00 / 0.0%
unknown 0.00 / 0.0% 10.99 / 64.7% 0.00 / 0.0%
FALSE ALARM 0.00 / 0.0% 0.00 / 0.0%
---------------------------------------------
*** Performance analysis for Speaker Diarization for ALL ***
EVAL TIME = 16.99 secs
EVAL SPEECH = 16.99 secs (100.0 percent of evaluated time)
SCORED TIME = 16.99 secs (100.0 percent of evaluated time)
SCORED SPEECH = 16.99 secs (100.0 percent of scored time)
EVAL WORDS = 17
SCORED WORDS = 17 (100.0 percent of evaluated words)
---------------------------------------------
MISSED SPEECH = 0.00 secs ( 0.0 percent of scored time)
FALARM SPEECH = 0.00 secs ( 0.0 percent of scored time)
MISSED WORDS = 0 ( 0.0 percent of scored words)
---------------------------------------------
SCORED SPEAKER TIME = 16.99 secs (100.0 percent of scored speech)
MISSED SPEAKER TIME = 0.00 secs ( 0.0 percent of scored speaker time)
FALARM SPEAKER TIME = 0.00 secs ( 0.0 percent of scored speaker time)
SPEAKER ERROR TIME = 0.00 secs ( 0.0 percent of scored speaker time)
SPEAKER ERROR WORDS = 0 ( 0.0 percent of scored speaker words)
---------------------------------------------
OVERALL SPEAKER DIARIZATION ERROR = 0.00 percent of scored speaker time `(ALL)
---------------------------------------------
Speaker type confusion matrix -- speaker weighted
REF\SYS (count) child unknown MISS
child 1 / 33.3% 0 / 0.0% 0 / 0.0%
unknown 0 / 0.0% 2 / 66.7% 0 / 0.0%
FALSE ALARM 0 / 0.0% 0 / 0.0%
---------------------------------------------
Speaker type confusion matrix -- time weighted
REF\SYS (seconds) child unknown MISS
child 6.00 / 35.3% 0.00 / 0.0% 0.00 / 0.0%
unknown 0.00 / 0.0% 10.99 / 64.7% 0.00 / 0.0%
FALSE ALARM 0.00 / 0.0% 0.00 / 0.0%
---------------------------------------------