Blame view
tools/sctk-2.4.10/src/hubscr/test1-sastt.base/sastt-case1.sys.rttm.filt.mdeval
6.97 KB
8dcb6dfcb
first commit
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 |
command line (run on 2009 May 11 at 13:31:08) Version: 22 ../../md-eval/md-eval.pl -nafcs -c 0.25 -o -r sastt-case1.ref.rttm.filt -s sastt-case1.sys.rttm.filt -M sastt-case1.sys.rttm.filt.mdeval.spkrmap
Time-based metadata alignment
Metadata evaluation parameters:
time-optimized metadata mapping
max gap between matching metadata events = 1 sec
max extent to match for SU's = 0.5 sec
Speaker Diarization evaluation parameters:
The max time to extend no-score zones for NON-LEX exclusions is 0.5 sec
The no-score collar at SPEAKER boundaries is 0.25 sec
Exclusion zones for evaluation and scoring are:
-----MetaData----- -----SpkrData-----
exclusion set name: DEFAULT DEFAULT DEFAULT DEFAULT
token type/subtype no-eval no-score no-eval no-score
(UEM) X X
LEXEME/un-lex X
NON-LEX/breath X
NON-LEX/cough X
NON-LEX/laugh X
NON-LEX/lipsmack X
NON-LEX/other X
NON-LEX/sneeze X
NOSCORE/<na> X X X X
NO_RT_METADATA/<na> X
SU/unannotated X
*** Performance analysis for Speaker Diarization for c=1 f=ICSI_20011030-1030_d*_NONE ***
EVAL TIME = 30.00 secs
EVAL SPEECH = 30.00 secs (100.0 percent of evaluated time)
SCORED TIME = 28.50 secs ( 95.0 percent of evaluated time)
SCORED SPEECH = 28.50 secs (100.0 percent of scored time)
EVAL WORDS = 14
SCORED WORDS = 14 (100.0 percent of evaluated words)
---------------------------------------------
MISSED SPEECH = 0.00 secs ( 0.0 percent of scored time)
FALARM SPEECH = 0.00 secs ( 0.0 percent of scored time)
MISSED WORDS = 0 ( 0.0 percent of scored words)
---------------------------------------------
SCORED SPEAKER TIME = 28.50 secs (100.0 percent of scored speech)
MISSED SPEAKER TIME = 0.00 secs ( 0.0 percent of scored speaker time)
FALARM SPEAKER TIME = 0.00 secs ( 0.0 percent of scored speaker time)
SPEAKER ERROR TIME = 0.00 secs ( 0.0 percent of scored speaker time)
SPEAKER ERROR WORDS = 0 ( 0.0 percent of scored speaker words)
---------------------------------------------
OVERALL SPEAKER DIARIZATION ERROR = 0.00 percent of scored speaker time `(c=1 f=ICSI_20011030-1030_d*_NONE)
---------------------------------------------
Speaker type confusion matrix -- speaker weighted
REF\SYS (count) adult_male MISS
adult_male 1 / 100.0% 0 / 0.0%
FALSE ALARM 0 / 0.0%
---------------------------------------------
Speaker type confusion matrix -- time weighted
REF\SYS (seconds) adult_male MISS
adult_male 28.50 / 100.0% 0.00 / 0.0%
FALSE ALARM 0.00 / 0.0%
---------------------------------------------
*** Performance analysis for Speaker Diarization for c=1 f=VT_20051027-1400 ***
EVAL TIME = 7.50 secs
EVAL SPEECH = 7.50 secs (100.0 percent of evaluated time)
SCORED TIME = 5.40 secs ( 72.0 percent of evaluated time)
SCORED SPEECH = 5.40 secs (100.0 percent of scored time)
EVAL WORDS = 9
SCORED WORDS = 7 ( 77.8 percent of evaluated words)
---------------------------------------------
MISSED SPEECH = 0.75 secs ( 13.9 percent of scored time)
FALARM SPEECH = 0.00 secs ( 0.0 percent of scored time)
MISSED WORDS = 0 ( 0.0 percent of scored words)
---------------------------------------------
SCORED SPEAKER TIME = 6.80 secs (125.9 percent of scored speech)
MISSED SPEAKER TIME = 2.15 secs ( 31.6 percent of scored speaker time)
FALARM SPEAKER TIME = 0.25 secs ( 3.7 percent of scored speaker time)
SPEAKER ERROR TIME = 0.00 secs ( 0.0 percent of scored speaker time)
SPEAKER ERROR WORDS = 3 ( 42.9 percent of scored speaker words)
---------------------------------------------
OVERALL SPEAKER DIARIZATION ERROR = 35.29 percent of scored speaker time `(c=1 f=VT_20051027-1400)
---------------------------------------------
Speaker type confusion matrix -- speaker weighted
REF\SYS (count) unknown MISS
unknown 2 / 100.0% 0 / 0.0%
FALSE ALARM 0 / 0.0%
---------------------------------------------
Speaker type confusion matrix -- time weighted
REF\SYS (seconds) unknown MISS
unknown 4.65 / 68.4% 2.15 / 31.6%
FALSE ALARM 0.25 / 3.7%
---------------------------------------------
*** Performance analysis for Speaker Diarization for ALL ***
EVAL TIME = 37.50 secs
EVAL SPEECH = 37.50 secs (100.0 percent of evaluated time)
SCORED TIME = 33.90 secs ( 90.4 percent of evaluated time)
SCORED SPEECH = 33.90 secs (100.0 percent of scored time)
EVAL WORDS = 23
SCORED WORDS = 21 ( 91.3 percent of evaluated words)
---------------------------------------------
MISSED SPEECH = 0.75 secs ( 2.2 percent of scored time)
FALARM SPEECH = 0.00 secs ( 0.0 percent of scored time)
MISSED WORDS = 0 ( 0.0 percent of scored words)
---------------------------------------------
SCORED SPEAKER TIME = 35.30 secs (104.1 percent of scored speech)
MISSED SPEAKER TIME = 2.15 secs ( 6.1 percent of scored speaker time)
FALARM SPEAKER TIME = 0.25 secs ( 0.7 percent of scored speaker time)
SPEAKER ERROR TIME = 0.00 secs ( 0.0 percent of scored speaker time)
SPEAKER ERROR WORDS = 3 ( 14.3 percent of scored speaker words)
---------------------------------------------
OVERALL SPEAKER DIARIZATION ERROR = 6.80 percent of scored speaker time `(ALL)
---------------------------------------------
Speaker type confusion matrix -- speaker weighted
REF\SYS (count) adult_male unknown MISS
adult_male 1 / 33.3% 0 / 0.0% 0 / 0.0%
unknown 0 / 0.0% 2 / 66.7% 0 / 0.0%
FALSE ALARM 0 / 0.0% 0 / 0.0%
---------------------------------------------
Speaker type confusion matrix -- time weighted
REF\SYS (seconds) adult_male unknown MISS
adult_male 28.50 / 80.7% 0.00 / 0.0% 0.00 / 0.0%
unknown 0.00 / 0.0% 4.65 / 13.2% 2.15 / 6.1%
FALSE ALARM 0.00 / 0.0% 0.25 / 0.7%
---------------------------------------------
|