Sc_stats Commandline Options

The commandline options for sc_stats can be broken into four categories:

1. Input File Options:
   -p,
2. Output Options:
   -e, -n, -O,
3. Report Generation Options:
   -r
4. Statistical Test Options:
   -t -v -u -g

Input File Options:

These options control/define the input to sc_stats. Input must come from stdin and the -p option must be used. (Forcing the user to use the -p option enables future expandability while maintaining backward compatability.)

-p
Alignments are read from 'stdin' as input to sc_stats. The format of the input must be in the "sgml" output format, created either by '-o sgml' or by piped input from another sctk utility.

-e desc
Description of the ensemble of hyp files.
-O output_dir
Writes all output files into output_dir. Defaults to the hypfile's directory
-n name
Writes all multiple hypothesis file reports to files beginning with 'name'. Using '-' writes to stdout. Default: 'Ensemble'

-g [ range | grange | grange2 ]
Generate per speaker range graphs, based on the formula defined by '-f'. The reports are written to files whose root name begins with the values defined by '-n'. There are two graphs produced, one showing speaker performance variability across systems and the second showing system performance variablity for across speakers.

- The 'range' graphs are an ASCII representation of the of the variablity in error rates for a given speaker. The graph is sorted be the mean of statistic computed for each speaker. EXAMPLE

- The 'grange' graph is a gnuplot version of the same data ploted in 'range. There are two sets of files created. The first set, which is called '*.grange.spk.plt' and '*.grange.spk.dat', contains the gnuplot command files and data files respectively for the speaker performance variability across systems graph. The second set, which is called '*.grange.sys.plt' and '*.grange.sys.dat', contains the gnuplot command files and data files respectively for the system performance variability across speakers graph. EXAMPLE

- The 'grange2' graph is similar to the 'grange' graph except that each systems speaker word error scores are identified by a unique symbol. EXAMPLE


-r [ sum | rsum | lur | es | res | none ]

prn -

Example

sum -

Example

rsum -

Example

lur -

Example

es -

Example

res -

Example

none -

Produce no output reports, Default.

-t [ mcn | mapsswe | sign | wilc | anovar | std4 ]

mcn -

Perform the McNemar Test.

mapsswe -

Perform the Matched Pairs Sentence Segment Word Error Test

sign -

Perform the Sign Test

wilc -

Perform the Wilcoxon Signed Rank Test

anovar -

Perform the Analysis of Variance by Rank Test

std -

This is a shorthand notation to do the 'standard' four tests: mcn, mapsswe, wilc and sign.


-v
For each test performed on a pair of systems files, output a detailed analysis.

-u
Rather than creating a comparison matrix for each test, unify statistical test results into a single comparision matrix

-f [ E | R | W ]
Use the identified formula for statistical tests: sign, wilcoxon and anovar tests. The formulas are:
1. E -> Percentage Word Error
2. R -> Percentage Words Correctly Recognized
3. E -> Percentage Word Accuracy
By default 'E'