Jean-François Rey / otmedia

Download as
Browse Code »

Commit 52d318a618c7f6210299e11c3ee7d12b638a426c

Authored by Jean-François Rey
1 parent b94490daf7
Exists in master

update doc

Showing 2 changed files with 25 additions and 2 deletions Inline Diff

1 ################# 1 #################
2 # OTMEDIALIA # 2 # OTMEDIALIA #
3 # FAQ # 3 # FAQ #
4 ################# 4 #################
5 5
6 1/ I've got "directory is locked" 6 1\ I've got "directory is locked"
7 --------------------------------- 7 ---------------------------------
8 That happens when a process is working on the show 8 That happens when a process is working on the show
9 or when a pass has not ending correctly. 9 or when a pass has not ending correctly.
10 To force the lock use option "-r" 10 To force the lock use option "-r"
11
12 2\ Where are the log files ?
13 ----------------------------
14 When OTMedia is launch with the option -c, it will create log files.
15 Each show got their own log file in their output directories.
16
17 Info files :
18 info_p1.log
19 info_p2.log
20 info_p3.log
21 info_conf.log
22 info_exploitconf.log
23
24 Error files :
25 error_p1.log
26 error_p2.log
27 error_p3.log
28 error_conf.log
29 error_exploitconf.log
30
31
11 32
12 33
13 34
14 35
1 #---------------# 1 #---------------#
2 # OTMEDIA LIA # 2 # OTMEDIA LIA #
3 # HOWTO # 3 # HOWTO #
4 # version 1.0 # 4 # version 1.0 #
5 #---------------# 5 #---------------#
6 6
7 1\ Main scripts options 7 1\ Main scripts options
8 2\ Main scripts 8 2\ Main scripts
9 2.1\ FirstPass.sh 9 2.1\ FirstPass.sh
10 2.2\ SecondPass.sh 10 2.2\ SecondPass.sh
11 2.3\ ConfPass.sh 11 2.3\ ConfPass.sh
12 2.4\ ExploitConfidencePass.sh 12 2.4\ ExploitConfidencePass.sh
13 2.5\ ThirdPass.sh 13 2.5\ ThirdPass.sh
14 2.6\ RecomposePass.sh 14 2.6\ RecomposePass.sh
15 2.7\ ScoringRes.sh 15 2.7\ ScoringRes.sh
16 2.8\ CheckResults.sh 16 2.8\ CheckResults.sh
17 3\ OneScriptToRuleThemAll.sh 17 3\ OneScriptToRuleThemAll.sh
18 4\ Modify configuration 18 4\ Modify configuration
19 4.1\ Scripts configurations 19 4.1\ Scripts configurations
20 4.2\ Speeral configurations 20 4.2\ Speeral configurations
21 5\ Modify binaries 21 5\ Modify binaries
22 6\ Exemples 22 6\ Exemples
23 23
24 24
25 1\ Main scripts options 25 1\ Main scripts options
26 ----------------------- 26 -----------------------
27 27
28 There are five main options for otmedia scripts. 28 There are five main options for otmedia scripts.
29 -h : for help 29 -h : for help
30 -D : Debug mode 30 -D : Debug mode
31 -v n : Verbose mode 1 low to 3 high 31 -v n : Verbose mode 1 low to 3 high
32 -c : Check results 32 -c : Check results (will check process and create log files)
33 -r : force to rerun a script, without deleting work already done 33 -r : force to rerun a script, without deleting work already done
34 34
35 2\ Main scripts 35 2\ Main scripts
36 --------------- 36 ---------------
37 37
38 Each script got a configuration file in OTMEDIA_HOME/cfg/<scriptname>.cfg . 38 Each script got a configuration file in OTMEDIA_HOME/cfg/<scriptname>.cfg .
39 The main options can be modify individually through the arguments or in the configuration file. 39 The main options can be modify individually through the arguments or in the configuration file.
40 40
41 2.1\ FirstPass.sh 41 2.1\ FirstPass.sh
42 ----------------- 42 -----------------
43 43
44 FirstPass.sh do speaker diarization and transcription of an audio file. Convert it into wav format if not already done (16000Hz, 16 bits, mono). 44 FirstPass.sh do speaker diarization and transcription of an audio file. Convert it into wav format if not already done (16000Hz, 16 bits, mono).
45 If a .SRT file is present in the same directory of the audio file it will copy it. 45 If a .SRT file is present in the same directory of the audio file it will copy it.
46 46
47 $> FisrtPass.sh [options] 110624FR2_20002100.wav result_directory 47 $> FisrtPass.sh [options] 110624FR2_20002100.wav result_directory
48 48
49 Options: 49 Options:
50 -f n : number of forks for speeral 50 -f n : number of forks for speeral
51 51
52 Output : result_directory/110624FR2_20002100/res_p1/ 52 Output : result_directory/110624FR2_20002100/res_p1/
53 and .ctm, .trs and .txt files. 53 and .ctm, .trs and .txt files.
54 54
55 2.2\ SecondPass.sh 55 2.2\ SecondPass.sh
56 ------------------ 56 ------------------
57 57
58 SecondPass.sh do speaker adaptation and transcriptions base on the first pass. 58 SecondPass.sh do speaker adaptation and transcriptions base on the first pass.
59 59
60 $> SecondPass.sh [options] result_directory/110624FR2_20002100/ 60 $> SecondPass.sh [options] result_directory/110624FR2_20002100/
61 61
62 Options: 62 Options:
63 -f n : number of forks for speeral 63 -f n : number of forks for speeral
64 64
65 Output : result_directory/110624FR2_20002100/res_p2/ 65 Output : result_directory/110624FR2_20002100/res_p2/
66 and .ctm, .trs and .txt files. 66 and .ctm, .trs and .txt files.
67 67
68 2.3\ ConfPass.sh 68 2.3\ ConfPass.sh
69 ---------------- 69 ----------------
70 70
71 ConfPass.sh do confidence measure using the second or third pass. 71 ConfPass.sh do confidence measure using the second or third pass.
72 72
73 $> Confpass.sh [options] result_directory/110624FR2_20002100/ <res_p2|res_p3> 73 $> Confpass.sh [options] result_directory/110624FR2_20002100/ <res_p2|res_p3>
74 74
75 Output : result_directory/110624FR2_20002100/conf/res_p2/scored_ctm/ 75 Output : result_directory/110624FR2_20002100/conf/res_p2/scored_ctm/
76 and result_directory/110624FR2_20002100.usf file 76 and result_directory/110624FR2_20002100.usf file
77 77
78 2.4\ ExploitConfidencePass.sh 78 2.4\ ExploitConfidencePass.sh
79 ----------------------------- 79 -----------------------------
80 80
81 It exploits confidence pass measure to : 81 It exploits confidence pass measure to :
82 - boost confidente zone 82 - boost confidente zone
83 - find alternative in non confidente zone (using SOLR DB) 83 - find alternative in non confidente zone (using SOLR DB)
84 - extend the lexicon 84 - extend the lexicon
85 85
86 $> ExploitConfidencePass.sh [options] result_directory/110624FR2_20002100 86 $> ExploitConfidencePass.sh [options] result_directory/110624FR2_20002100
87 87
88 Output : result_directory/110624FR2_20002100/trigg/speeral 88 Output : result_directory/110624FR2_20002100/trigg/speeral
89 result_directory/110624FR2_20002100/LEX/speeral/_ext 89 result_directory/110624FR2_20002100/LEX/speeral/_ext
90 90
91 2.5\ ThirdPass.sh 91 2.5\ ThirdPass.sh
92 ------------------ 92 ------------------
93 93
94 ThirdPass.sh do transcriptions using SecondPass speaker adaptation and ExploitConfidencePass trigg files and new lexicon. 94 ThirdPass.sh do transcriptions using SecondPass speaker adaptation and ExploitConfidencePass trigg files and new lexicon.
95 95
96 $> ThirdPass.sh [options] result_directory/110624FR2_20002100/ 96 $> ThirdPass.sh [options] result_directory/110624FR2_20002100/
97 97
98 Options : 98 Options :
99 -f n : number of forks for speeral 99 -f n : number of forks for speeral
100 100
101 Output : result_directory/110624FR2_20002100/conf/res_p3 101 Output : result_directory/110624FR2_20002100/conf/res_p3
102 and .ctm, .trs and .txt files. 102 and .ctm, .trs and .txt files.
103 103
104 2.6\ RecomposePass.sh 104 2.6\ RecomposePass.sh
105 -------------------- 105 --------------------
106 106
107 RecomposePass.sh copy results that missing in ThirsPass from the Second and First Pass. 107 RecomposePass.sh copy results that missing in ThirsPass from the Second and First Pass.
108 108
109 $> RecomposePass.sh [options] result_directory/110624FR2_20002100/ 109 $> RecomposePass.sh [options] result_directory/110624FR2_20002100/
110 110
111 Output : result_directory/110624FR2_20002100/res_all 111 Output : result_directory/110624FR2_20002100/res_all
112 and .ctm, .trs and .txt files. 112 and .ctm, .trs and .txt files.
113 113
114 2.7\ ScoringRes.sh 114 2.7\ ScoringRes.sh
115 ------------------ 115 ------------------
116 116
117 ScoringRes.sh run differents scoring tools to score the results using SRT file if exists. 117 ScoringRes.sh run differents scoring tools to score the results using SRT file if exists.
118 118
119 $> ScoringRes.sh [options] result_directory/110624FR2_20002100/ 119 $> ScoringRes.sh [options] result_directory/110624FR2_20002100/
120 120
121 Output : result_directory/110624FR2_20002100/scoring 121 Output : result_directory/110624FR2_20002100/scoring
122 122
123 2.8\ CheckResults.sh 123 2.8\ CheckResults.sh
124 -------------------- 124 --------------------
125 125
126 CheckResults.sh parse results directories to synthesize works already done. 126 CheckResults.sh parse results directories to synthesize works already done.
127 127
128 $> CheckResults.sh [options] result_directory 128 $> CheckResults.sh [options] result_directory
129 129
130 Output : "Directory name #plp #res_p1 #treil_p2 #treil_p3 usf_p2 usf_p3" 130 Output : "Directory name #plp #res_p1 #treil_p2 #treil_p3 usf_p2 usf_p3"
131 #plp number of plp files 131 #plp number of plp files
132 #res_p1 number of .res files at first pass 132 #res_p1 number of .res files at first pass
133 #treil_p2 number of .treil files at second pass 133 #treil_p2 number of .treil files at second pass
134 #treil_p3 number of .treil files at third pass 134 #treil_p3 number of .treil files at third pass
135 usf_p2 usf file from confidence pass result on second pass (OK|ERR|NAN) 135 usf_p2 usf file from confidence pass result on second pass (OK|ERR|NAN)
136 usf_p3 usf file from confidence pass result on third pass (OK|ERR|NAN) 136 usf_p3 usf file from confidence pass result on third pass (OK|ERR|NAN)
137 137
138 3\ OneScriptToRuleThemAll.sh 138 3\ OneScriptToRuleThemAll.sh
139 ---------------------------- 139 ----------------------------
140 140
141 The script to do all OTMEDIA LIA pass in one call. 141 The script to do all OTMEDIA LIA pass in one call.
142 142
143 $> OneScriptToRuleThemAll.sh [options] 110624FR2_20002100.wav result_directory 143 $> OneScriptToRuleThemAll.sh [options] 110624FR2_20002100.wav result_directory
144 144
145 Options : (default options are availables) 145 Options : (default options are availables)
146 -a Do every pass 146 -a Do every pass
147 -1 Do First pass 147 -1 Do First pass
148 -2 Do Second pass 148 -2 Do Second pass
149 -3 Do Third pass 149 -3 Do Third pass
150 -C Do Confidence pass 150 -C Do Confidence pass
151 -e Do Exploit Confidence pass 151 -e Do Exploit Confidence pass
152 -R Do Recompose pass 152 -R Do Recompose pass
153 -s Do Scoring pass 153 -s Do Scoring pass
154 154
155 4\ Modify configuration 155 4\ Modify configuration
156 ----------------------- 156 -----------------------
157 157
158 Most of the main scripts got a configuration file (cfg/ directory). 158 Most of the main scripts got a configuration file (cfg/ directory).
159 You can change script behaviour and data used. 159 You can change script behaviour and data used.
160 Speeral configuration file can be also change (tools/Speeral/CFG/ directory) 160 Speeral configuration file can be also change (tools/Speeral/CFG/ directory)
161 161
162 4.1\ Scripts configurations 162 4.1\ Scripts configurations
163 --------------------------- 163 ---------------------------
164 164
165 In scripts configuration files (OTMEDIA_HOME/cfg/) you can change default options as architecture, verbose ... 165 In scripts configuration files (OTMEDIA_HOME/cfg/) you can change default options as architecture, verbose ...
166 Scripts using Speeral got information on binaries, models path and name, and the configuration file for speeral. 166 Scripts using Speeral got information on binaries, models path and name, and the configuration file for speeral.
167 167
168 4.2\ Speeral configurations 168 4.2\ Speeral configurations
169 --------------------------- 169 ---------------------------
170 170
171 Speeral configuration files are in OTMEDIA_HOME/tools/Speeral/CFG directory. 171 Speeral configuration files are in OTMEDIA_HOME/tools/Speeral/CFG directory.
172 The .tmp files are use to generate .xml file from install.sh. 172 The .tmp files are use to generate .xml file from install.sh.
173 You can modify .xml files for your needs, but most of data informations are pass through arguments at speeral call in scripts. 173 You can modify .xml files for your needs, but most of data informations are pass through arguments at speeral call in scripts.
174 174
175 5\ Modify binaries 175 5\ Modify binaries
176 ------------------ 176 ------------------
177 177
178 Binaries can be find in bin and tools directory. 178 Binaries can be find in bin and tools directory.
179 Some binaries are compiled in 32 and 64 bits. By default all binaries are compiled in 32 bits. 179 Some binaries are compiled in 32 and 64 bits. By default all binaries are compiled in 32 bits.
180 You can update binaries as you need. 180 You can update binaries as you need.
181 181
182 To modify tools binaries, you need to download a compatible version. 182 To modify tools binaries, you need to download a compatible version.
183 lia_ltbox can be found in /labo/Tools/ 183 lia_ltbox can be found in /labo/Tools/
184 Speeral (binaries) can be compiled from the git remote git@gitlia.univ-avignon.fr:vaudriguard/libspeeral.git . Do not modify Speeral data from OTMEDIA (unless you know what you do). 184 Speeral (binaries) can be compiled from the git remote git@gitlia.univ-avignon.fr:vaudriguard/libspeeral.git . Do not modify Speeral data from OTMEDIA (unless you know what you do).
185 In PACKAGES_MESURES_V1.0 you can update icsiboost binary (in bin) from the projet page : https://code.google.com/p/icsiboost/ 185 In PACKAGES_MESURES_V1.0 you can update icsiboost binary (in bin) from the projet page : https://code.google.com/p/icsiboost/
186 For QUOTE_FINDER and SIGMUND please contact support. 186 For QUOTE_FINDER and SIGMUND please contact support.
187 187
188 6\ Exemples 188 6\ Exemples
189 ----------- 189 -----------
190 190
191 Conventional use : 191 Conventional use :
192 $> FirstPass.sh 110624FR2_20002100.wav /my/output/directory/ && SecondPass.sh /my/output/directory/110624FR2_20002100 && ConfPass.sh /my/output/directory/110624FR2_20002100 res_p2 && ExploitConfPass.sh /my/output/directory/110624FR2_20002100 && ConfPass.sh /my/output/directory/110624FR2_20002100 res_p3 && ThirdPass.sh /my/output/directory/110624FR2_20002100 && RecomposePass.sh /my/output/directory/110624FR2_20002100 && ScoringRes.sh /my/output/directory/110624FR2_20002100 192 $> FirstPass.sh 110624FR2_20002100.wav /my/output/directory/ && SecondPass.sh /my/output/directory/110624FR2_20002100 && ConfPass.sh /my/output/directory/110624FR2_20002100 res_p2 && ExploitConfPass.sh /my/output/directory/110624FR2_20002100 && ConfPass.sh /my/output/directory/110624FR2_20002100 res_p3 && ThirdPass.sh /my/output/directory/110624FR2_20002100 && RecomposePass.sh /my/output/directory/110624FR2_20002100 && ScoringRes.sh /my/output/directory/110624FR2_20002100
193 or 193 or
194 $> OneScriptToRuleThemAll.sh -a 110624FR2_20002100.wav /my/output/directory/ 194 $> OneScriptToRuleThemAll.sh -a 110624FR2_20002100.wav /my/output/directory/
195 195
196 Rerun SecondPass and ConfPass with verbose and speeral fork to 4 : 196 Rerun SecondPass and ConfPass with verbose and speeral fork to 4 :
197 $> SecondPass.sh -r -f 4 -v 3 /my/output/directory/110624FR2_20002100 && ConfPass.sh -r -v 3 /my/output/directory/110624FR2_20002100 res_p2 197 $> SecondPass.sh -r -f 4 -v 3 /my/output/directory/110624FR2_20002100 && ConfPass.sh -r -v 3 /my/output/directory/110624FR2_20002100 res_p2
198 or 198 or
199 $> OneScriptToRuleThemAll.sh -r -2 -C 2 -v 3 -f 4 110624FR2_20002100.wav /my/output/directory/ 199 $> OneScriptToRuleThemAll.sh -r -2 -C 2 -v 3 -f 4 110624FR2_20002100.wav /my/output/directory/
200 200
201 Run with log file
202 $> OneScriptToRuleThemAll.sh -a -c -v3 -f4 110624FR2_20002100.wav /my/output/directory/
201 203
202 204
203 Good Luck ! Luke ! 205 Good Luck ! Luke !
204 And the force be with you ! 206 And the force be with you !
205 207