Jean-François Rey / otmedia

Download as
Browse Code »

Commit 5a14b86787e34200360c3f5d4ef4616d7fceccf3

Authored by Jean-François Rey
1 parent c00c2f1895
Exists in master

update doc

Showing 5 changed files with 60 additions and 14 deletions Side-by-side Diff

... ... @@ -28,6 +28,7 @@
28 28 -f n : number of forks for speeral
29 29  
30 30 Output : result_directory/110624FR2_20002100/res_p1/
  31 + and .ctm, .trs and .txt files.
31 32  
32 33 2.2\ SecondPass.sh
33 34 ------------------
... ... @@ -40,6 +41,7 @@
40 41 -f n : number of forks for speeral
41 42  
42 43 Output : result_directory/110624FR2_20002100/res_p2/
  44 + and .ctm, .trs and .txt files.
43 45  
44 46 2.3\ ConfPass.sh
45 47 ----------------
... ... @@ -75,6 +77,7 @@
75 77 -f n : number of forks for speeral
76 78  
77 79 Output : result_directory/110624FR2_20002100/conf/res_p3
  80 + and .ctm, .trs and .txt files.
78 81  
79 82 2.6\ RecomposePass.sh
80 83 --------------------
... ... @@ -84,6 +87,7 @@
84 87 $> RecomposePass.sh [options] result_directory/110624FR2_20002100/
85 88  
86 89 Output : result_directory/110624FR2_20002100/res_all
  90 + and .ctm, .trs and .txt files.
87 91  
88 92 2.7\ ScoringRes.sh
89 93 ------------------
90 94  
91 95  
92 96  
93 97  
... ... @@ -127,9 +131,36 @@
127 131 -s Do Scoring pass
128 132  
129 133 4\ Modify configuration
  134 +-----------------------
130 135  
  136 + Most of the main scripts got a configuration file (cfg/ directory).
  137 + You can change script behaviour and data used.
  138 + Speeral configuration file can be also change (tools/Speeral/CFG/ directory)
  139 +
131 140 4.1\ Scripts configurations
  141 +
  142 + In scripts configuration files (OTMEDIA_HOME/cfg/) you can change default options as architecture, verbose ...
  143 + Scripts using Speeral got information on binaries, models path and name, and the configuration file for speeral.
  144 +
132 145 4.2\ Speeral configurations
133 146  
  147 + Speeral configuration files are in OTMEDIA_HOME/tools/Speeral/CFG directory.
  148 + The .tmp files are use to generate .xml file from install.sh.
  149 + You can modify .xml files for your needs, but most of data informations are pass through arguments at speeral call in scripts.
  150 +
134 151 5\ Modify binaries
  152 +------------------
  153 +
  154 + Binaries can be find in bin and tools directory.
  155 + Some binaries are compiled in 32 and 64 bits. By default all binaries are compiled in 32 bits.
  156 + You can update binaries as you need.
  157 +
  158 + To modify tools binaries, you need to download a compatible version.
  159 + lia_ltbox can be found in /labo/Tools/
  160 + Speeral (binaries) can be compiled from the git remote git@gitlia.univ-avignon.fr:vaudriguard/libspeeral.git . Do not modify Speeral data from OTMEDIA (unless you know what you do).
  161 + In PACKAGES_MESURES_V1.0 you can update icsiboost binary (in bin) from the projet page : https://code.google.com/p/icsiboost/
  162 + For QUOTE_FINDER and SIGMUND please contact support.
  163 +
  164 +Good Luck ! Luke !
  165 +And the force be with you !
... ... @@ -14,6 +14,7 @@
14 14 1\ Before installation
15 15 2\ install.sh script
16 16 3\ SOLR install
  17 +4\ Install descriptions
17 18  
18 19  
19 20 1\ Before installation
... ... @@ -24,7 +25,7 @@
24 25 - Have 300 Go of free space.
25 26 - Have acces to the network and the nyx server.
26 27  
27   -2/ install.sh script
  28 +2\ install.sh script
28 29 --------------------
29 30  
30 31 install.sh script will do most of the work.
... ... @@ -37,6 +38,8 @@
37 38 To disable confidence measure put CONFPASS to 0.
38 39 To disable second and third pass put PASS2 to 0.
39 40  
  41 +If your login name differ from your nyx login name, edit install.sh and change the username variable to your nyx login name.
  42 +
40 43 run install.sh and follow the white rabbit.
41 44  
42 45 3\ SOLR install
... ... @@ -46,6 +49,7 @@
46 49 See SOLR.INSTALL file to install OTMEDIA SOLR DB.
47 50  
48 51 4\ Install descriptions
  52 +-----------------------
49 53  
50 54 OTMEDIA_HOME
51 55 |-> bin/
... ... @@ -55,7 +59,7 @@
55 59 |-> lia_plp_mt.64
56 60 |-> LIUM_SpkDiarization-4.2.jar
57 61 |-> sclite
58   - |-> cfg/
  62 + |-> cfg/ # Main scripts configurations files
59 63 |-> ConfidenceMeasure.cfg
60 64 |-> ConfPass.cfg
61 65 |-> ExploitConfidencePass.cfg
... ... @@ -65,7 +69,7 @@
65 69 |-> Scoring.cfg
66 70 |-> Secondass.cfg
67 71 |-> ThirdPass.cfg
68   - |-> data/
  72 + |-> data/ # Some data
69 73 |-> rules/
70 74 |-> asupp
71 75 |-> basic
... ... @@ -75,7 +79,7 @@
75 79 |-> postprocess.regex
76 80 |-> preprocess.regex
77 81 |-> random_regex.tab
78   - |-> main_tools/
  82 + |-> main_tools/ # Main scripts
79 83 |-> CheckResults.sh
80 84 |-> ConfidenceMeasure.sh
81 85 |-> ConfPass.sh
82 86  
... ... @@ -86,11 +90,11 @@
86 90 |-> ScoringRes.sh
87 91 |-> SecondPass.sh
88 92 |-> ThirdPass.sh
89   - |-> tools/
  93 + |-> tools/ # Tools
90 94 |-> lia_ltbox/
91 95 |-> PACKAGE_MESURES_V1.0/
92 96 |-> QUOTE_FINDER/
93   - |-> scripts/
  97 + |-> scripts/ # Secondary scripts (but useful)
94 98 |-> ApplyCorrectionRules.pl
95 99 |-> BdlexUC.pl
96 100 |-> CheckConfPass.sh
... ... @@ -127,6 +131,8 @@
127 131 |-> Tools.sh
128 132 |-> UrlConverter.pl
129 133 |-> SIGMUND/
  134 + |-> SOLR/
  135 + |-> Speeral/
130 136 |-> COPYING
131 137 |-> CorpusOTMedia.txt
132 138 |-> HOWTO
... ... @@ -28,7 +28,8 @@
28 28 - Confidence pass : calcul confidence measure from transcription output.
29 29 - Exploit Confidence Measure : use SOLR DB data to extend the lexicon on low confidence measure and create trigg files.
30 30 - Third pass : second pass using the new lexicon and trigg files.
31   -
  31 +
  32 + From GIT : git@gitlia.univ-avignon.fr/otmedia.git
32 33  
33 34 DEPENDENCIES
34 35 ------------
... ... @@ -3,9 +3,10 @@
3 3 #-------------------#
4 4 # OTMEDIA LIA #
5 5 # Install script #
6   -# version : 1.0.0 #
  6 +# version : 1.1.0 #
7 7 #-------------------#
8 8  
  9 +# nyx login name
9 10 username=${LOGNAME}
10 11  
11 12 # Color variables
... ... @@ -143,8 +144,12 @@
143 144 ### Speeral Configuration ###
144 145  
145 146 echo -e "\n\t${txtblu}Speeral configuration${txtrst}\n"
146   -echo -e "Download Speeral bin and data :"
147   -scp -r ${username}@nyx:~/OTMEDIA_DATA/Speeral $OTMEDIA_HOME/tools/
  147 +read -e -p "Download Speeral bin and data ? (y/n) " speeral
  148 +if [ "$speeral" == "y" ]
  149 +then
  150 + echo -e "Download Speeral bin and data :"
  151 + scp -r ${username}@nyx:/local/OTMEDIA/OTMEDIA_DATA/Speeral $OTMEDIA_HOME/tools/
  152 +fi
148 153 echo -e "\n\t${txtblu}Generating Speeral configuration files :${txtrst}\n"
149 154 cat $PWD/tools/Speeral/CFG/SpeeralFirstPass.xml.tmp | sed -e "s|<nom>[^<]*</nom>|<nom>$PWD/tools/Speeral/LEX/LEXIQUE_V6.speer</nom>|g" \
150 155 | sed -e "s|<ngramme>[^<]*</ngramme>|<ngramme>$PWD/tools/Speeral/LM/ML_4gOTMEDIA_LEXIQUE_V6</ngramme>|g" \
... ... @@ -204,7 +209,7 @@
204 209  
205 210 echo -e "Download SOLR DB\r"
206 211 mkdir -p $OTMEDIA_HOME/tools/SOLR 2> /dev/null
207   - scp -r ${username}@nyx:~/OTMEDIA_DATA/SOLR/otmedia-2013-04.tar.gz $OTMEDIA_HOME/tools/SOLR
  212 + scp -r ${username}@nyx:/local/OTMEDIA/OTMEDIA_DATA/SOLR/otmedia-2013-04.tar.gz $OTMEDIA_HOME/tools/SOLR
208 213 echo -e "Unzip SOLR DB\r"
209 214 res=0
210 215 #res = $(tar -xvzf "$OTMEDIA_HOME/tools/SOLR/otmedia-2013-04.tar.gz" "$OTMEDIA_HOME/tools/SOLR/")
211 216  
... ... @@ -259,10 +264,10 @@
259 264 echo "export SRILM_BIN=$SRILM/bin/$MACHINE_TYPE"
260 265  
261 266  
262   -echo " \\ "
  267 +echo "${txtwht} \\ "
263 268 echo " ,-~~~-\\\\_"
264 269 echo " ( .\ "
265   -echo " @\___(__--'"
  270 +echo " @\___(__--'${txtrst}"
266 271  
267 272 echo "${txtgrn}Yes${txtylw}I${txtred}Rastafari${txtrst}"
main_tools/ConfidenceMeasure.sh
Diff comments   View file @ 5a14b86
... ... @@ -102,8 +102,11 @@
102 102 mkdir -p $SEGCTM > /dev/null 2>&1
103 103 mkdir -p $SUPER_CTM > /dev/null 2>&1
104 104 mkdir -p $SCORED_CTM > /dev/null 2>&1
105   -if [ -z $BOOST_BIN ];then
  105 +if [ -z $BOOST_BIN ] && [ $ARCH == ".64"] ;then
106 106 BOOST_BIN=$ROOT/bin/icsiboost-64bit-static-r160
  107 +fi
  108 +if [ -z $BOOST_BIN ] ;then
  109 + BOOST_BIN=$ROOT/bin/icsiboost-32bit-static-r176
107 110 fi
108 111 #-----------------------------------------------------------------
109 112 # STEP 1 - Extension des treillis + ajout posteriors (format htk)