Compare View

switch
from
...
to
 
Commits (2)

Changes

Showing 4 changed files Inline Diff

1 #---------------# 1 #---------------#
2 # OTMEDIA LIA # 2 # OTMEDIA LIA #
3 # INSTALL # 3 # INSTALL #
4 # version : 1.0 # 4 # version : 1.0 #
5 #---------------# 5 #---------------#
6 6
7 OTMEDIA LIA ready to use ? Really ? 7 OTMEDIA LIA ready to use ? Really ?
8 No ! You have to do manualy configuartion for some features. 8 No ! You have to do manualy configuartion for some features.
9 Let see... 9 Let see...
10 10
11 SUMMARY 11 SUMMARY
12 ------- 12 -------
13 13
14 1\ Before installation 14 1\ Before installation
15 2\ install.sh script 15 2\ install.sh script
16 3\ SOLR install 16 3\ SOLR install
17 4\ Install descriptions 17 4\ Install descriptions
18 18
19 19
20 1\ Before installation 20 1\ Before installation
21 ---------------------- 21 ----------------------
22 22
23 - Check and install dependencies. 23 - Check and install dependencies.
24 - In 64 bits architcture be sure you can run 32 bits programs. 24 - In 64 bits architcture be sure you can run 32 bits programs.
25 - Have 300 Go of free space. 25 - Have 300 Go of free space.
26 - Have acces to the network and the nyx server. 26 - Have acces to the network and the nyx server.
27 27
28 2\ install.sh script 28 2\ install.sh script
29 -------------------- 29 --------------------
30 30
31 install.sh script will do most of the work. 31 install.sh script will do most of the work.
32 It will check dependencies and configure pass tools. 32 It will check dependencies and configure pass tools.
33 By default it will do a complet install (300 Go). 33 By default it will do a complet install (300 Go).
34 34
35 You can modifiy behavior by editing install.sh : 35 You can modifiy behavior by editing install.sh :
36 36
37 To disable lexicon adaption using SOLR DB put EXPLOITCONFPASS to 0 (mainly the 290 Go). 37 To disable lexicon adaption using SOLR DB put EXPLOITCONFPASS to 0 (mainly the 290 Go).
38 To disable confidence measure put CONFPASS to 0. 38 To disable confidence measure put CONFPASS to 0.
39 To disable second and third pass put PASS2 to 0. 39 To disable second and third pass put PASS2 to 0.
40 To disable Scoring compilation tools put SCORING to 0.
40 To disable Scoring compilation tools put SCORING to 0. 41
41 42 If your login name differ from your nyx login name, edit install.sh and change the username variable to your nyx login name.
42 If your login name differ from your nyx login name, edit install.sh and change the username variable to your nyx login name. 43
43 44 run install.sh and follow the white rabbit.
44 run install.sh and follow the white rabbit. 45
45 46 3\ SOLR install
46 3\ SOLR install 47 ---------------
47 --------------- 48
48 49 The install.sh script download otmedia-2013-04.tar.gz and untar it in OTMEDIA_HOME/tools/SOLR/ .
49 The install.sh script download otmedia-2013-04.tar.gz and untar it in OTMEDIA_HOME/tools/SOLR/ . 50 See SOLR.INSTALL file to install OTMEDIA SOLR DB.
50 See SOLR.INSTALL file to install OTMEDIA SOLR DB. 51
51 52 4\ Install descriptions
52 4\ Install descriptions 53 -----------------------
53 ----------------------- 54
54 55 OTMEDIA_HOME
55 OTMEDIA_HOME 56 |-> bin/
56 |-> bin/ 57 |-> aff_mat
57 |-> aff_mat 58 |-> aff_mat.64
58 |-> aff_mat.64 59 |-> lia_plp_mt
59 |-> lia_plp_mt 60 |-> lia_plp_mt.64
60 |-> lia_plp_mt.64 61 |-> LIUM_SpkDiarization-4.2.jar
61 |-> LIUM_SpkDiarization-4.2.jar 62 |-> sclite
62 |-> sclite 63 |-> cfg/ # Main scripts configurations files
63 |-> cfg/ # Main scripts configurations files 64 |-> ConfidenceMeasure.cfg
64 |-> ConfidenceMeasure.cfg 65 |-> ConfPass.cfg
65 |-> ConfPass.cfg 66 |-> ExploitConfidencePass.cfg
66 |-> ExploitConfidencePass.cfg 67 |-> FirstPass.cfg
67 |-> FirstPass.cfg 68 |-> main_cfg.cfg
68 |-> main_cfg.cfg 69 |-> RecomposePass.cfg
69 |-> RecomposePass.cfg 70 |-> Scoring.cfg
70 |-> Scoring.cfg 71 |-> Secondass.cfg
71 |-> Secondass.cfg 72 |-> ThirdPass.cfg
72 |-> ThirdPass.cfg 73 |-> data/ # Some data
73 |-> data/ # Some data 74 |-> rules/
74 |-> rules/ 75 |-> asupp
75 |-> asupp 76 |-> basic
76 |-> basic 77 |-> lastprocess.regex
77 |-> lastprocess.regex 78 |-> muRules.tab
78 |-> muRules.tab 79 |-> numeric_rules
79 |-> numeric_rules 80 |-> postprocess.regex
80 |-> postprocess.regex 81 |-> preprocess.regex
81 |-> preprocess.regex 82 |-> random_regex.tab
82 |-> random_regex.tab 83 |-> main_tools/ # Main scripts
83 |-> main_tools/ # Main scripts 84 |-> CheckResults.sh
84 |-> CheckResults.sh 85 |-> ConfidenceMeasure.sh
85 |-> ConfidenceMeasure.sh 86 |-> ConfPass.sh
86 |-> ConfPass.sh 87 |-> ExploitConfidencePass.sh
87 |-> ExploitConfidencePass.sh 88 |-> FirstPass.sh
88 |-> FirstPass.sh 89 |-> OneScriptToRuleThemAll.sh
89 |-> OneScriptToRuleThemAll.sh 90 |-> RecomposePass.sh
90 |-> RecomposePass.sh 91 |-> ScoringRes.sh
91 |-> ScoringRes.sh 92 |-> SecondPass.sh
92 |-> SecondPass.sh 93 |-> ThirdPass.sh
93 |-> ThirdPass.sh 94 |-> tools/ # Tools
95 |-> CRF++-0.58.tar.gz
94 |-> tools/ # Tools 96 |-> lia_ltbox/
95 |-> CRF++-0.58.tar.gz 97 |-> PACKAGE_MESURES_V1.0/
96 |-> lia_ltbox/ 98 |-> QUOTE_FINDER/
97 |-> PACKAGE_MESURES_V1.0/ 99 |-> scripts/ # Secondary scripts (but useful)
98 |-> QUOTE_FINDER/ 100 |-> ApplyCorrectionRules.pl
99 |-> scripts/ # Secondary scripts (but useful) 101 |-> BdlexUC.pl
100 |-> ApplyCorrectionRules.pl 102 |-> CheckConfPass.sh
101 |-> BdlexUC.pl 103 |-> CheckExploitConfPass.sh
102 |-> CheckConfPass.sh 104 |-> CheckFirstPass.sh
103 |-> CheckExploitConfPass.sh 105 |-> CheckSecondPass.sh
104 |-> CheckFirstPass.sh 106 |-> CheckThirdPass.sh
105 |-> CheckSecondPass.sh 107 |-> CleanFilter.sh
106 |-> CheckThirdPass.sh 108 |-> CoverageReportMaker.pl
107 |-> CleanFilter.sh 109 |-> ctm2show.pl
108 |-> CoverageReportMaker.pl 110 |-> Date2txt.pl
109 |-> ctm2show.pl 111 |-> daybefore2after.sh
110 |-> Date2txt.pl 112 |-> ExtractAudioFromTV.sh
111 |-> daybefore2after.sh 113 |-> FindNormRules.pl
112 |-> ExtractAudioFromTV.sh 114 |-> formatRES.pl
113 |-> FindNormRules.pl 115 |-> GenerateSOLRQueries.pl
114 |-> formatRES.pl 116 |-> intersec.pl
115 |-> GenerateSOLRQueries.pl 117 |-> KeepConfZone.pl
116 |-> intersec.pl 118 |-> LexPhonFilter.pl
117 |-> KeepConfZone.pl 119 |-> MergeLexicon.pl
118 |-> LexPhonFilter.pl 120 |-> NbMaxWordsFilter.pl
119 |-> MergeLexicon.pl 121 |-> Number2txt.pl
120 |-> NbMaxWordsFilter.pl 122 |-> perlmod/
121 |-> Number2txt.pl 123 |-> Utils.pm
122 |-> perlmod/ 124 |-> PhonFormatter.pl
123 |-> Utils.pm 125 |-> ProcessSOLRQueries.py
124 |-> PhonFormatter.pl 126 |-> RandomRegex.pl
125 |-> ProcessSOLRQueries.py 127 |-> RemoveLineContaining.pl
126 |-> RandomRegex.pl 128 |-> res2out.pl
127 |-> RemoveLineContaining.pl 129 |-> ScoreCtm2trigg.pl
128 |-> res2out.pl 130 |-> scoredCtmAndTaggedLem2All.pl
129 |-> ScoreCtm2trigg.pl 131 |-> Sentencer.pl
130 |-> scoredCtmAndTaggedLem2All.pl 132 |-> srt2stm.pl
131 |-> Sentencer.pl 133 |-> Tools.sh
132 |-> srt2stm.pl 134 |-> UrlConverter.pl
133 |-> Tools.sh 135 |-> SIGMUND/
134 |-> UrlConverter.pl 136 |-> SOLR/
135 |-> SIGMUND/ 137 |-> Speeral/
136 |-> SOLR/ 138 |-> COPYING
137 |-> Speeral/ 139 |-> CorpusOTMedia.txt
138 |-> COPYING 140 |-> HOWTO
139 |-> CorpusOTMedia.txt 141 |-> INSTALL
140 |-> HOWTO 142 |-> README
141 |-> INSTALL 143 |-> SOLR.INSTALL
142 |-> README 144 |-> TODO
143 |-> SOLR.INSTALL 145
144 |-> TODO 146
145 147
146 148
147 149
148 150
149 151
150 152
151 153
152 154
153 155
154 156
155 157
1 #!/bin/bash 1 #!/bin/bash
2 2
3 #-------------------# 3 #-------------------#
4 # OTMEDIA LIA # 4 # OTMEDIA LIA #
5 # Install script # 5 # Install script #
6 # version : 1.1.0 # 6 # version : 1.1.0 #
7 #-------------------# 7 #-------------------#
8 8
9 # nyx login name 9 # nyx login name
10 username=${LOGNAME} 10 username=${LOGNAME}
11 11
12 # Color variables 12 # Color variables
13 txtred=$(tput setaf 1) # red 13 txtred=$(tput setaf 1) # red
14 txtgrn=$(tput setaf 2) # Green 14 txtgrn=$(tput setaf 2) # Green
15 txtylw=$(tput setaf 3) # Yellow 15 txtylw=$(tput setaf 3) # Yellow
16 txtblu=$(tput setaf 4) # Blue 16 txtblu=$(tput setaf 4) # Blue
17 txtred=$(tput setaf 5) # Purple 17 txtred=$(tput setaf 5) # Purple
18 txtcyn=$(tput setaf 6) # Cyan 18 txtcyn=$(tput setaf 6) # Cyan
19 txtwht=$(tput setaf 7) # White 19 txtwht=$(tput setaf 7) # White
20 txtrst=$(tput sgr0) # Text reset. 20 txtrst=$(tput sgr0) # Text reset.
21 #/color 21 #/color
22 22
23 # 23 #
24 ### Global Variables 24 ### Global Variables
25 # 25 #
26 PWD=$(pwd) 26 PWD=$(pwd)
27 OTMEDIA_HOME=$PWD 27 OTMEDIA_HOME=$PWD
28 test=$(arch) 28 test=$(arch)
29 if [ "$test" == "x86_64" ]; then ARCH=".64"; else ARCH=""; fi 29 if [ "$test" == "x86_64" ]; then ARCH=".64"; else ARCH=""; fi
30 #/Global 30 #/Global
31 31
32 32
33 # 33 #
34 # Put to 0 to disable dependencies of a pass 34 # Put to 0 to disable dependencies of a pass
35 # and 1 to enable 35 # and 1 to enable
36 # 36 #
37 PASS1=1 # First Pass 37 PASS1=1 # First Pass
38 PASS2=1 # Second and Third Pass 38 PASS2=1 # Second and Third Pass
39 CONFPASS=1 # Confidence Pass 39 CONFPASS=1 # Confidence Pass
40 EXPLOITCONFPASS=1 # SOLR query and trigg 40 EXPLOITCONFPASS=1 # SOLR query and trigg
41 SCORING=1 # Compile scoring tools (crf++) 41 SCORING=1 # Compile scoring tools (crf++)
42 42
43 echo -e "\nWill do install for :" 43 echo -e "\nWill do install for :"
44 if [ $PASS1 -eq 1 ];then echo "- Pass 1";fi 44 if [ $PASS1 -eq 1 ];then echo "- Pass 1";fi
45 if [ $PASS2 -eq 1 ];then echo "- Pass 2";fi 45 if [ $PASS2 -eq 1 ];then echo "- Pass 2";fi
46 if [ $CONFPASS -eq 1 ];then echo "- Confidence Pass";fi 46 if [ $CONFPASS -eq 1 ];then echo "- Confidence Pass";fi
47 if [ $EXPLOITCONFPASS -eq 1 ];then echo "- Exploit Confidence Pass";fi 47 if [ $EXPLOITCONFPASS -eq 1 ];then echo "- Exploit Confidence Pass";fi
48 if [ $SCORING -eq 1 ];then echo "- Scoring Tools";fi 48 if [ $SCORING -eq 1 ];then echo "- Scoring Tools";fi
49 49
50 # 50 #
51 ### CHECK Dependencies ### 51 ### CHECK Dependencies ###
52 # 52 #
53 echo -e "\n\t${txtblu}Check Dependencies${txtrst}\n" 53 echo -e "\n\t${txtblu}Check Dependencies${txtrst}\n"
54 54
55 ## make 55 ## make
56 test=$(whereis make) 56 test=$(whereis make)
57 if [ "$test" == "make:" ] 57 if [ "$test" == "make:" ]
58 then 58 then
59 echo -e "${txtred}ERROR${txtrst} make not found\n You have to install make\n sudo apt-get install make" 59 echo -e "${txtred}ERROR${txtrst} make not found\n You have to install make\n sudo apt-get install make"
60 exit 1; 60 exit 1;
61 fi 61 fi
62 echo -e "make \t ${txtgrn}OK${txtrst}" 62 echo -e "make \t ${txtgrn}OK${txtrst}"
63 63
64 ## CC 64 ## CC
65 test=$(whereis cc) 65 test=$(whereis cc)
66 if [ "$test" == "cc:" ] 66 if [ "$test" == "cc:" ]
67 then 67 then
68 echo -e "${txtred}ERROR${txtrst} cc not found\n You have to install cc\n sudo apt-get install gcc" 68 echo -e "${txtred}ERROR${txtrst} cc not found\n You have to install cc\n sudo apt-get install gcc"
69 exit 1; 69 exit 1;
70 fi 70 fi
71 echo -e "cc \t ${txtgrn}OK${txtrst}" 71 echo -e "cc \t ${txtgrn}OK${txtrst}"
72 72
73 73
74 ## C++ 74 ## C++
75 if [ $SCORING -eq 1 ] 75 if [ $SCORING -eq 1 ]
76 then 76 then
77 test=$(whereis g++) 77 test=$(whereis g++)
78 if [ "$test" == "g++:" ] 78 if [ "$test" == "g++:" ]
79 then 79 then
80 echo -e "${txtred}ERROR${txtrst} g++ not found\n You have to install g++\n sudo apt-get install g++" 80 echo -e "${txtred}ERROR${txtrst} g++ not found\n You have to install g++\n sudo apt-get install g++"
81 exit 1; 81 exit 1;
82 fi 82 fi
83 echo -e "g++ \t ${txtgrn}OK${txtrst}" 83 echo -e "g++ \t ${txtgrn}OK${txtrst}"
84 fi 84 fi
85 85
86 ## AVCONV 86 ## AVCONV
87 test=$(whereis avconv) 87 test=$(whereis avconv)
88 if [ "$test" == "avconv:" ] 88 if [ "$test" == "avconv:" ]
89 then 89 then
90 echo -e "${txtred}ERROR${txtrst} avconv not found\n You have to install avconv\n sudo apt-get install libav-tools" 90 echo -e "${txtred}ERROR${txtrst} avconv not found\n You have to install avconv\n sudo apt-get install libav-tools"
91 exit 1; 91 exit 1;
92 fi 92 fi
93 echo -e "libav-tools : avconv \t ${txtgrn}OK${txtrst}" 93 echo -e "libav-tools : avconv \t ${txtgrn}OK${txtrst}"
94 94
95 ## JAVA 95 ## JAVA
96 test=$(whereis java) 96 test=$(whereis java)
97 if [ "$test" == "java:" ] 97 if [ "$test" == "java:" ]
98 then 98 then
99 echo -e "${txtred}ERROR${txtrst} java not found\n You have to install java JRE\n sudo apt-get install openjdk-7-jre" 99 echo -e "${txtred}ERROR${txtrst} java not found\n You have to install java JRE\n sudo apt-get install openjdk-7-jre"
100 exit 1; 100 exit 1;
101 fi 101 fi
102 echo -e "Java : JRE \t ${txtgrn}OK${txtrst}" 102 echo -e "Java : JRE \t ${txtgrn}OK${txtrst}"
103 test=$(whereis javac) 103 test=$(whereis javac)
104 if [ "$test" == "javac:" ] 104 if [ "$test" == "javac:" ]
105 then 105 then
106 echo -e "${txtred}ERROR${txtrst} javac not found\n You have to install java JDK\n sudo apt-get install openjdk-7-jdk" 106 echo -e "${txtred}ERROR${txtrst} javac not found\n You have to install java JDK\n sudo apt-get install openjdk-7-jdk"
107 exit 1; 107 exit 1;
108 fi 108 fi
109 echo -e "Java : JDK \t ${txtgrn}OK${txtrst}" 109 echo -e "Java : JDK \t ${txtgrn}OK${txtrst}"
110 110
111 if [ $EXPLOITCONFPASS -eq 1 ] 111 if [ $EXPLOITCONFPASS -eq 1 ]
112 then 112 then
113 ## Python 113 ## Python
114 test=$(whereis python) 114 test=$(whereis python)
115 if [ "$test" == "python:" ] 115 if [ "$test" == "python:" ]
116 then 116 then
117 echo -e "${txtred}ERROR${txtrst} python not found\n You have to install python\n sudo apt-get install python" 117 echo -e "${txtred}ERROR${txtrst} python not found\n You have to install python\n sudo apt-get install python"
118 exit 1; 118 exit 1;
119 fi 119 fi
120 echo -e "python : \t ${txtgrn}OK${txtrst}" 120 echo -e "python : \t ${txtgrn}OK${txtrst}"
121 121
122 ## csh shell 122 ## csh shell
123 test=$(whereis csh) 123 test=$(whereis csh)
124 if [ "$test" == "csh:" ] 124 if [ "$test" == "csh:" ]
125 then 125 then
126 echo -e "${txtred}ERROR${txtrst} csh shell not found\n You have to install csh shell\n sudo apt-get install csh" 126 echo -e "${txtred}ERROR${txtrst} csh shell not found\n You have to install csh shell\n sudo apt-get install csh"
127 exit 1; 127 exit 1;
128 fi 128 fi
129 echo -e "csh shell : \t ${txtgrn}OK${txtrst}" 129 echo -e "csh shell : \t ${txtgrn}OK${txtrst}"
130 fi 130 fi
131 131
132 ## Perl 132 ## Perl
133 test=$(whereis perl) 133 test=$(whereis perl)
134 if [ "$test" == "perl:" ] 134 if [ "$test" == "perl:" ]
135 then 135 then
136 echo -e "${txtred}ERROR${txtrst} perl not found\n You have to install perl\n sudo apt-get install perl" 136 echo -e "${txtred}ERROR${txtrst} perl not found\n You have to install perl\n sudo apt-get install perl"
137 exit 1; 137 exit 1;
138 fi 138 fi
139 echo -e "perl : \t ${txtgrn}OK${txtrst}" 139 echo -e "perl : \t ${txtgrn}OK${txtrst}"
140 140
141 ## iconv 141 ## iconv
142 test=$(whereis iconv) 142 test=$(whereis iconv)
143 if [ "$test" == "iconv:" ] 143 if [ "$test" == "iconv:" ]
144 then 144 then
145 echo -e "${txtred}ERROR${txtrst} iconv not found\n You have to install iconv\n sudo apt-cache search iconv" 145 echo -e "${txtred}ERROR${txtrst} iconv not found\n You have to install iconv\n sudo apt-cache search iconv"
146 exit 1; 146 exit 1;
147 fi 147 fi
148 echo -e "iconv : \t ${txtgrn}OK${txtrst}" 148 echo -e "iconv : \t ${txtgrn}OK${txtrst}"
149 149
150 ## SRI LM 150 ## SRI LM
151 if [ -z "$SRILM" ] || [ -z "$MACHINE_TYPE" ] 151 if [ -z "$SRILM" ] || [ -z "$MACHINE_TYPE" ]
152 then 152 then
153 echo -e "${txtred}ERROR${txtrst} SRILM toolkit variables are not defined (SRILM and MACHINE_TYPE)\n You have to install SRILM Toolkit\n" 153 echo -e "${txtred}ERROR${txtrst} SRILM toolkit variables are not defined (SRILM and MACHINE_TYPE)\n You have to install SRILM Toolkit\n"
154 exit 1; 154 exit 1;
155 fi 155 fi
156 export SRILM_BIN=$SRILM/bin/$MACHINE_TYPE 156 export SRILM_BIN=$SRILM/bin/$MACHINE_TYPE
157 echo -e "SRILM toolkit : \t ${txtgrn}OK${txtrst}" 157 echo -e "SRILM toolkit : \t ${txtgrn}OK${txtrst}"
158 158
159 ### Speeral Configuration ### 159 ### Speeral Configuration ###
160 160
161 echo -e "\n\t${txtblu}Speeral configuration${txtrst}\n" 161 echo -e "\n\t${txtblu}Speeral configuration${txtrst}\n"
162 read -e -p "Download Speeral bin and data ? (y/n) " speeral 162 read -e -p "Download Speeral bin and data ? (y/n) " speeral
163 if [ "$speeral" == "y" ] 163 if [ "$speeral" == "y" ]
164 then 164 then
165 echo -e "Download Speeral bin and data :" 165 echo -e "Download Speeral bin and data :"
166 scp -r ${username}@nyx:/local/OTMEDIA/OTMEDIA_DATA/Speeral $OTMEDIA_HOME/tools/ 166 scp -r ${username}@nyx:/local/OTMEDIA/OTMEDIA_DATA/Speeral $OTMEDIA_HOME/tools/
167 fi 167 fi
168 echo -e "\n\t${txtblu}Generating Speeral configuration files :${txtrst}\n" 168 echo -e "\n\t${txtblu}Generating Speeral configuration files :${txtrst}\n"
169 cat $PWD/tools/Speeral/CFG/SpeeralFirstPass.xml.tmp | sed -e "s|<nom>[^<]*</nom>|<nom>$PWD/tools/Speeral/LEX/LEXIQUE_V6.speer</nom>|g" \ 169 cat $PWD/tools/Speeral/CFG/SpeeralFirstPass.xml.tmp | sed -e "s|<nom>[^<]*</nom>|<nom>$PWD/tools/Speeral/LEX/LEXIQUE_V6.speer</nom>|g" \
170 | sed -e "s|<ngramme>[^<]*</ngramme>|<ngramme>$PWD/tools/Speeral/LM/ML_4gOTMEDIA_LEXIQUE_V6</ngramme>|g" \ 170 | sed -e "s|<ngramme>[^<]*</ngramme>|<ngramme>$PWD/tools/Speeral/LM/ML_4gOTMEDIA_LEXIQUE_V6</ngramme>|g" \
171 | sed -e "s|<binode>[^<]*</binode>|<binode>$PWD/tools/Speeral/LEX/LEXIQUE_V6.speer.bin</binode>|g" \ 171 | sed -e "s|<binode>[^<]*</binode>|<binode>$PWD/tools/Speeral/LEX/LEXIQUE_V6.speer.bin</binode>|g" \
172 > $PWD/tools/Speeral/CFG/SpeeralFirstPass.xml 172 > $PWD/tools/Speeral/CFG/SpeeralFirstPass.xml
173 echo $PWD/tools/Speeral/CFG/SpeeralFirstPass.xml 173 echo $PWD/tools/Speeral/CFG/SpeeralFirstPass.xml
174 cat $PWD/tools/Speeral/CFG/SpeeralSecondPass.xml.tmp | sed -e "s|<nom>[^<]*</nom>|<nom>$PWD/tools/Speeral/LEX/LEXIQUE_V6.speer</nom>|g" \ 174 cat $PWD/tools/Speeral/CFG/SpeeralSecondPass.xml.tmp | sed -e "s|<nom>[^<]*</nom>|<nom>$PWD/tools/Speeral/LEX/LEXIQUE_V6.speer</nom>|g" \
175 | sed -e "s|<ngramme>[^<]*</ngramme>|<ngramme>$PWD/tools/Speeral/LM/ML_4gOTMEDIA_LEXIQUE_V6</ngramme>|g" \ 175 | sed -e "s|<ngramme>[^<]*</ngramme>|<ngramme>$PWD/tools/Speeral/LM/ML_4gOTMEDIA_LEXIQUE_V6</ngramme>|g" \
176 | sed -e "s|<binode>[^<]*</binode>|<binode>$PWD/tools/Speeral/LEX/LEXIQUE_V6.speer.bin</binode>|g" \ 176 | sed -e "s|<binode>[^<]*</binode>|<binode>$PWD/tools/Speeral/LEX/LEXIQUE_V6.speer.bin</binode>|g" \
177 > $PWD/tools/Speeral/CFG/SpeeralSecondPass.xml 177 > $PWD/tools/Speeral/CFG/SpeeralSecondPass.xml
178 echo $PWD/tools/Speeral/CFG/SpeeralSecondPass.xml 178 echo $PWD/tools/Speeral/CFG/SpeeralSecondPass.xml
179 cat $PWD/tools/Speeral/CFG/SpeeralThirdPass.xml.tmp | sed -e "s|<nom>[^<]*</nom>|<nom>$PWD/tools/Speeral/LEX/LEXIQUE_V6.speer</nom>|g" \ 179 cat $PWD/tools/Speeral/CFG/SpeeralThirdPass.xml.tmp | sed -e "s|<nom>[^<]*</nom>|<nom>$PWD/tools/Speeral/LEX/LEXIQUE_V6.speer</nom>|g" \
180 | sed -e "s|<ngramme>[^<]*</ngramme>|<ngramme>$PWD/tools/Speeral/LM/ML_4gOTMEDIA_LEXIQUE_V6</ngramme>|g" \ 180 | sed -e "s|<ngramme>[^<]*</ngramme>|<ngramme>$PWD/tools/Speeral/LM/ML_4gOTMEDIA_LEXIQUE_V6</ngramme>|g" \
181 | sed -e "s|<binode>[^<]*</binode>|<binode>$PWD/tools/Speeral/LEX/LEXIQUE_V6.speer.bin</binode>|g" \ 181 | sed -e "s|<binode>[^<]*</binode>|<binode>$PWD/tools/Speeral/LEX/LEXIQUE_V6.speer.bin</binode>|g" \
182 > $PWD/tools/Speeral/CFG/SpeeralThirdPass.xml 182 > $PWD/tools/Speeral/CFG/SpeeralThirdPass.xml
183 echo $PWD/tools/Speeral/CFG/SpeeralThirdPass.xml 183 echo $PWD/tools/Speeral/CFG/SpeeralThirdPass.xml
184 184
185 185
186 if [ $EXPLOITCONFPASS -eq 1 ] 186 if [ $EXPLOITCONFPASS -eq 1 ]
187 then 187 then
188 ### LIA ltbox ### 188 ### LIA ltbox ###
189 echo -e "\t${txtblu}Install lia_ltbox${txtrst}\n" 189 echo -e "\t${txtblu}Install lia_ltbox${txtrst}\n"
190 export LIA_TAGG_LANG="french" 190 export LIA_TAGG_LANG="french"
191 export LIA_TAGG="$OTMEDIA_HOME/tools/lia_ltbox/lia_tagg/" 191 export LIA_TAGG="$OTMEDIA_HOME/tools/lia_ltbox/lia_tagg/"
192 export LIA_PHON_REP="$OTMEDIA_HOME/tools/lia_ltbox/lia_phon/" 192 export LIA_PHON_REP="$OTMEDIA_HOME/tools/lia_ltbox/lia_phon/"
193 export LIA_BIGLEX="$OTMEDIA_HOME/tools/lia_ltbox/lia_biglex/" 193 export LIA_BIGLEX="$OTMEDIA_HOME/tools/lia_ltbox/lia_biglex/"
194 194
195 ### config lia_phon 195 ### config lia_phon
196 cd $LIA_PHON_REP 196 cd $LIA_PHON_REP
197 make all > /dev/null 197 make all > /dev/null
198 make ressource > /dev/null 198 make ressource > /dev/null
199 ### config lia_tagg 199 ### config lia_tagg
200 cd $LIA_TAGG 200 cd $LIA_TAGG
201 make all > /dev/null 201 make all > /dev/null
202 make ressource.french > /dev/null 202 make ressource.french > /dev/null
203 ### config lia_biglex 203 ### config lia_biglex
204 cd $LIA_BIGLEX 204 cd $LIA_BIGLEX
205 make -f makefile.biglex > /dev/null 205 make -f makefile.biglex > /dev/null
206 cd $OTMEDIA_HOME 206 cd $OTMEDIA_HOME
207 207
208 if [ $SCORING -eq 1 ] 208 if [ $SCORING -eq 1 ]
209 then 209 then
210 echo -e "\t${txtblu}Install CRF++${txtrst}\n" 210 echo -e "\t${txtblu}Install CRF++${txtrst}\n"
211 tar -xvzf "$OTMEDIA_HOME/tools/CRF++-0.58.tar.gz" -C "$OTMEDIA_HOME/tools/" 211 tar -xvzf "$OTMEDIA_HOME/tools/CRF++-0.58.tar.gz" -C "$OTMEDIA_HOME/tools/"
212 cd $OTMEDIA_HOME/tools/CRF++-0.58 212 cd $OTMEDIA_HOME/tools/CRF++-0.58
213 ./configure --prefix="$OTMEDIA_HOME/tools/install" 213 ./configure --prefix="$OTMEDIA_HOME/tools/install"
214 make 214 make
215 make install 215 make install
216 echo -e "\t${txtblu}Install lia_ne${txtrst}\n" 216 echo -e "\t${txtblu}Install lia_ne${txtrst}\n"
217 export PATH=$PATH:$OTMEDIA_HOME/tools/install/bin 217 export PATH=$PATH:$OTMEDIA_HOME/tools/install/bin
218 export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$OTMEDIA_HOME/tools/install/lib 218 export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$OTMEDIA_HOME/tools/install/lib
219 export LIA_NE=$OTMEDIA_HOME/tools/lia_ltbox/lia_ne_v2.2 219 export LIA_NE=$OTMEDIA_HOME/tools/lia_ltbox/lia_ne_v2.2
220 cd $LIA_NE 220 cd $LIA_NE
221 mkdir $LIA_NE/bin 2> /dev/null 221 mkdir $LIA_NE/bin 2> /dev/null
222 make all 222 make all
223 cd $OTMEDIA_HOME 223 cd $OTMEDIA_HOME
224 fi 224 fi
225 225
226 ### SOLR DB ### 226 ### SOLR DB ###
227 # Tomcat fisrtly 227 # Tomcat fisrtly
228 test=$(dpkg -l | grep "^ii" | grep tomcat) 228 test=$(dpkg -l | grep "^ii" | grep tomcat)
229 if [ "$test" == "" ] 229 if [ "$test" == "" ]
230 then 230 then
231 echo -e "${txtred}ERROR${txtrst} TOMCAT seems to not be installed)\n You have to install TOMCAT\n" 231 echo -e "${txtred}ERROR${txtrst} TOMCAT seems to not be installed)\n You have to install TOMCAT\n"
232 #exit 1; 232 #exit 1;
233 echo -e "\nTOMCAT : \t ${txtred}NOT OK${txtrst}\n" 233 echo -e "\nTOMCAT : \t ${txtred}NOT OK${txtrst}\n"
234 else 234 else
235 echo -e "\nTOMCAT : \t ${txtgrn}OK${txtrst}\n" 235 echo -e "\nTOMCAT : \t ${txtgrn}OK${txtrst}\n"
236 fi 236 fi
237 # SOLR secondly 237 # SOLR secondly
238 echo -e "\t${txtblu}Install SOLR DB${txtrst}\n" 238 echo -e "\t${txtblu}Install SOLR DB${txtrst}\n"
239 echo -e "You will need 300 Go of free space to install SOLR DB" 239 echo -e "You will need 300 Go of free space to install SOLR DB"
240 read -p "Continue ? (y/n) " solr 240 read -p "Continue ? (y/n) " solr
241 if [ "$solr" == "y" ] 241 if [ "$solr" == "y" ]
242 then 242 then
243 243
244 echo -e "Download SOLR DB\r" 244 echo -e "Download SOLR DB\r"
245 mkdir -p $OTMEDIA_HOME/tools/SOLR 2> /dev/null 245 mkdir -p $OTMEDIA_HOME/tools/SOLR 2> /dev/null
246 scp -r ${username}@nyx:/local/OTMEDIA/OTMEDIA_DATA/SOLR/otmedia-2013-04.tar.gz $OTMEDIA_HOME/tools/SOLR 246 scp -r ${username}@nyx:/local/OTMEDIA/OTMEDIA_DATA/SOLR/otmedia-2013-04.tar.gz $OTMEDIA_HOME/tools/SOLR
247 echo -e "Unzip SOLR DB\r" 247 echo -e "Unzip SOLR DB\r"
248 tar -xvzf "$OTMEDIA_HOME/tools/SOLR/otmedia-2013-04.tar.gz" -C "$OTMEDIA_HOME/tools/SOLR/" 248 tar -xvzf "$OTMEDIA_HOME/tools/SOLR/otmedia-2013-04.tar.gz" -C "$OTMEDIA_HOME/tools/SOLR/"
249 else 249 else
250 echo "Skipping SOLR install" 250 echo "Skipping SOLR install"
251 fi 251 fi
252 read -e -p "Configure SOLR DB server ? (y/n) " solr 252 read -e -p "Configure SOLR DB server ? (y/n) " solr
253 if [ "$solr" == "y" ] 253 if [ "$solr" == "y" ]
254 then 254 then
255 read -p "Enter SOLR server IP :" ip 255 read -p "Enter SOLR server IP :" ip
256 if [ "${ip}" == "" ];then ip="localhost";fi 256 if [ "${ip}" == "" ];then ip="localhost";fi
257 echo "machine = \"${ip}\"" > $OTMEDIA_HOME/tools/scripts/solrinfo.py 257 echo "machine = \"${ip}\"" > $OTMEDIA_HOME/tools/scripts/solrinfo.py
258 read -p "Enter SOLR server port :" port 258 read -p "Enter SOLR server port :" port
259 if [ "${port}" == "" ]; then port="8080";fi 259 if [ "${port}" == "" ]; then port="8080";fi
260 echo -e "\n\tSOLR server IP ${ip}" 260 echo -e "\n\tSOLR server IP ${ip}"
261 echo -e "\tSOLR server port ${port}" 261 echo -e "\tSOLR server port ${port}"
262 echo "port = \"${port}\"" >> $OTMEDIA_HOME/tools/scripts/solrinfo.py 262 echo "port = \"${port}\"" >> $OTMEDIA_HOME/tools/scripts/solrinfo.py
263 else 263 else
264 echo "Skipping SOLR DB Configuration" 264 echo "Skipping SOLR DB Configuration"
265 fi 265 fi
266 echo -e "\nSee SOLR.INSTALL file for more information\n" 266 echo -e "\nSee SOLR.INSTALL file for more information\n"
267 fi 267 fi
268 268
269 ### Set Variables in bashrc ### 269 ### Set Variables in bashrc ###
270 cat ~/.bashrc | grep -v "OTMEDIA_HOME" | grep -v "SRILM_BIN" | grep -v "main_tools" > ~/.bashrc.org 270 cat ~/.bashrc | grep -v "OTMEDIA_HOME" | grep -v "SRILM_BIN" | grep -v "main_tools" > ~/.bashrc.org
271 #cat ~/.bashrc | grep -v "OTMEDIA_HOME" | grep -v "SRILM_BIN" | grep -v "LIA_TAGG" | grep -v "LIA_PHON" | grep -v "LIA_BIGLEX" > ~/.bashrc.org 271 #cat ~/.bashrc | grep -v "OTMEDIA_HOME" | grep -v "SRILM_BIN" | grep -v "LIA_TAGG" | grep -v "LIA_PHON" | grep -v "LIA_BIGLEX" > ~/.bashrc.org
272 cp ~/.bashrc.org ~/.bashrc 272 cp ~/.bashrc.org ~/.bashrc
273 export OTMEDIA_HOME=$PWD 273 export OTMEDIA_HOME=$PWD
274 echo "export OTMEDIA_HOME=$PWD" >> ~/.bashrc 274 echo "export OTMEDIA_HOME=$PWD" >> ~/.bashrc
275 echo "export PATH=\$PATH:$PWD/main_tools" >> ~/.bashrc 275 echo "export PATH=\$PATH:$PWD/main_tools" >> ~/.bashrc
276 echo "export SRILM_BIN=$SRILM/bin/$MACHINE_TYPE" >> ~/.bashrc 276 echo "export SRILM_BIN=$SRILM/bin/$MACHINE_TYPE" >> ~/.bashrc
277 #echo "export LIA_TAGG_LANG=french" >> ~/.bashrc 277 #echo "export LIA_TAGG_LANG=french" >> ~/.bashrc
278 #echo "export LIA_TAGG=$OTMEDIA_HOME/tools/lia_ltbox/lia_tagg/" >> ~/.bashrc 278 #echo "export LIA_TAGG=$OTMEDIA_HOME/tools/lia_ltbox/lia_tagg/" >> ~/.bashrc
279 #echo "export LIA_PHON_REP=$OTMEDIA_HOME/tools/lia_ltbox/lia_phon/" >> ~/.bashrc 279 #echo "export LIA_PHON_REP=$OTMEDIA_HOME/tools/lia_ltbox/lia_phon/" >> ~/.bashrc
280 #echo "export LIA_BIGLEX=$OTMEDIA_HOME/tools/lia_ltbox/lia_biglex/" >> ~/.bashrc 280 #echo "export LIA_BIGLEX=$OTMEDIA_HOME/tools/lia_ltbox/lia_biglex/" >> ~/.bashrc
281 #echo "export LIA_NE=$OTMEDIA_HOME/tools/lia_ltbox/lia_ne_v2.2" >> ~/.bashrc
281 #echo "export LIA_NE=$OTMEDIA_HOME/tools/lia_ltbox/lia_ne_v2.2" >> ~/.bashrc 282
282 283 # set global configuration file
283 # set global configuration file 284 echo "OTMEDIA_HOME=\"$PWD\"" > $OTMEDIA_HOME/cfg/main_cfg.cfg
284 echo "OTMEDIA_HOME=\"$PWD\"" > $OTMEDIA_HOME/cfg/main_cfg.cfg 285 echo "ARCH=\"$ARCH\"" >> $OTMEDIA_HOME/cfg/main_cfg.cfg
285 echo "ARCH=\"$ARCH\"" >> $OTMEDIA_HOME/cfg/main_cfg.cfg 286 echo "VERBOSE=0" >> $OTMEDIA_HOME/cfg/main_cfg.cfg
286 echo "VERBOSE=0" >> $OTMEDIA_HOME/cfg/main_cfg.cfg 287 echo "DEBUG=0" >> $OTMEDIA_HOME/cfg/main_cfg.cfg
287 echo "DEBUG=0" >> $OTMEDIA_HOME/cfg/main_cfg.cfg 288 echo "CHECK=0" >> $OTMEDIA_HOME/cfg/main_cfg.cfg
288 echo "CHECK=0" >> $OTMEDIA_HOME/cfg/main_cfg.cfg 289 echo "RERUN=0" >> $OTMEDIA_HOME/cfg/main_cfg.cfg
289 echo "RERUN=0" >> $OTMEDIA_HOME/cfg/main_cfg.cfg 290
290 291 echo -e "\n\t${txtgrn}### Install completed ###${txtrst}\n"
291 echo -e "\n\t${txtgrn}### Install completed ###${txtrst}\n" 292 echo -e "do : source ~/.bashrc"
292 echo -e "do : source ~/.bashrc" 293 echo -e "or set variable :\n"
293 echo -e "or set variable :\n" 294 echo "export OTMEDIA_HOME=$PWD"
294 echo "export OTMEDIA_HOME=$PWD" 295 echo "export PATH=\$PATH:$OTMEDIA_HOME/main_tools"
295 echo "export PATH=\$PATH:$OTMEDIA_HOME/main_tools" 296 echo "export SRILM_BIN=$SRILM/bin/$MACHINE_TYPE"
296 echo "export SRILM_BIN=$SRILM/bin/$MACHINE_TYPE" 297
297 298
298 299 echo "${txtwht} \\\\ "
299 echo "${txtwht} \\\\ " 300 echo " ,-~~~-\\\\_"
300 echo " ,-~~~-\\\\_" 301 echo " ( .\ "
301 echo " ( .\ " 302 echo " @\___(__--'${txtrst}"
302 echo " @\___(__--'${txtrst}" 303
303 304 echo "${txtgrn}Yes${txtylw}I${txtred}Rastafari${txtrst}"
304 echo "${txtgrn}Yes${txtylw}I${txtred}Rastafari${txtrst}" 305
main_tools/ScoringRes.sh
1 #!/bin/bash 1 #!/bin/bash
2 2
3 ##################################################### 3 #####################################################
4 # File : ScoringRes.sh # 4 # File : ScoringRes.sh #
5 # Brief : Score result using SRT file # 5 # Brief : Score result using SRT file #
6 # Author : Jean-François Rey # 6 # Author : Jean-François Rey #
7 # Version : 1.0 # 7 # Version : 1.0 #
8 # Date : 11/07/13 # 8 # Date : 11/07/13 #
9 ##################################################### 9 #####################################################
10 10
11 echo "### ScoringRes.sh ###" 11 echo "### ScoringRes.sh ###"
12 12
13 # Check OTMEDIA_HOME env var 13 # Check OTMEDIA_HOME env var
14 if [ -z ${OTMEDIA_HOME} ] 14 if [ -z ${OTMEDIA_HOME} ]
15 then 15 then
16 OTMEDIA_HOME=$(dirname $(dirname $(readlink -e $0))) 16 OTMEDIA_HOME=$(dirname $(dirname $(readlink -e $0)))
17 export OTMEDIA_HOME=$OTMEDIA_HOME 17 export OTMEDIA_HOME=$OTMEDIA_HOME
18 fi 18 fi
19 19
20 # where is ScoringRes.sh 20 # where is ScoringRes.sh
21 MAIN_SCRIPT_PATH=$OTMEDIA_HOME/main_tools 21 MAIN_SCRIPT_PATH=$OTMEDIA_HOME/main_tools
22 22
23 # Scripts Path 23 # Scripts Path
24 SCRIPT_PATH=$OTMEDIA_HOME/tools/scripts 24 SCRIPT_PATH=$OTMEDIA_HOME/tools/scripts
25 25
26 # Include scripts 26 # Include scripts
27 . $SCRIPT_PATH"/Tools.sh" 27 . $SCRIPT_PATH"/Tools.sh"
28 28
29 # where is ScoringRes.cfg 29 # where is ScoringRes.cfg
30 SCORINGRES_CONFIG_FILE=$OTMEDIA_HOME"/cfg/ScoringRes.cfg" 30 SCORINGRES_CONFIG_FILE=$OTMEDIA_HOME"/cfg/ScoringRes.cfg"
31 if [ -e $SCORINGRES_CONFIG_FILE ] 31 if [ -e $SCORINGRES_CONFIG_FILE ]
32 then 32 then
33 . $SCORINGRES_CONFIG_FILE 33 . $SCORINGRES_CONFIG_FILE
34 else 34 else
35 echo "ERROR : Can't find configuration file $SCORINGRES_CONFIG_FILE" >&2 35 echo "ERROR : Can't find configuration file $SCORINGRES_CONFIG_FILE" >&2
36 exit 1 36 exit 1
37 fi 37 fi
38 38
39 #---------------# 39 #---------------#
40 # Parse Options # 40 # Parse Options #
41 #---------------# 41 #---------------#
42 while getopts ":hDv:" opt 42 while getopts ":hDv:" opt
43 do 43 do
44 case $opt in 44 case $opt in
45 h) 45 h)
46 echo -e "$0 [OPTIONS] <PASS_DIRECTORY>\n" 46 echo -e "$0 [OPTIONS] <PASS_DIRECTORY>\n"
47 echo -e "\t Options:" 47 echo -e "\t Options:"
48 echo -e "\t\t-h :\tprint this message" 48 echo -e "\t\t-h :\tprint this message"
49 echo -e "\t\t-D :\tDEBUG mode on" 49 echo -e "\t\t-D :\tDEBUG mode on"
50 echo -e "\t\t-v l :\tVerbose mode, l=(1|2|3) level mode" 50 echo -e "\t\t-v l :\tVerbose mode, l=(1|2|3) level mode"
51 exit 1 51 exit 1
52 ;; 52 ;;
53 D) 53 D)
54 DEBUG=1 54 DEBUG=1
55 ;; 55 ;;
56 v) 56 v)
57 VERBOSE=$OPTARG 57 VERBOSE=$OPTARG
58 ;; 58 ;;
59 :) 59 :)
60 echo "Option -$OPTARG requires an argument." >&2 60 echo "Option -$OPTARG requires an argument." >&2
61 exit 1 61 exit 1
62 ;; 62 ;;
63 \?) 63 \?)
64 echo "BAD USAGE : unknow opton -$OPTARG" 64 echo "BAD USAGE : unknow opton -$OPTARG"
65 #exit 1 65 #exit 1
66 ;; 66 ;;
67 esac 67 esac
68 68
69 done 69 done
70 70
71 # mode debug enable 71 # mode debug enable
72 if [ $DEBUG -eq 1 ] 72 if [ $DEBUG -eq 1 ]
73 then 73 then
74 set -x 74 set -x
75 echo -e "## Mode DEBUG ON ##" 75 echo -e "## Mode DEBUG ON ##"
76 fi 76 fi
77 77
78 # mode verbose enable 78 # mode verbose enable
79 if [ $VERBOSE -gt 0 ]; then echo -e "## Verbose level : $VERBOSE ##" ; fi 79 if [ $VERBOSE -gt 0 ]; then echo -e "## Verbose level : $VERBOSE ##" ; fi
80 80
81 # Check USAGE by arguments number 81 # Check USAGE by arguments number
82 if [ $(($#-($OPTIND-1))) -ne 1 ] 82 if [ $(($#-($OPTIND-1))) -ne 1 ]
83 then 83 then
84 echo "BAD USAGE : SecondPass.sh [OPTIONS] <PASS_DIR>" 84 echo "BAD USAGE : SecondPass.sh [OPTIONS] <PASS_DIR>"
85 echo "$0 -h for more info" 85 echo "$0 -h for more info"
86 exit 1 86 exit 1
87 fi 87 fi
88 88
89 shift $((OPTIND-1)) 89 shift $((OPTIND-1))
90 # check Pass directory - First argument 90 # check Pass directory - First argument
91 if [ -e $1 ] && [ -d $1 ] 91 if [ -e $1 ] && [ -d $1 ]
92 then 92 then
93 PASS_DIR=$(readlink -e $1) 93 PASS_DIR=$(readlink -e $1)
94 else 94 else
95 print_error "can't find $1 directory" 95 print_error "can't find $1 directory"
96 exit 1 96 exit 1
97 fi 97 fi
98 98
99 #-------------# 99 #-------------#
100 # GLOBAL VARS # 100 # GLOBAL VARS #
101 #-------------# 101 #-------------#
102 BASENAME=$(basename $PASS_DIR) 102 BASENAME=$(basename $PASS_DIR)
103 SCORING_DIR=$PASS_DIR/scoring 103 SCORING_DIR=$PASS_DIR/scoring
104 LOGFILE="$PASS_DIR/info_scoring.log" 104 LOGFILE="$PASS_DIR/info_scoring.log"
105 ERRORFILE="$PASS_DIR/error_scoring.log" 105 ERRORFILE="$PASS_DIR/error_scoring.log"
106 106
107 print_info "[${BASENAME}] => ScoringRes start | $(date +'%d/%m/%y %H:%M:%S')" 1 107 print_info "[${BASENAME}] => ScoringRes start | $(date +'%d/%m/%y %H:%M:%S')" 1
108 108
109 #------------------# 109 #------------------#
110 # Create WORKSPACE # 110 # Create WORKSPACE #
111 #------------------# 111 #------------------#
112 112
113 # Lock directory 113 # Lock directory
114 if [ -e $PASS_DIR/*.lock ] 114 if [ -e $PASS_DIR/*.lock ]
115 then 115 then
116 print_warn "[${BASENAME}] Directory is locked -> exit" 2; 116 print_warn "[${BASENAME}] Directory is locked -> exit" 2;
117 exit 1 117 exit 1
118 fi 118 fi
119 119
120 rm -r $SCORING_DIR > /dev/null 2>&1 120 rm -r $SCORING_DIR > /dev/null 2>&1
121 mkdir -p $SCORING_DIR 121 mkdir -p $SCORING_DIR
122 print_info "Make directory $SCORING_DIR" 1 122 print_info "Make directory $SCORING_DIR" 1
123 123
124 if [ ! -e $PASS_DIR/$BASENAME.SRT ] 124 if [ ! -e $PASS_DIR/$BASENAME.SRT ]
125 then 125 then
126 print_warn "[${BASENAME}] No SRT file -> exit" 2 126 print_warn "[${BASENAME}] No SRT file -> exit" 2
127 exit 0; 127 exit 0;
128 else 128 else
129 SRT_FILE=$PASS_DIR/$BASENAME.SRT 129 SRT_FILE=$PASS_DIR/$BASENAME.SRT
130 print_info "[${BASENAME}] SRT file : $BASENAME.SRT" 1 130 print_info "[${BASENAME}] SRT file : $BASENAME.SRT" 1
131 fi 131 fi
132 132
133 if [ ! -e $PASS_DIR/$BASENAME.1pass.ctm ] && [ ! -e $PASS_DIR/$BASENAME.2pass.ctm ] && [ ! -e $PASS_DIR/$BASENAME.3pass.ctm ] && [ ! -e $PASS_DIR/$BASENAME.allpass.ctm ] 133 if [ ! -e $PASS_DIR/$BASENAME.1pass.ctm ] && [ ! -e $PASS_DIR/$BASENAME.2pass.ctm ] && [ ! -e $PASS_DIR/$BASENAME.3pass.ctm ] && [ ! -e $PASS_DIR/$BASENAME.allpass.ctm ]
134 then 134 then
135 print_error "[${BASENAME}] No CTM files -> exit\n"; 135 print_error "[${BASENAME}] No CTM files -> exit\n";
136 exit 1; 136 exit 1;
137 fi 137 fi
138 138
139 print_info "[${BASENAME}] Create stm file from SRT" 1 139 print_info "[${BASENAME}] Create stm file from SRT" 1
140 i=$((-3)) 140 i=$((-3))
141 while read line 141 while read line
142 do 142 do
143 i=$(($i + 1)); 143 i=$(($i + 1));
144 if [ $i -eq 4 ] 144 if [ $i -eq 4 ]
145 then 145 then
146 echo " $line" 146 echo " $line"
147 i=$((0)); 147 i=$((0));
148 fi 148 fi
149 done < $SRT_FILE > $SCORING_DIR/$BASENAME.tmp.txt 149 done < $SRT_FILE > $SCORING_DIR/$BASENAME.tmp.txt
150 150
151 cat $SCORING_DIR/$BASENAME.tmp.txt | sed -e "s|\n| |g" | $SCRIPT_PATH/CleanFilter.sh | $SCRIPT_PATH/ApplyCorrectionRules.pl ${LEXICON}.regex | $SCRIPT_PATH/BdlexUC.pl $RULES/basic -t | sed -e "s|<s>||g" | sed -e "s|</s>||g" | sed -e "s|\n+| |g" > $SCORING_DIR/$BASENAME.tmp2.txt 151 cat $SCORING_DIR/$BASENAME.tmp.txt | sed -e "s|\n| |g" | $SCRIPT_PATH/CleanFilter.sh | $SCRIPT_PATH/ApplyCorrectionRules.pl ${LEXICON}.regex | $SCRIPT_PATH/BdlexUC.pl $RULES/basic -t | sed -e "s|<s>||g" | sed -e "s|</s>||g" | sed -e "s|\n+| |g" > $SCORING_DIR/$BASENAME.tmp2.txt
152 #cat $SCORING_DIR/$BASENAME.tmp.txt | sed -e "s|\n| |g" | $SCRIPT_PATH/BdlexUC.pl $RULES/basic -t | sed -f $RULES/preprocess.regex | sed -f $RULES/lastprocess.regex | sed -e "s|<s>||g" | sed -e "s|</s>||g" | sed -e "s|\n+| |g" > $SCORING_DIR/$BASENAME.tmp2.txt 152 #cat $SCORING_DIR/$BASENAME.tmp.txt | sed -e "s|\n| |g" | $SCRIPT_PATH/BdlexUC.pl $RULES/basic -t | sed -f $RULES/preprocess.regex | sed -f $RULES/lastprocess.regex | sed -e "s|<s>||g" | sed -e "s|</s>||g" | sed -e "s|\n+| |g" > $SCORING_DIR/$BASENAME.tmp2.txt
153 153
154 $SCRIPT_PATH/srt2stm.pl $SCORING_DIR/$BASENAME.tmp2.txt > "$SCORING_DIR/$BASENAME.stm" 154 $SCRIPT_PATH/srt2stm.pl $SCORING_DIR/$BASENAME.tmp2.txt > "$SCORING_DIR/$BASENAME.stm"
155 155
156 #export LIA_TAGG_LANG="french" 156 export LIA_TAGG_LANG="french"
157 #export LIA_TAGG="$OTMEDIA_HOME/tools/lia_ltbox/lia_tagg/" 157 export LIA_TAGG="$OTMEDIA_HOME/tools/lia_ltbox/lia_tagg/"
158 #cat $SCORING_DIR/$BASENAME.tmp2.txt | $OTMEDIA_HOME/tools/lia_ltbox/lia_tagg/script/lia_tagg | grep -e "MOTINC\|NMS\|NMP\|NFS\|NFP\|X[A-Z]{3,5}" | cut -f1 -d " " > $SCORING_DIR/$BASENAME.tmp2.tag 158 export LIA_NE="$OTMEDIA_HOME/tools/lia_ltbox/lia_ne_v2.2/"
159 cat $SCORING_DIR/$BASENAME.tmp2.txt | $OTMEDIA_HOME/tools/lia_ltbox/lia_ne_v2.2/script/lia_ne_tagg_txt > $SCORING_DIR/$BASENAME.tmp2.tag
159 160
160 rm $SCORING_DIR/$BASENAME.tmp.txt $SCORING_DIR/$BASENAME.tmp2.txt 161 rm $SCORING_DIR/$BASENAME.tmp.txt $SCORING_DIR/$BASENAME.tmp2.txt
161 162
162 print_info "Scoring using sclite" 1 163 print_info "Scoring using sclite" 1
163 for ctmfile in $(ls $PASS_DIR/*.ctm) 164 for ctmfile in $(ls $PASS_DIR/*.ctm)
164 do 165 do
165 cp $ctmfile $SCORING_DIR/ 166 cp $ctmfile $SCORING_DIR/
166 print_info "$BIN_PATH/sclite -D -F -r $SCORING_DIR/$BASENAME.stm stm -h $SCORING_DIR/$(basename $ctmfile) ctm -o all" 3 167 print_info "$BIN_PATH/sclite -D -F -r $SCORING_DIR/$BASENAME.stm stm -h $SCORING_DIR/$(basename $ctmfile) ctm -o all" 3
167 $BIN_PATH/sclite -D -F -r $SCORING_DIR/$BASENAME.stm stm -h $SCORING_DIR/$(basename $ctmfile) ctm -o all 168 $BIN_PATH/sclite -D -F -r $SCORING_DIR/$BASENAME.stm stm -h $SCORING_DIR/$(basename $ctmfile) ctm -o all
168 done 169 done
169 170
170 print_info "[${BASENAME}] <= ScoringRes End | $(date +'%d/%m/%y %H:%M:%S')" 1 171 print_info "[${BASENAME}] <= ScoringRes End | $(date +'%d/%m/%y %H:%M:%S')" 1
171 172
172 173
173 174
tools/lia_ltbox/lia_ne_v2.2/.README.txt.swp
No preview for this file type