Commit 160656fdfc417ed31be5052d43701e7fdfc4ee91

Authored by Jean-François Rey
1 parent 328c205196
Exists in master

update

Showing 2 changed files with 25 additions and 3 deletions Inline Diff

1 ___ _____ __ __ _____ ____ ___ _ _ ___ _ 1 ___ _____ __ __ _____ ____ ___ _ _ ___ _
2 / _ \_ _| \/ | ____| _ \_ _| / \ | | |_ _| / \ 2 / _ \_ _| \/ | ____| _ \_ _| / \ | | |_ _| / \
3 | | | || | | |\/| | _| | | | | | / _ \ | | | | / _ \ 3 | | | || | | |\/| | _| | | | | | / _ \ | | | | / _ \
4 | |_| || | | | | | |___| |_| | | / ___ \ | |___ | | / ___ \ 4 | |_| || | | | | | |___| |_| | | / ___ \ | |___ | | / ___ \
5 \___/ |_| |_| |_|_____|____/___/_/ \_\ |_____|___/_/ \_\ 5 \___/ |_| |_| |_|_____|____/___/_/ \_\ |_____|___/_/ \_\
6 6
7 7
8 #---------------# 8 #---------------#
9 # OTMEDIA LIA # 9 # OTMEDIA LIA #
10 # README # 10 # README #
11 # version 1.0 # 11 # version 1.0 #
12 #---------------# 12 #---------------#
13 13
14 DESCRIPTION 14 DESCRIPTION
15 ----------- 15 -----------
16 16
17 OTMEDIA means "Observatoire Transmedia", its main objective is to study the evolution and transformation of the media world. 17 OTMEDIA means "Observatoire Transmedia", its main objective is to study the evolution and transformation of the media world.
18 The scientific objective of the project is the creation of a new generation of media observatory 18 The scientific objective of the project is the creation of a new generation of media observatory
19 based on an interactive automatic analysis system (semi-automatic) transmedia to understand 19 based on an interactive automatic analysis system (semi-automatic) transmedia to understand
20 the world of information and developments. 20 the world of information and developments.
21 21
22 Web Site : http://www.otmedia.fr 22 Web Site : http://www.otmedia.fr
23 23
24 OTMEDIA LIA project is a set of tools to transcribe radio and TV shows. 24 OTMEDIA LIA project is a set of tools to transcribe radio and TV shows.
25 It does multiple things : 25 It does multiple things :
26 - First pass : default transcription with speeral and speaker diarization. 26 - First pass : default transcription with speeral and speaker diarization.
27 - Second pass : speaker adaptation and a second transcription pass with speeral. 27 - Second pass : speaker adaptation and a second transcription pass with speeral.
28 - Confidence pass : calcul confidence measure from transcription output. 28 - Confidence pass : calcul confidence measure from transcription output.
29 - Exploit Confidence Measure : use SOLR DB data to extend the lexicon on low confidence measure and create trigg files. 29 - Exploit Confidence Measure : use SOLR DB data to extend the lexicon on low confidence measure and create trigg files.
30 - Third pass : second pass using the new lexicon and trigg files. 30 - Third pass : second pass using the new lexicon and trigg files.
31 31
32 From GIT : http://gitlia.univ-avignon.fr/jean-francois.rey/otmedia 32 From GIT : http://gitlia.univ-avignon.fr/jean-francois.rey/otmedia
33 33
34 DEPENDENCIES 34 DEPENDENCIES
35 ------------ 35 ------------
36 36
37 GNU Toolchain 37 GNU Toolchain
38 Available from : http://www.gnu.org 38 Available from : http://www.gnu.org
39 and debian packages 39 and debian packages
40 40
41 Compiling, linking, and building applications. 41 Compiling, linking, and building applications.
42 (g++ will be needed if you install scoring tools)
42 43
43 44
44 avconv (libav-tools >= 0.8) 45 avconv (libav-tools >= 0.8)
45 Available from : http://libav.org 46 Available from : http://libav.org
46 and debian package 47 and debian package
47 48
48 avconv is a very fast video and audio converter. 49 avconv is a very fast video and audio converter.
49 50
50 JAVA JDK and JRE ( >= 6) 51 JAVA JDK and JRE ( >= 6)
51 Available from : http://www.oralce.com 52 Available from : http://www.oralce.com
52 and debian packages 53 and debian packages
53 54
54 JAVA Developpment kit and JAVA runtime environment. 55 JAVA Developpment kit and JAVA runtime environment.
55 56
56 Python ( >= 2.7.0) 57 Python ( >= 2.7.0)
57 Available from : http://http://www.python.org/ 58 Available from : http://http://www.python.org/
58 and debian packages 59 and debian packages
59 60
60 Python is a programming language. 61 Python is a programming language.
61 62
62 Perl ( >= 5.0.0) 63 Perl ( >= 5.0.0)
63 Available from : http://www.perl.org/ 64 Available from : http://www.perl.org/
64 and debian packages 65 and debian packages
65 66
66 Perl is a programming language. 67 Perl is a programming language.
67 68
68 iconv ( >= 2.0.0) 69 iconv ( >= 2.0.0)
69 Available from : http://www.gnu.org 70 Available from : http://www.gnu.org
70 and debian package 71 and debian package
71 72
72 Character set conversion. 73 Character set conversion.
73 74
74 csh shell (csh) 75 csh shell (csh)
75 Available on debian packages. 76 Available on debian packages.
76 77
77 The C shell was originally written at UCB to overcome limitations in the 78 The C shell was originally written at UCB to overcome limitations in the
78 Bourne shell. Its flexibility and comfort (at that time) quickly made it 79 Bourne shell. Its flexibility and comfort (at that time) quickly made it
79 the shell of choice until more advanced shells like ksh, bash, zsh or 80 the shell of choice until more advanced shells like ksh, bash, zsh or
80 tcsh appeared. Most of the latter incorporate features original to csh 81 tcsh appeared. Most of the latter incorporate features original to csh
81 82
82 The SRI Language Modeling Toolkit (SRILM >= 1.6.0) 83 The SRI Language Modeling Toolkit (SRILM >= 1.6.0)
83 Available from : http://www.speech.sri.com/projects/srilm/ 84 Available from : http://www.speech.sri.com/projects/srilm/
84 85
85 SRILM is a toolkit for building and applying statistical language models. 86 SRILM is a toolkit for building and applying statistical language models.
86 87
87 Tomcat ( >= 7.0.0) 88 Tomcat ( >= 7.0.0)
88 Available from : http://tomcat.apache.org/ 89 Available from : http://tomcat.apache.org/
89 and debian packages 90 and debian packages
90 91
91 Apache Tomcat is an open source software implementation of the Java Servlet and JavaServer Pages technologies. 92 Apache Tomcat is an open source software implementation of the Java Servlet and JavaServer Pages technologies.
93
94 libxml2-dev ( >= 2.7 ) [needed for scoring only]
95 Available from http://www.xmlsoft.org/
96 and debian packages
97
98 Libxml2 is the XML C parser and toolkit.
92 99
93 INSTALL 100 INSTALL
94 ------- 101 -------
95 102
96 See the INSTALL file for the installation procedure. 103 See the INSTALL file for the installation procedure.
97 104
98 Quick install below. 105 Quick install below.
99 106
100 Before launching installation : 107 Before launching installation :
101 108
102 Be certain that all dependencies are satisfied. 109 Be certain that all dependencies are satisfied.
103 Have 300 Go of free space for complet install. 110 Have 300 Go of free space for complet install.
104 111
105 Issue the following commands to the shell : 112 Issue the following commands to the shell :
106 $> ./install.sh 113 $> ./install.sh
107 $> export OTMEDIA_HOME=path/to/OTMEDIA/directory 114 $> export OTMEDIA_HOME=path/to/OTMEDIA/directory
108 115
109 Read SOLR.INSTALL part 3 to install SOLRDB. 116 Read SOLR.INSTALL part 3 to install SOLRDB.
110 117
111 RUNNING 118 RUNNING
112 ------- 119 -------
113 120
114 See HOWTO file. 121 See HOWTO file.
115 122
116 ACKNOWLEDGEMENTS 123 ACKNOWLEDGEMENTS
117 ---------------- 124 ----------------
118 125
119 Many thanks to Jean-François Rey for useful help and work done. 126 Many thanks to Jean-François Rey for useful help and work done.
120 127
121 KNOWN BUGS 128 KNOWN BUGS
122 ---------- 129 ----------
123 130
124 Many. 131 Many.
125 For Bug report, please contact Pascal Nocera at pascal.nocera@univ-avignon.fr 132 For Bug report, please contact Pascal Nocera at pascal.nocera@univ-avignon.fr
126 133
127 COPYRIGHT 134 COPYRIGHT
128 --------- 135 ---------
129 136
130 See the COPYING file. 137 See the COPYING file.
131 138
132 AUTHORS 139 AUTHORS
133 ------- 140 -------
134 141
135 Jean-François Rey <jean-francois.rey@univ-avignon.fr> 142 Jean-François Rey <jean-francois.rey@univ-avignon.fr>
136 Hugo Mauchrétien <hugo.mauchretien@univ-avignon.fr> 143 Hugo Mauchrétien <hugo.mauchretien@univ-avignon.fr>
137 Emmanuel Ferreira <emmanuel.ferreira@univ-avignon.fr> 144 Emmanuel Ferreira <emmanuel.ferreira@univ-avignon.fr>
138 145
139 146
1 #!/bin/bash 1 #!/bin/bash
2 2
3 #-------------------# 3 #-------------------#
4 # OTMEDIA LIA # 4 # OTMEDIA LIA #
5 # Install script # 5 # Install script #
6 # version : 1.1.0 # 6 # version : 1.1.0 #
7 #-------------------# 7 #-------------------#
8 8
9 # nyx login name 9 # nyx login name
10 username=${LOGNAME} 10 username=${LOGNAME}
11 11
12 # Color variables 12 # Color variables
13 txtred=$(tput setaf 1) # red 13 txtred=$(tput setaf 1) # red
14 txtgrn=$(tput setaf 2) # Green 14 txtgrn=$(tput setaf 2) # Green
15 txtylw=$(tput setaf 3) # Yellow 15 txtylw=$(tput setaf 3) # Yellow
16 txtblu=$(tput setaf 4) # Blue 16 txtblu=$(tput setaf 4) # Blue
17 txtred=$(tput setaf 5) # Purple 17 txtred=$(tput setaf 5) # Purple
18 txtcyn=$(tput setaf 6) # Cyan 18 txtcyn=$(tput setaf 6) # Cyan
19 txtwht=$(tput setaf 7) # White 19 txtwht=$(tput setaf 7) # White
20 txtrst=$(tput sgr0) # Text reset. 20 txtrst=$(tput sgr0) # Text reset.
21 #/color 21 #/color
22 22
23 # 23 #
24 ### Global Variables 24 ### Global Variables
25 # 25 #
26 PWD=$(pwd) 26 PWD=$(pwd)
27 OTMEDIA_HOME=$PWD 27 OTMEDIA_HOME=$PWD
28 test=$(arch) 28 test=$(arch)
29 if [ "$test" == "x86_64" ]; then ARCH=".64"; else ARCH=""; fi 29 if [ "$test" == "x86_64" ]; then ARCH=".64"; else ARCH=""; fi
30 #/Global 30 #/Global
31 31
32 32
33 # 33 #
34 # Put to 0 to disable dependencies of a pass 34 # Put to 0 to disable dependencies of a pass
35 # and 1 to enable 35 # and 1 to enable
36 # 36 #
37 PASS1=1 # First Pass 37 PASS1=1 # First Pass
38 PASS2=1 # Second and Third Pass 38 PASS2=1 # Second and Third Pass
39 CONFPASS=1 # Confidence Pass 39 CONFPASS=1 # Confidence Pass
40 EXPLOITCONFPASS=1 # SOLR query and trigg 40 EXPLOITCONFPASS=1 # SOLR query and trigg
41 SCORING=1 # Compile scoring tools (crf++) 41 SCORING=1 # Compile scoring tools (crf++)
42 42
43 echo -e "\nWill do install for :" 43 echo -e "\nWill do install for :"
44 if [ $PASS1 -eq 1 ];then echo "- Pass 1";fi 44 if [ $PASS1 -eq 1 ];then echo "- Pass 1";fi
45 if [ $PASS2 -eq 1 ];then echo "- Pass 2";fi 45 if [ $PASS2 -eq 1 ];then echo "- Pass 2";fi
46 if [ $CONFPASS -eq 1 ];then echo "- Confidence Pass";fi 46 if [ $CONFPASS -eq 1 ];then echo "- Confidence Pass";fi
47 if [ $EXPLOITCONFPASS -eq 1 ];then echo "- Exploit Confidence Pass";fi 47 if [ $EXPLOITCONFPASS -eq 1 ];then echo "- Exploit Confidence Pass";fi
48 if [ $SCORING -eq 1 ];then echo "- Scoring Tools";fi 48 if [ $SCORING -eq 1 ];then echo "- Scoring Tools";fi
49 49
50 # 50 #
51 ### CHECK Dependencies ### 51 ### CHECK Dependencies ###
52 # 52 #
53 echo -e "\n\t${txtblu}Check Dependencies${txtrst}\n" 53 echo -e "\n\t${txtblu}Check Dependencies${txtrst}\n"
54 54
55 ## make 55 ## make
56 test=$(whereis make) 56 test=$(whereis make)
57 if [ "$test" == "make:" ] 57 if [ "$test" == "make:" ]
58 then 58 then
59 echo -e "${txtred}ERROR${txtrst} make not found\n You have to install make\n sudo apt-get install make" 59 echo -e "${txtred}ERROR${txtrst} make not found\n You have to install make\n sudo apt-get install make"
60 exit 1; 60 exit 1;
61 fi 61 fi
62 echo -e "make \t ${txtgrn}OK${txtrst}" 62 echo -e "make \t ${txtgrn}OK${txtrst}"
63 63
64 ## CC 64 ## CC
65 test=$(whereis cc) 65 test=$(whereis cc)
66 if [ "$test" == "cc:" ] 66 if [ "$test" == "cc:" ]
67 then 67 then
68 echo -e "${txtred}ERROR${txtrst} cc not found\n You have to install cc\n sudo apt-get install gcc" 68 echo -e "${txtred}ERROR${txtrst} cc not found\n You have to install cc\n sudo apt-get install gcc"
69 exit 1; 69 exit 1;
70 fi 70 fi
71 echo -e "cc \t ${txtgrn}OK${txtrst}" 71 echo -e "cc \t ${txtgrn}OK${txtrst}"
72 72
73
74 ## C++
75 if [ $SCORING -eq 1 ]
76 then
77 test=$(whereis g++)
78 if [ "$test" == "g++:" ]
79 then
80 echo -e "${txtred}ERROR${txtrst} g++ not found\n You have to install g++\n sudo apt-get install g++"
81 exit 1;
82 fi
83 echo -e "g++ \t ${txtgrn}OK${txtrst}"
84 fi
85
73 ## AVCONV 86 ## AVCONV
74 test=$(whereis avconv) 87 test=$(whereis avconv)
75 if [ "$test" == "avconv:" ] 88 if [ "$test" == "avconv:" ]
76 then 89 then
77 echo -e "${txtred}ERROR${txtrst} avconv not found\n You have to install avconv\n sudo apt-get install libav-tools" 90 echo -e "${txtred}ERROR${txtrst} avconv not found\n You have to install avconv\n sudo apt-get install libav-tools"
78 exit 1; 91 exit 1;
79 fi 92 fi
80 echo -e "libav-tools : avconv \t ${txtgrn}OK${txtrst}" 93 echo -e "libav-tools : avconv \t ${txtgrn}OK${txtrst}"
81 94
82 ## JAVA 95 ## JAVA
83 test=$(whereis java) 96 test=$(whereis java)
84 if [ "$test" == "java:" ] 97 if [ "$test" == "java:" ]
85 then 98 then
86 echo -e "${txtred}ERROR${txtrst} java not found\n You have to install java JRE\n sudo apt-get install openjdk-7-jre" 99 echo -e "${txtred}ERROR${txtrst} java not found\n You have to install java JRE\n sudo apt-get install openjdk-7-jre"
87 exit 1; 100 exit 1;
88 fi 101 fi
89 echo -e "Java : JRE \t ${txtgrn}OK${txtrst}" 102 echo -e "Java : JRE \t ${txtgrn}OK${txtrst}"
90 test=$(whereis javac) 103 test=$(whereis javac)
91 if [ "$test" == "javac:" ] 104 if [ "$test" == "javac:" ]
92 then 105 then
93 echo -e "${txtred}ERROR${txtrst} javac not found\n You have to install java JDK\n sudo apt-get install openjdk-7-jdk" 106 echo -e "${txtred}ERROR${txtrst} javac not found\n You have to install java JDK\n sudo apt-get install openjdk-7-jdk"
94 exit 1; 107 exit 1;
95 fi 108 fi
96 echo -e "Java : JDK \t ${txtgrn}OK${txtrst}" 109 echo -e "Java : JDK \t ${txtgrn}OK${txtrst}"
97 110
98 if [ $EXPLOITCONFPASS -eq 1 ] 111 if [ $EXPLOITCONFPASS -eq 1 ]
99 then 112 then
100 ## Python 113 ## Python
101 test=$(whereis python) 114 test=$(whereis python)
102 if [ "$test" == "python:" ] 115 if [ "$test" == "python:" ]
103 then 116 then
104 echo -e "${txtred}ERROR${txtrst} python not found\n You have to install python\n sudo apt-get install python" 117 echo -e "${txtred}ERROR${txtrst} python not found\n You have to install python\n sudo apt-get install python"
105 exit 1; 118 exit 1;
106 fi 119 fi
107 echo -e "python : \t ${txtgrn}OK${txtrst}" 120 echo -e "python : \t ${txtgrn}OK${txtrst}"
108 121
109 ## csh shell 122 ## csh shell
110 test=$(whereis csh) 123 test=$(whereis csh)
111 if [ "$test" == "csh:" ] 124 if [ "$test" == "csh:" ]
112 then 125 then
113 echo -e "${txtred}ERROR${txtrst} csh shell not found\n You have to install csh shell\n sudo apt-get install csh" 126 echo -e "${txtred}ERROR${txtrst} csh shell not found\n You have to install csh shell\n sudo apt-get install csh"
114 exit 1; 127 exit 1;
115 fi 128 fi
116 echo -e "csh shell : \t ${txtgrn}OK${txtrst}" 129 echo -e "csh shell : \t ${txtgrn}OK${txtrst}"
117 fi 130 fi
118 131
119 ## Perl 132 ## Perl
120 test=$(whereis perl) 133 test=$(whereis perl)
121 if [ "$test" == "perl:" ] 134 if [ "$test" == "perl:" ]
122 then 135 then
123 echo -e "${txtred}ERROR${txtrst} perl not found\n You have to install perl\n sudo apt-get install perl" 136 echo -e "${txtred}ERROR${txtrst} perl not found\n You have to install perl\n sudo apt-get install perl"
124 exit 1; 137 exit 1;
125 fi 138 fi
126 echo -e "perl : \t ${txtgrn}OK${txtrst}" 139 echo -e "perl : \t ${txtgrn}OK${txtrst}"
127 140
128 ## iconv 141 ## iconv
129 test=$(whereis iconv) 142 test=$(whereis iconv)
130 if [ "$test" == "iconv:" ] 143 if [ "$test" == "iconv:" ]
131 then 144 then
132 echo -e "${txtred}ERROR${txtrst} iconv not found\n You have to install iconv\n sudo apt-cache search iconv" 145 echo -e "${txtred}ERROR${txtrst} iconv not found\n You have to install iconv\n sudo apt-cache search iconv"
133 exit 1; 146 exit 1;
134 fi 147 fi
135 echo -e "iconv : \t ${txtgrn}OK${txtrst}" 148 echo -e "iconv : \t ${txtgrn}OK${txtrst}"
136 149
137 ## SRI LM 150 ## SRI LM
138 if [ -z "$SRILM" ] || [ -z "$MACHINE_TYPE" ] 151 if [ -z "$SRILM" ] || [ -z "$MACHINE_TYPE" ]
139 then 152 then
140 echo -e "${txtred}ERROR${txtrst} SRILM toolkit variables are not defined (SRILM and MACHINE_TYPE)\n You have to install SRILM Toolkit\n" 153 echo -e "${txtred}ERROR${txtrst} SRILM toolkit variables are not defined (SRILM and MACHINE_TYPE)\n You have to install SRILM Toolkit\n"
141 exit 1; 154 exit 1;
142 fi 155 fi
143 export SRILM_BIN=$SRILM/bin/$MACHINE_TYPE 156 export SRILM_BIN=$SRILM/bin/$MACHINE_TYPE
144 echo -e "SRILM toolkit : \t ${txtgrn}OK${txtrst}" 157 echo -e "SRILM toolkit : \t ${txtgrn}OK${txtrst}"
145 158
146 ### Speeral Configuration ### 159 ### Speeral Configuration ###
147 160
148 echo -e "\n\t${txtblu}Speeral configuration${txtrst}\n" 161 echo -e "\n\t${txtblu}Speeral configuration${txtrst}\n"
149 read -e -p "Download Speeral bin and data ? (y/n) " speeral 162 read -e -p "Download Speeral bin and data ? (y/n) " speeral
150 if [ "$speeral" == "y" ] 163 if [ "$speeral" == "y" ]
151 then 164 then
152 echo -e "Download Speeral bin and data :" 165 echo -e "Download Speeral bin and data :"
153 scp -r ${username}@nyx:/local/OTMEDIA/OTMEDIA_DATA/Speeral $OTMEDIA_HOME/tools/ 166 scp -r ${username}@nyx:/local/OTMEDIA/OTMEDIA_DATA/Speeral $OTMEDIA_HOME/tools/
154 fi 167 fi
155 echo -e "\n\t${txtblu}Generating Speeral configuration files :${txtrst}\n" 168 echo -e "\n\t${txtblu}Generating Speeral configuration files :${txtrst}\n"
156 cat $PWD/tools/Speeral/CFG/SpeeralFirstPass.xml.tmp | sed -e "s|<nom>[^<]*</nom>|<nom>$PWD/tools/Speeral/LEX/LEXIQUE_V6.speer</nom>|g" \ 169 cat $PWD/tools/Speeral/CFG/SpeeralFirstPass.xml.tmp | sed -e "s|<nom>[^<]*</nom>|<nom>$PWD/tools/Speeral/LEX/LEXIQUE_V6.speer</nom>|g" \
157 | sed -e "s|<ngramme>[^<]*</ngramme>|<ngramme>$PWD/tools/Speeral/LM/ML_4gOTMEDIA_LEXIQUE_V6</ngramme>|g" \ 170 | sed -e "s|<ngramme>[^<]*</ngramme>|<ngramme>$PWD/tools/Speeral/LM/ML_4gOTMEDIA_LEXIQUE_V6</ngramme>|g" \
158 | sed -e "s|<binode>[^<]*</binode>|<binode>$PWD/tools/Speeral/LEX/LEXIQUE_V6.speer.bin</binode>|g" \ 171 | sed -e "s|<binode>[^<]*</binode>|<binode>$PWD/tools/Speeral/LEX/LEXIQUE_V6.speer.bin</binode>|g" \
159 > $PWD/tools/Speeral/CFG/SpeeralFirstPass.xml 172 > $PWD/tools/Speeral/CFG/SpeeralFirstPass.xml
160 echo $PWD/tools/Speeral/CFG/SpeeralFirstPass.xml 173 echo $PWD/tools/Speeral/CFG/SpeeralFirstPass.xml
161 cat $PWD/tools/Speeral/CFG/SpeeralSecondPass.xml.tmp | sed -e "s|<nom>[^<]*</nom>|<nom>$PWD/tools/Speeral/LEX/LEXIQUE_V6.speer</nom>|g" \ 174 cat $PWD/tools/Speeral/CFG/SpeeralSecondPass.xml.tmp | sed -e "s|<nom>[^<]*</nom>|<nom>$PWD/tools/Speeral/LEX/LEXIQUE_V6.speer</nom>|g" \
162 | sed -e "s|<ngramme>[^<]*</ngramme>|<ngramme>$PWD/tools/Speeral/LM/ML_4gOTMEDIA_LEXIQUE_V6</ngramme>|g" \ 175 | sed -e "s|<ngramme>[^<]*</ngramme>|<ngramme>$PWD/tools/Speeral/LM/ML_4gOTMEDIA_LEXIQUE_V6</ngramme>|g" \
163 | sed -e "s|<binode>[^<]*</binode>|<binode>$PWD/tools/Speeral/LEX/LEXIQUE_V6.speer.bin</binode>|g" \ 176 | sed -e "s|<binode>[^<]*</binode>|<binode>$PWD/tools/Speeral/LEX/LEXIQUE_V6.speer.bin</binode>|g" \
164 > $PWD/tools/Speeral/CFG/SpeeralSecondPass.xml 177 > $PWD/tools/Speeral/CFG/SpeeralSecondPass.xml
165 echo $PWD/tools/Speeral/CFG/SpeeralSecondPass.xml 178 echo $PWD/tools/Speeral/CFG/SpeeralSecondPass.xml
166 cat $PWD/tools/Speeral/CFG/SpeeralThirdPass.xml.tmp | sed -e "s|<nom>[^<]*</nom>|<nom>$PWD/tools/Speeral/LEX/LEXIQUE_V6.speer</nom>|g" \ 179 cat $PWD/tools/Speeral/CFG/SpeeralThirdPass.xml.tmp | sed -e "s|<nom>[^<]*</nom>|<nom>$PWD/tools/Speeral/LEX/LEXIQUE_V6.speer</nom>|g" \
167 | sed -e "s|<ngramme>[^<]*</ngramme>|<ngramme>$PWD/tools/Speeral/LM/ML_4gOTMEDIA_LEXIQUE_V6</ngramme>|g" \ 180 | sed -e "s|<ngramme>[^<]*</ngramme>|<ngramme>$PWD/tools/Speeral/LM/ML_4gOTMEDIA_LEXIQUE_V6</ngramme>|g" \
168 | sed -e "s|<binode>[^<]*</binode>|<binode>$PWD/tools/Speeral/LEX/LEXIQUE_V6.speer.bin</binode>|g" \ 181 | sed -e "s|<binode>[^<]*</binode>|<binode>$PWD/tools/Speeral/LEX/LEXIQUE_V6.speer.bin</binode>|g" \
169 > $PWD/tools/Speeral/CFG/SpeeralThirdPass.xml 182 > $PWD/tools/Speeral/CFG/SpeeralThirdPass.xml
170 echo $PWD/tools/Speeral/CFG/SpeeralThirdPass.xml 183 echo $PWD/tools/Speeral/CFG/SpeeralThirdPass.xml
171 184
172 185
173 if [ $EXPLOITCONFPASS -eq 1 ] 186 if [ $EXPLOITCONFPASS -eq 1 ]
174 then 187 then
175 ### LIA ltbox ### 188 ### LIA ltbox ###
176 echo -e "\t${txtblu}Install lia_ltbox${txtrst}\n" 189 echo -e "\t${txtblu}Install lia_ltbox${txtrst}\n"
177 export LIA_TAGG_LANG="french" 190 export LIA_TAGG_LANG="french"
178 export LIA_TAGG="$OTMEDIA_HOME/tools/lia_ltbox/lia_tagg/" 191 export LIA_TAGG="$OTMEDIA_HOME/tools/lia_ltbox/lia_tagg/"
179 export LIA_PHON_REP="$OTMEDIA_HOME/tools/lia_ltbox/lia_phon/" 192 export LIA_PHON_REP="$OTMEDIA_HOME/tools/lia_ltbox/lia_phon/"
180 export LIA_BIGLEX="$OTMEDIA_HOME/tools/lia_ltbox/lia_biglex/" 193 export LIA_BIGLEX="$OTMEDIA_HOME/tools/lia_ltbox/lia_biglex/"
181 194
182 ### config lia_phon 195 ### config lia_phon
183 cd $LIA_PHON_REP 196 cd $LIA_PHON_REP
184 make all > /dev/null 197 make all > /dev/null
185 make ressource > /dev/null 198 make ressource > /dev/null
186 ### config lia_tagg 199 ### config lia_tagg
187 cd $LIA_TAGG 200 cd $LIA_TAGG
188 make all > /dev/null 201 make all > /dev/null
189 make ressource.french > /dev/null 202 make ressource.french > /dev/null
190 ### config lia_biglex 203 ### config lia_biglex
191 cd $LIA_BIGLEX 204 cd $LIA_BIGLEX
192 make -f makefile.biglex > /dev/null 205 make -f makefile.biglex > /dev/null
193 cd $OTMEDIA_HOME 206 cd $OTMEDIA_HOME
194 207
195 if [ $SCORING -eq 1 ] 208 if [ $SCORING -eq 1 ]
196 then 209 then
197 tar xvzf $OTMEDIA_HOME/tools/CRF++-0.58.tar.gz -C $OTMEDIA_HOME/tools/ 210 echo -e "\t${txtblu}Install CRF++${txtrst}\n"
211 tar -xvzf "$OTMEDIA_HOME/tools/CRF++-0.58.tar.gz" -C "$OTMEDIA_HOME/tools/"
198 cd $OTMEDIA_HOME/tools/CRF++-0.58 212 cd $OTMEDIA_HOME/tools/CRF++-0.58
199 ./configure --prefix="$OTMEDIA_HOME/tools/install" 213 ./configure --prefix="$OTMEDIA_HOME/tools/install"
200 make 214 make
201 make install 215 make install
202 cd $OLDPATH 216 echo -e "\t${txtblu}Install lia_ne${txtrst}\n"
203 export PATH=$PATH:$OTMEDIA_HOME/tools/install/bin 217 export PATH=$PATH:$OTMEDIA_HOME/tools/install/bin
204 export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$OTMEDIA_HOME/tools/install/lib 218 export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$OTMEDIA_HOME/tools/install/lib
205 export LIA_NE=$OTMEDIA_HOME/tools/lia_ltbox/lia_ne_v2.2 219 export LIA_NE=$OTMEDIA_HOME/tools/lia_ltbox/lia_ne_v2.2
206 cd $LIA_NE 220 cd $LIA_NE
221 mkdir $LIA_NE/bin 2> /dev/null
207 make all 222 make all
208 cd $OLDPATH 223 cd $OTMEDIA_HOME
209 fi 224 fi
210 225
211 ### SOLR DB ### 226 ### SOLR DB ###
212 # Tomcat fisrtly 227 # Tomcat fisrtly
213 test=$(dpkg -l | grep "^ii" | grep tomcat) 228 test=$(dpkg -l | grep "^ii" | grep tomcat)
214 if [ "$test" == "" ] 229 if [ "$test" == "" ]
215 then 230 then
216 echo -e "${txtred}ERROR${txtrst} TOMCAT seems to not be installed)\n You have to install TOMCAT\n" 231 echo -e "${txtred}ERROR${txtrst} TOMCAT seems to not be installed)\n You have to install TOMCAT\n"
217 #exit 1; 232 #exit 1;
218 echo -e "\nTOMCAT : \t ${txtred}NOT OK${txtrst}\n" 233 echo -e "\nTOMCAT : \t ${txtred}NOT OK${txtrst}\n"
219 else 234 else
220 echo -e "\nTOMCAT : \t ${txtgrn}OK${txtrst}\n" 235 echo -e "\nTOMCAT : \t ${txtgrn}OK${txtrst}\n"
221 fi 236 fi
222 # SOLR secondly 237 # SOLR secondly
223 echo -e "\t${txtblu}Install SOLR DB${txtrst}\n" 238 echo -e "\t${txtblu}Install SOLR DB${txtrst}\n"
224 echo -e "You will need 300 Go of free space to install SOLR DB" 239 echo -e "You will need 300 Go of free space to install SOLR DB"
225 read -p "Continue ? (y/n) " solr 240 read -p "Continue ? (y/n) " solr
226 if [ "$solr" == "y" ] 241 if [ "$solr" == "y" ]
227 then 242 then
228 243
229 echo -e "Download SOLR DB\r" 244 echo -e "Download SOLR DB\r"
230 mkdir -p $OTMEDIA_HOME/tools/SOLR 2> /dev/null 245 mkdir -p $OTMEDIA_HOME/tools/SOLR 2> /dev/null
231 scp -r ${username}@nyx:/local/OTMEDIA/OTMEDIA_DATA/SOLR/otmedia-2013-04.tar.gz $OTMEDIA_HOME/tools/SOLR 246 scp -r ${username}@nyx:/local/OTMEDIA/OTMEDIA_DATA/SOLR/otmedia-2013-04.tar.gz $OTMEDIA_HOME/tools/SOLR
232 echo -e "Unzip SOLR DB\r" 247 echo -e "Unzip SOLR DB\r"
233 tar -xvzf "$OTMEDIA_HOME/tools/SOLR/otmedia-2013-04.tar.gz" -C "$OTMEDIA_HOME/tools/SOLR/" 248 tar -xvzf "$OTMEDIA_HOME/tools/SOLR/otmedia-2013-04.tar.gz" -C "$OTMEDIA_HOME/tools/SOLR/"
234 else 249 else
235 echo "Skipping SOLR install" 250 echo "Skipping SOLR install"
236 fi 251 fi
237 read -e -p "Configure SOLR DB server ? (y/n) " solr 252 read -e -p "Configure SOLR DB server ? (y/n) " solr
238 if [ "$solr" == "y" ] 253 if [ "$solr" == "y" ]
239 then 254 then
240 read -p "Enter SOLR server IP :" ip 255 read -p "Enter SOLR server IP :" ip
241 if [ "${ip}" == "" ];then ip="localhost";fi 256 if [ "${ip}" == "" ];then ip="localhost";fi
242 echo "machine = \"${ip}\"" > $OTMEDIA_HOME/tools/scripts/solrinfo.py 257 echo "machine = \"${ip}\"" > $OTMEDIA_HOME/tools/scripts/solrinfo.py
243 read -p "Enter SOLR server port :" port 258 read -p "Enter SOLR server port :" port
244 if [ "${port}" == "" ]; then port="8080";fi 259 if [ "${port}" == "" ]; then port="8080";fi
245 echo -e "\n\tSOLR server IP ${ip}" 260 echo -e "\n\tSOLR server IP ${ip}"
246 echo -e "\tSOLR server port ${port}" 261 echo -e "\tSOLR server port ${port}"
247 echo "port = \"${port}\"" >> $OTMEDIA_HOME/tools/scripts/solrinfo.py 262 echo "port = \"${port}\"" >> $OTMEDIA_HOME/tools/scripts/solrinfo.py
248 else 263 else
249 echo "Skipping SOLR DB Configuration" 264 echo "Skipping SOLR DB Configuration"
250 fi 265 fi
251 echo -e "\nSee SOLR.INSTALL file for more information\n" 266 echo -e "\nSee SOLR.INSTALL file for more information\n"
252 fi 267 fi
253 268
254 ### Set Variables in bashrc ### 269 ### Set Variables in bashrc ###
255 cat ~/.bashrc | grep -v "OTMEDIA_HOME" | grep -v "SRILM_BIN" | grep -v "main_tools" > ~/.bashrc.org 270 cat ~/.bashrc | grep -v "OTMEDIA_HOME" | grep -v "SRILM_BIN" | grep -v "main_tools" > ~/.bashrc.org
256 #cat ~/.bashrc | grep -v "OTMEDIA_HOME" | grep -v "SRILM_BIN" | grep -v "LIA_TAGG" | grep -v "LIA_PHON" | grep -v "LIA_BIGLEX" > ~/.bashrc.org 271 #cat ~/.bashrc | grep -v "OTMEDIA_HOME" | grep -v "SRILM_BIN" | grep -v "LIA_TAGG" | grep -v "LIA_PHON" | grep -v "LIA_BIGLEX" > ~/.bashrc.org
257 cp ~/.bashrc.org ~/.bashrc 272 cp ~/.bashrc.org ~/.bashrc
258 export OTMEDIA_HOME=$PWD 273 export OTMEDIA_HOME=$PWD
259 echo "export OTMEDIA_HOME=$PWD" >> ~/.bashrc 274 echo "export OTMEDIA_HOME=$PWD" >> ~/.bashrc
260 echo "export PATH=\$PATH:$PWD/main_tools" >> ~/.bashrc 275 echo "export PATH=\$PATH:$PWD/main_tools" >> ~/.bashrc
261 echo "export SRILM_BIN=$SRILM/bin/$MACHINE_TYPE" >> ~/.bashrc 276 echo "export SRILM_BIN=$SRILM/bin/$MACHINE_TYPE" >> ~/.bashrc
262 #echo "export LIA_TAGG_LANG=french" >> ~/.bashrc 277 #echo "export LIA_TAGG_LANG=french" >> ~/.bashrc
263 #echo "export LIA_TAGG=$OTMEDIA_HOME/tools/lia_ltbox/lia_tagg/" >> ~/.bashrc 278 #echo "export LIA_TAGG=$OTMEDIA_HOME/tools/lia_ltbox/lia_tagg/" >> ~/.bashrc
264 #echo "export LIA_PHON_REP=$OTMEDIA_HOME/tools/lia_ltbox/lia_phon/" >> ~/.bashrc 279 #echo "export LIA_PHON_REP=$OTMEDIA_HOME/tools/lia_ltbox/lia_phon/" >> ~/.bashrc
265 #echo "export LIA_BIGLEX=$OTMEDIA_HOME/tools/lia_ltbox/lia_biglex/" >> ~/.bashrc 280 #echo "export LIA_BIGLEX=$OTMEDIA_HOME/tools/lia_ltbox/lia_biglex/" >> ~/.bashrc
266 281
267 # set global configuration file 282 # set global configuration file
268 echo "OTMEDIA_HOME=\"$PWD\"" > $OTMEDIA_HOME/cfg/main_cfg.cfg 283 echo "OTMEDIA_HOME=\"$PWD\"" > $OTMEDIA_HOME/cfg/main_cfg.cfg
269 echo "ARCH=\"$ARCH\"" >> $OTMEDIA_HOME/cfg/main_cfg.cfg 284 echo "ARCH=\"$ARCH\"" >> $OTMEDIA_HOME/cfg/main_cfg.cfg
270 echo "VERBOSE=0" >> $OTMEDIA_HOME/cfg/main_cfg.cfg 285 echo "VERBOSE=0" >> $OTMEDIA_HOME/cfg/main_cfg.cfg
271 echo "DEBUG=0" >> $OTMEDIA_HOME/cfg/main_cfg.cfg 286 echo "DEBUG=0" >> $OTMEDIA_HOME/cfg/main_cfg.cfg
272 echo "CHECK=0" >> $OTMEDIA_HOME/cfg/main_cfg.cfg 287 echo "CHECK=0" >> $OTMEDIA_HOME/cfg/main_cfg.cfg
273 echo "RERUN=0" >> $OTMEDIA_HOME/cfg/main_cfg.cfg 288 echo "RERUN=0" >> $OTMEDIA_HOME/cfg/main_cfg.cfg
274 289
275 echo -e "\n\t${txtgrn}### Install completed ###${txtrst}\n" 290 echo -e "\n\t${txtgrn}### Install completed ###${txtrst}\n"
276 echo -e "do : source ~/.bashrc" 291 echo -e "do : source ~/.bashrc"
277 echo -e "or set variable :\n" 292 echo -e "or set variable :\n"
278 echo "export OTMEDIA_HOME=$PWD" 293 echo "export OTMEDIA_HOME=$PWD"
279 echo "export PATH=\$PATH:$OTMEDIA_HOME/main_tools" 294 echo "export PATH=\$PATH:$OTMEDIA_HOME/main_tools"
280 echo "export SRILM_BIN=$SRILM/bin/$MACHINE_TYPE" 295 echo "export SRILM_BIN=$SRILM/bin/$MACHINE_TYPE"
281 296
282 297
283 echo "${txtwht} \\\\ " 298 echo "${txtwht} \\\\ "
284 echo " ,-~~~-\\\\_" 299 echo " ,-~~~-\\\\_"
285 echo " ( .\ " 300 echo " ( .\ "
286 echo " @\___(__--'${txtrst}" 301 echo " @\___(__--'${txtrst}"
287 302
288 echo "${txtgrn}Yes${txtylw}I${txtred}Rastafari${txtrst}" 303 echo "${txtgrn}Yes${txtylw}I${txtred}Rastafari${txtrst}"
289 304