Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Lucene and all its subprojects:

Switch to Threaded View
Mahout >> mail # user >> packaging a recommender as a war file


Copy link to this message
-
Re: packaging a recommender as a war file
Hi Ben,
basically there are a lot of differences between a command line recommender and a web application.

First thing is the packaging. If you want your recommender to work as a web application you have to put all the necessary libs into the WEB-INF/lib directory. This has nothing todo with Mahout itself it is just how web applications work. In this email you will find the complete list of files how I package them in the facebook-recommender-demo.
https://github.com/ManuelB/facebook-recommender-demo
You can open the facebook-recommender-demo directly in NetBeans and should be able to inspect it. So you have to tell NetBeans that you need the mahout libs. I am using maven to find the dependencies and maven does package them automatically including all transitive dependencies.

Next thing is life cycle of application. A command line recommender starts as a process does what it has to do and dies afterwards. A web application runs as part of the servlet container (tomcat, jetty, glassfish, etc.). This means that it will run for a long time. You should instantiate you recommender only once and then reuse it for all user requests. The recommender will use a lot of memory (in my tests about 0,2 kb per entry. meaning 60mb for 300.000 entries). As far as I know taste is thread safe. Therefore you need only one recommender in your whole web application server process. You do this in the init method. You can use a static variable and the singleton pattern. An alternative would be to put the recommender in the ServletContext by using set attribute function.

Here are some links of how to do this:
 * http://javahowto.blogspot.com/2011/07/servlet-init-method-vs-postconstruct.html
 * http://www.codestyle.org/java/servlets/faq-Threads.shtml
 * http://en.wikipedia.org/wiki/Singleton_pattern
 * http://docs.oracle.com/javaee/6/api/javax/servlet/ServletContext.html
 * http://docs.oracle.com/javaee/6/api/javax/servlet/ServletContext.html#setAttribute(java.lang.String, java.lang.Object)

Another difference is where to get your data from. I do not know if you are using a file, a database, cassandra, mongo db or any other source. I will expect that you are using a file. In a command line environment you can just pull your data from everywhere. In a web application this is far more complicated. The web application uses it own class loader and therefore it is searching resources in different places.

Again in https://github.com/ManuelB/facebook-recommender-demo/blob/master/src/main/java/de/apaxo/bedcon/FacebookRecommender.java I show how to read the values from a file which is prepacked in the  web application.

* http://tomcat.apache.org/tomcat-6.0-doc/class-loader-howto.html

...
138                        // get the file which is part of the WAR as
139 URL url = getClass().getClassLoader().getResource(DATA_FILE_NAME);
140
141 // create a file out of the resource
142 File data = new File(url.toURI());
...

I hope that helps. If you need more help you can buy some professional support. For example my company (http://www.apaxo.de) is selling professional support. Feel free to concat me personally.

/Manuel

$ jar -tf facebook-recommender-demo.war
META-INF/
META-INF/MANIFEST.MF
WEB-INF/
WEB-INF/classes/
WEB-INF/classes/de/
WEB-INF/classes/de/apaxo/
WEB-INF/classes/de/apaxo/bedcon/
WEB-INF/lib/
WEB-INF/classes/de/apaxo/bedcon/FacebookRecommender.class
WEB-INF/classes/de/apaxo/bedcon/FacebookRecommenderREST.class
WEB-INF/classes/de/apaxo/bedcon/FacebookRecommenderSOAP.class
WEB-INF/classes/DemoFriendsLikes.csv
WEB-INF/lib/commons-beanutils-1.7.0.jar
WEB-INF/lib/commons-beanutils-core-1.8.0.jar
WEB-INF/lib/commons-cli-1.2.jar
WEB-INF/lib/commons-cli-2.0-mahout.jar
WEB-INF/lib/commons-codec-1.4.jar
WEB-INF/lib/commons-collections-3.2.1.jar
WEB-INF/lib/commons-configuration-1.6.jar
WEB-INF/lib/commons-digester-1.8.jar
WEB-INF/lib/commons-httpclient-3.0.1.jar
WEB-INF/lib/commons-lang-2.6.jar
WEB-INF/lib/commons-logging-1.0.3.jar
WEB-INF/lib/commons-math-2.2.jar
WEB-INF/lib/guava-r09.jar
WEB-INF/lib/hadoop-core-0.20.204.0.jar
WEB-INF/lib/jackson-core-asl-1.8.2.jar
WEB-INF/lib/jackson-mapper-asl-1.8.2.jar
WEB-INF/lib/lucene-analyzers-3.4.0.jar
WEB-INF/lib/lucene-core-3.4.0.jar
WEB-INF/lib/mahout-collections-1.0.jar
WEB-INF/lib/mahout-core-0.6.jar
WEB-INF/lib/mahout-math-0.6.jar
WEB-INF/lib/slf4j-api-1.6.1.jar
WEB-INF/lib/solr-commons-csv-3.1.0.jar
WEB-INF/lib/uncommons-maths-1.2.2.jar
WEB-INF/lib/watchmaker-framework-0.6.2.jar
WEB-INF/lib/xpp3_min-1.1.4c.jar
WEB-INF/lib/xstream-1.3.1.jar
WEB-INF/web.xml
META-INF/maven/
META-INF/maven/de.apaxo.bedcon/
META-INF/maven/de.apaxo.bedcon/facebook-recommender-demo/
META-INF/maven/de.apaxo.bedcon/facebook-recommender-demo/pom.xml
META-INF/maven/de.apaxo.bedcon/facebook-recommender-demo/pom.properties

On 07.03.2012, at 22:52, Ben Brodie wrote:
Manuel Blechschmidt
Dortustr. 57
14467 Potsdam
Mobil: 0173/6322621
Twitter: http://twitter.com/Manuel_B
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB