Session

Crawl the web on a large scale with StormCrawler and Elasticsearch

StormCrawler is a popular and mature open-source web crawler. It is written in Java and is both lightweight and scalable, thanks to the distribution layer based on Apache Storm. One of the attractions of the crawler is that it is extensible and modular, as well as versatile. In this presentation, we will have a closer look at the Elasticsearch module of StormCrawler and see how it is being used in production by various organizations, sometimes on a very large scale.

Julien Nioche

Director

Bristol, United Kingdom

View Speaker Profile

Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.

Jump to top