LucidWorks preps Solr stack as Splunk killer

LucidWorks combines the LogStash log engine with the highly scalable Lucene search engine

Enterprise search software provider LucidWorks has introduced a package based on its open-source Apache Solr search engine that will allow administrators and business analysts to extract more information from IT system logs.

SiLK "is a solution that relies on open core components that organizations can use to manage log data at scale," said Will Hayes, LucidWorks chief product officer.

The SiLK package combines Apache Lucene/Solr with a number of open-source analysis tools, namely Apache Flume, LogStash and Kibana.

Kibana offers the reporting visualization capabilities and LogStash is used to collect, store and parse logs. Flume provides a way to connect with Hadoop repositories. Apache Solr, which LucidWorks oversees, provides the searching and indexing capabilities. Solar used to be called Lucene/Solr, after two technologies that were combined, before the name was shortened to Lucene. LucidWorks employs about 25 percent of the core developers who manage and update Lucene and Solr.

The software package could help in security analysis, business intelligence, fraud detection and other use cases, according to the company. It can offer time-series analysis, data discovery and correlation.

The open source software stack that powers SiLK is not new -- many organizations have already combined LogStash and Kibana to analyze log data. Most implementations have used another open source search engine, Elasticsearch, however.

"A lot of people out in the community were talking about using LogStash with Solr," Hayes said. "A number of organizations are running into issues with using Elasticsearch at scale."

LucidWorks says that using Lucene instead of Elasticsearch will allow an organization to aggregate and search across more data, Hayes said. SiLK is aimed at organizations that have anywhere from hundreds of gigabytes to terabytes of data to ingest daily.

According to Hayes, SiLK can also provide some scalability advantages over the commercial market leader of log-data analysis, Splunk. Splunk charges, at least in part, based on how much data is being analyzed, which can add up when dealing with extremely large amounts of data.

SiLK also works well with Hadoop deployments, Hayes said. It has been certified to work with the Cloudera Enterprise 5 commercial Hadoop package and Solr is frequently incorporated into other Hadoop distributions, such as those offered by MapR and Hortonworks.

SiLK will be a component of the Lucene subscription services that LucidWorks offers, which start at about US$6,000 per node.

Joab Jackson covers enterprise software and general technology breaking news for The IDG News Service. Follow Joab on Twitter at @Joab_Jackson. Joab's e-mail address is

Join the PC World newsletter!

Error: Please check your email address.

Tags diagnosticsapplicationsUtilitiesLucidWorksdata miningsoftwareData managementsystem management

Our Back to Business guide highlights the best products for you to boost your productivity at home, on the road, at the office, or in the classroom.

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Joab Jackson

IDG News Service
Show Comments

Cool Tech

Crucial® BX200 SATA 2.5” 7mm (with 9.5mm adapter) Internal Solid State Drive

Learn more >

ASUS ROG Swift PG279Q – Reign beyond virtual world

Learn more >

D-Link TAIPAN AC3200 Ultra Wi-Fi Modem Router (DSL-4320L)

Learn more >

Xiro Drone Xplorer V -3 Axis Gimbal & 1080p Full HD 14MP Camera

Learn more >

Lexar® Professional 1000x microSDHC™/microSDXC™ UHS-II cards

Learn more >

D-Link PowerLine AV2 2000 Gigabit Network Kit

Learn more >

Gadgets & Things


Learn more >

Lexar Professional 2000x SDHC™/SDXC™ UHS-II cards

Learn more >

Lexar® Professional 1000x microSDHC™/microSDXC™ UHS-II cards

Learn more >

Family Friendly

Lexar Professional 2000x SDHC™/SDXC™ UHS-II cards

Learn more >

Lexar® Professional 1000x microSDHC™/microSDXC™ UHS-II cards

Learn more >

ASUS VivoPC VM62 - Incredibly Powerful, Unbelievably Small

Learn more >

Stocking Stuffer

Lexar Professional 2000x SDHC™/SDXC™ UHS-II cards

Learn more >

Lexar® Professional 1000x microSDHC™/microSDXC™ UHS-II cards

Learn more >

Christmas Gift Guide

Click for more ›

Most Popular Reviews

Best Deals on PC World

Latest News Articles


GGG Evaluation Team

Kathy Cassidy


First impression on unpacking the Q702 test unit was the solid feel and clean, minimalist styling.

Anthony Grifoni


For work use, Microsoft Word and Excel programs pre-installed on the device are adequate for preparing short documents.

Steph Mundell


The Fujitsu LifeBook UH574 allowed for great mobility without being obnoxiously heavy or clunky. Its twelve hours of battery life did not disappoint.

Andrew Mitsi


The screen was particularly good. It is bright and visible from most angles, however heat is an issue, particularly around the Windows button on the front, and on the back where the battery housing is located.

Simon Harriott


My first impression after unboxing the Q702 is that it is a nice looking unit. Styling is somewhat minimalist but very effective. The tablet part, once detached, has a nice weight, and no buttons or switches are located in awkward or intrusive positions.


Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?