• Stars
    star
    135
  • Rank 269,297 (Top 6 %)
  • Language
    JavaScript
  • License
    Apache License 2.0
  • Created over 10 years ago
  • Updated 8 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Elasticsearch/Solr Sandbox for exploring explain information and tweaking

The Splainer!

Build Status

The sandbox that explains your search results for you so you don't have to go digging through explain debug! Paste in your Solr or Elasticsearch query URL and go. Try it out here.

Why?

You're a search developer trying to tune search results with Solr or Elasticsearch. You're engaged in search relevancy work.

You're probably stuck with the question of why? Why do search results come back in the order that they do? Solr and Elasticsearch exposes an explain syntax for you to try to explain search scoring. Unfortunately outside the simplest tasks, its a nightmare to read through. There are parsers like explain.solr.pl but they require a lot of manual copy/pasting of explain information to the tool.

Splainer is different by being a sandbox. Paste in your Solr or Elasticsearch URL, query parameters and all. As you work with your query, changing parameters, Splainer shows you parsed and summarized explain information alongside your documents. Continue working and see how the search results change.

Read the blog post introducing Splainer here

Splainer forms the core of the open source tool Quepid that allows you to do this over multiple queries against expert-graded search results to track search changes over a longer period of time.

Using Splainer

Take the tour to see how you'd use Splainer.

Using Splainer locally

We have a Docker image published at https://hub.docker.com/r/o19s/splainer that you can use:

docker run -d -p 9000:9000 o19s/splainer and then go to http://localhost:9000

Developing Splainer

Npm/Yarn Dev Environment

Splainer is written using AngularJS project. It requires npm, grunt, and yarn.

Be sure you've installed npm, yarn, and grunt on your machine.

npm install -g grunt-cli

With Npm/Yarn installed

From the root of the project, you should be able to run the following:

yarn
grunt test
grunt serve

Now browse to http://localhost:9000.

To build the project, simply run grunt dist to build the static artifacts in the dist/ folder.

grunt dist

You can test out the static artifacts via ruby -run -e httpd -- -p 5000 ./dist and going to http://localhost:5000.

With Docker installed

From the root of the project, you should run:

docker build -t splainer  .
docker run -p 9000:9000 splainer:latest

or use the following shortcuts if you have ruby installed:

bin/docker b

then to run the server run

bin/docker s

Using docker-compose

From the root of the project,

docker-compose build
docker-compose run --rm --service-ports app

Using Docker Compose to test splainer-search with splainer

  • docker-compose.override.yml.example can be copied to docker-compose.override.yml and use it to override environment variables or work with a local copy of the splainer-search JS library during development defined in docker-compose.yml. Example is included. Just update the path to splainer-search with your local checkout!

https://docs.docker.com/compose/extends/

Testing Notes

  • Unit tests are written using Karma.

  • The ./tests/splainer_test_links.html file is a list of links that invoke Splainer, both the local version and the deployed version against Solr and Elasticsearch, and is a great test to make sure the behavior hasn't reverted. Use this to make sure existing links still work!

Who?

Created by OpenSource Connections.

Thanks to all the community contributors for finding bugs and sharing fixes!.

License

Released under Apache 2

More Repositories

1

elasticsearch-learning-to-rank

Plugin to integrate Learning to Rank (aka machine learning for better relevance) with Elasticsearch
Java
1,480
star
2

relevant-search-book

Code and Examples for Relevant Search
Jupyter Notebook
300
star
3

quepid

Improve your Elasticsearch, OpenSearch, Solr, Vectara, Algolia and Custom Search search quality.
Ruby
284
star
4

hello-ltr

Set of Jupyter notebooks demonstrating Learning to Rank integrated with Solr and Elasticsearch
Jupyter Notebook
165
star
5

elyzer

"Stop worrying about Elasticsearch analyzers", my therapist says
Python
153
star
6

hello-nlp

A natural language search microservice
Python
96
star
7

awesome-search-relevance

Tools and other things for people who work on search relevance & information retrieval
82
star
8

Spyglass

Simple search results with Solr and EmberJS
JavaScript
58
star
9

solr-to-es

Migrate a Solr node to an Elasticsearch index.
Python
54
star
10

lucene-query-example

Educational Examle of a custom Lucene Query & Scorer
Java
48
star
11

solr_nginx

Starter Reverse Proxy Configuration for Solr
47
star
12

RankyMcRankFace

Hardened Fork of Ranklib learning to rank library
Java
44
star
13

SemanticSearchInNumpy

XSLT
44
star
14

hangry

Vector search in Lucene based search attempting to use just the existing Lucene data structures (experimental)
Java
43
star
15

trireme

Migration tool providing support for Apache Cassandra, DataStax Enterprise Cassandra, & DataStax Enterprise Solr.
Python
37
star
16

elastic-graph-recommender

Building recommenders with Elastic Graph!
JavaScript
37
star
17

elasticsearch-ltr-demo

This demo uses data from TheMovieDB (TMDB) to demonstrate using Ranklib learning to rank models with Elasticsearch.
HTML
36
star
18

lazy-semantic-indexing

Elasticsearch Latent Semantic Indexing experimentation
Python
33
star
19

pdf-discovery-demo

Demonstration of searching PDF document with Solr, Tika, and Tesseract
JavaScript
30
star
20

match-query-parser

Search a single field with different query time analyzers in Solr
Java
25
star
21

splainer-search

Angular JS Solr and Elasticsearch and OpenSearch Diagnostic Search Services
JavaScript
25
star
22

tmdb_dump

Dump TheMovieDB
Python
23
star
23

es-tmdb

Elasticsearch TMDB examples
Python
21
star
24

solr-tmdb

TheMovieDB in Solr
Python
19
star
25

StackExchangeSolrIndexing

AutoTaxonomyExtractionAndTagging
XML
18
star
26

skipchunk

Extracts a latent knowledge graph from text and index/query it in elasticsearch or solr
Python
18
star
27

cfn-solr

Cloud formation script for solr servers
Shell
16
star
28

search-metrics

Python functions for popular relevance metrics (ndcg, err, etc)
Python
15
star
29

solr_angular_demo

A little search widget for instant Solr search with angular
JavaScript
15
star
30

lucene-bm25f

BM25F demo with lucene using BlendedTermQuery and a custom similarity
Java
15
star
31

bearded-wookie

An experiment in visualizing your Solr index via term counts, document counts, and memory usage per field and data type.
CSS
15
star
32

elasticsearch-image-search

Stupid Experiments in Elasticsearch Image Search
Jupyter Notebook
14
star
33

ubi

User Behavior Insights standard schema
13
star
34

solr-movielens-recommender

Movielens collaborative filtering with Solr streaming expression
Python
11
star
35

grand_central

Docker & Kubernetes deployment system for dynamic environments.
Java
11
star
36

agent_q

Headless agent for test driven relevancy with Quepid.com
Ruby
10
star
37

ltr-synth-judg

Experiments in creating synthetic training data for learning to rank
Python
9
star
38

payload-component

Solr component that surfaces payloads for matching terms
Java
9
star
39

goRank

click tracking for creating judgement lists for search-y stuff
Go
8
star
40

puppet-solr

Puppet module for installing solr with a stand alone jetty server
Shell
7
star
41

SolrSwan

SolrSwan is a query parser and highlighter for Solr that accepts proximity and Boolean queries.
Java
7
star
42

semantic-search-course

Semantic Search Course, Originally delivered at Code4Lib
Python
7
star
43

Sample-Spark-Project

Sample Spark project with Scala and SBT
Scala
7
star
44

solr_dump

Dump Solr docs to file; Write dumped docs to a Solr
Python
7
star
45

lucene_codec_hello_world

Starting point and instructions on developing a Lucene Codec
Java
7
star
46

solr-docker

Sample Dockerfiles for running Solr in a container
6
star
47

o19s-lambda

AWS Lambda Functions to make your life easier.
JavaScript
6
star
48

StackExchangeElasticSearch

Playing with ElasticSearch and the SciFi Stackexchange Dataset
Python
6
star
49

highlighting-pdf-viewer

A component (written in Vue) that supports highlighting of words in the PDF document.
Vue
6
star
50

opensearch-ubi

OpenSearch plugin for User Behavior Insights
Java
6
star
51

elasticsearch-vagrant

An ubuntu 14.04 vagrant box running Elasticsearch
Shell
5
star
52

jackhanna

Simple CLI for Zookeeper
Java
5
star
53

tlre-nlp

Materials for "Think Like A Relevance Engineer - NLP" Training
Jupyter Notebook
5
star
54

keel

This gem provides a few easy to run rake tasks to deploy your Rails application to a Kubernetes cluster.
Ruby
5
star
55

bad-libs

πŸ“ Automatically converts any book into a Mad-Libs style game of silliness using spaCy. Free Charles Dickens included!
Jupyter Notebook
4
star
56

elasticsearch-query-builder-example

Basic Elasticsearch Query Builder Plugin
Java
4
star
57

natural-language-search

Colaboratory notebooks for OSC's Natural Language Search training
Jupyter Notebook
4
star
58

word2vec-experiments

Some experimentation with word2vec
Jupyter Notebook
3
star
59

trec-news-index

Index for the TREC Washington Post corpus
Jupyter Notebook
3
star
60

twittalytics

Twitter Analytics with Cassandra
Python
3
star
61

solr-monitor

Java
2
star
62

search-viz

Various experiments demonstrating pairing realtime visualizations with search results.
JavaScript
2
star
63

tm-import

Importing public domain Trademark XML from Google
Go
2
star
64

elasticsearch-heatmap

Java
2
star
65

o19s-blog-ltr

Using the Elasticsearch LTR demo w/ some hand-created judgments
Python
2
star
66

JodaTimeCodecs

A collection of Cassandra TypeCodecs for serializing and deserializing Joda Time objects.
Java
2
star
67

Spark-Cassandra-Demo

Demo code for loading data into Cassandra and Solr with Spark.
Java
2
star
68

trec-podcasts-index

Index Spotify's 100k podcasts dataset into Elasticsearch
Python
2
star
69

ispy_component

Relevance debugging component for Solr
Java
2
star
70

quepid-jupyterlite

Jupyter notebooks to help with search relevancy measurements, optimized for Quepid.
Jupyter Notebook
2
star
71

clustering-lowes-grouts

Code to support a blog post about extracting tags from Lowes.com for clustering unsanded grout search results
JavaScript
2
star
72

visualizing-signals

A Practical Introduction to Exploring and Visualizing E-Commerce Search Signal Data
Shell
2
star
73

solr-query-parser-demo

A "surround"-like and capitalization custom query parsers demo
Java
2
star
74

user-behavior-insights-elasticsearch

User Behavior Insights (UBI) plugin for Elasticsearch
Java
2
star
75

metric-plots

Plots for search metrics nDCG and ERR
JavaScript
1
star
76

jupyter-blogs

Drafts of Doug's Jupyter Notebook Blog Posts
Python
1
star
77

os-tmdb

TLRE OpenSearch
Python
1
star
78

ndoch-trademark-challenge

Applications built for the National Day of Civic Hacking's USPTO Trademarks Challenge
Ruby
1
star
79

movielens-judgments

experiments using movielens genome tags as an experimental ltr training set
Python
1
star
80

training_coms

R scripts to manage bulk training communications and certificate generation
R
1
star
81

jarjar

Joint Analysis Review of Judgements And Raters
Jupyter Notebook
1
star
82

puppet-modules

puppet modules for o19s
Puppet
1
star
83

thats-trackable

Running app for XC team.
Ruby
1
star
84

ggoodggraphics

The grammer of graphics is powerful and now in Python thanks for `plotnine`!
Jupyter Notebook
1
star