• Stars
    star
    135
  • Rank 263,590 (Top 6 %)
  • Language
    JavaScript
  • License
    Apache License 2.0
  • Created almost 10 years ago
  • Updated 4 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Elasticsearch/Solr Sandbox for exploring explain information and tweaking

The Splainer!

Build Status

The sandbox that explains your search results for you so you don't have to go digging through explain debug! Paste in your Solr or Elasticsearch query URL and go. Try it out here.

Why?

You're a search developer trying to tune search results with Solr or Elasticsearch. You're engaged in search relevancy work.

You're probably stuck with the question of why? Why do search results come back in the order that they do? Solr and Elasticsearch exposes an explain syntax for you to try to explain search scoring. Unfortunately outside the simplest tasks, its a nightmare to read through. There are parsers like explain.solr.pl but they require a lot of manual copy/pasting of explain information to the tool.

Splainer is different by being a sandbox. Paste in your Solr or Elasticsearch URL, query parameters and all. As you work with your query, changing parameters, Splainer shows you parsed and summarized explain information alongside your documents. Continue working and see how the search results change.

Read the blog post introducing Splainer here

Splainer forms the core of the open source tool Quepid that allows you to do this over multiple queries against expert-graded search results to track search changes over a longer period of time.

Using Splainer

Take the tour to see how you'd use Splainer.

Using Splainer locally

We have a Docker image published at https://hub.docker.com/r/o19s/splainer that you can use:

docker run -d -p 9000:9000 o19s/splainer and then go to http://localhost:9000

Developing Splainer

Npm/Yarn Dev Environment

Splainer is written using AngularJS project. It requires npm, grunt, and yarn.

Be sure you've installed npm, yarn, and grunt on your machine.

npm install -g grunt-cli

With Npm/Yarn installed

From the root of the project, you should be able to run the following:

yarn
grunt test
grunt serve

Now browse to http://localhost:9000.

To build the project, simply run grunt dist to build the static artifacts in the dist/ folder.

grunt dist

You can test out the static artifacts via ruby -run -e httpd -- -p 5000 ./dist and going to http://localhost:5000.

With Docker installed

From the root of the project, you should run:

docker build -t splainer  .
docker run -p 9000:9000 splainer:latest

or use the following shortcuts if you have ruby installed:

bin/docker b

then to run the server run

bin/docker s

Using docker-compose

From the root of the project,

docker-compose build
docker-compose run --rm --service-ports app

Using Docker Compose to test splainer-search with splainer

  • docker-compose.override.yml.example can be copied to docker-compose.override.yml and use it to override environment variables or work with a local copy of the splainer-search JS library during development defined in docker-compose.yml. Example is included. Just update the path to splainer-search with your local checkout!

https://docs.docker.com/compose/extends/

Testing Notes

  • Unit tests are written using Karma.

  • The ./tests/splainer_test_links.html file is a list of links that invoke Splainer, both the local version and the deployed version against Solr and Elasticsearch, and is a great test to make sure the behavior hasn't reverted. Use this to make sure existing links still work!

Who?

Created by OpenSource Connections.

Thanks to all the community contributors for finding bugs and sharing fixes!.

License

Released under Apache 2

More Repositories

1

elasticsearch-learning-to-rank

Plugin to integrate Learning to Rank (aka machine learning for better relevance) with Elasticsearch
Java
1,466
star
2

relevant-search-book

Code and Examples for Relevant Search
Jupyter Notebook
292
star
3

quepid

Improve your Elasticsearch, OpenSearch, Solr, Vectara, Algolia and Custom Search search quality.
Ruby
271
star
4

hello-ltr

Set of Jupyter notebooks demonstrating Learning to Rank integrated with Solr and Elasticsearch
Jupyter Notebook
156
star
5

elyzer

"Stop worrying about Elasticsearch analyzers", my therapist says
Python
153
star
6

hello-nlp

A natural language search microservice
Python
94
star
7

awesome-search-relevance

Tools and other things for people who work on search relevance & information retrieval
81
star
8

Spyglass

Simple search results with Solr and EmberJS
JavaScript
58
star
9

solr-to-es

Migrate a Solr node to an Elasticsearch index.
Python
53
star
10

lucene-query-example

Educational Examle of a custom Lucene Query & Scorer
Java
48
star
11

solr_nginx

Starter Reverse Proxy Configuration for Solr
47
star
12

SemanticSearchInNumpy

XSLT
44
star
13

hangry

Vector search in Lucene based search attempting to use just the existing Lucene data structures (experimental)
Java
43
star
14

RankyMcRankFace

Hardened Fork of Ranklib learning to rank library
Java
43
star
15

trireme

Migration tool providing support for Apache Cassandra, DataStax Enterprise Cassandra, & DataStax Enterprise Solr.
Python
37
star
16

elastic-graph-recommender

Building recommenders with Elastic Graph!
JavaScript
37
star
17

elasticsearch-ltr-demo

This demo uses data from TheMovieDB (TMDB) to demonstrate using Ranklib learning to rank models with Elasticsearch.
HTML
36
star
18

lazy-semantic-indexing

Elasticsearch Latent Semantic Indexing experimentation
Python
33
star
19

pdf-discovery-demo

Demonstration of searching PDF document with Solr, Tika, and Tesseract
JavaScript
29
star
20

match-query-parser

Search a single field with different query time analyzers in Solr
Java
25
star
21

splainer-search

Angular JS Solr and Elasticsearch and OpenSearch Diagnostic Search Services
JavaScript
25
star
22

tmdb_dump

Dump TheMovieDB
Python
21
star
23

es-tmdb

Elasticsearch TMDB examples
Python
20
star
24

solr-tmdb

TheMovieDB in Solr
Python
19
star
25

skipchunk

Extracts a latent knowledge graph from text and index/query it in elasticsearch or solr
Python
19
star
26

StackExchangeSolrIndexing

AutoTaxonomyExtractionAndTagging
XML
18
star
27

cfn-solr

Cloud formation script for solr servers
Shell
16
star
28

solr_angular_demo

A little search widget for instant Solr search with angular
JavaScript
15
star
29

lucene-bm25f

BM25F demo with lucene using BlendedTermQuery and a custom similarity
Java
15
star
30

bearded-wookie

An experiment in visualizing your Solr index via term counts, document counts, and memory usage per field and data type.
CSS
15
star
31

search-metrics

Python functions for popular relevance metrics (ndcg, err, etc)
Python
14
star
32

elasticsearch-image-search

Stupid Experiments in Elasticsearch Image Search
Jupyter Notebook
14
star
33

solr-movielens-recommender

Movielens collaborative filtering with Solr streaming expression
Python
11
star
34

grand_central

Docker & Kubernetes deployment system for dynamic environments.
Java
11
star
35

agent_q

Headless agent for test driven relevancy with Quepid.com
Ruby
10
star
36

ltr-synth-judg

Experiments in creating synthetic training data for learning to rank
Python
9
star
37

payload-component

Solr component that surfaces payloads for matching terms
Java
9
star
38

goRank

click tracking for creating judgement lists for search-y stuff
Go
8
star
39

puppet-solr

Puppet module for installing solr with a stand alone jetty server
Shell
7
star
40

semantic-search-course

Semantic Search Course, Originally delivered at Code4Lib
Python
7
star
41

Sample-Spark-Project

Sample Spark project with Scala and SBT
Scala
7
star
42

solr_dump

Dump Solr docs to file; Write dumped docs to a Solr
Python
7
star
43

lucene_codec_hello_world

Starting point and instructions on developing a Lucene Codec
Java
7
star
44

SolrSwan

SolrSwan is a query parser and highlighter for Solr that accepts proximity and Boolean queries.
Java
6
star
45

solr-docker

Sample Dockerfiles for running Solr in a container
6
star
46

o19s-lambda

AWS Lambda Functions to make your life easier.
JavaScript
6
star
47

StackExchangeElasticSearch

Playing with ElasticSearch and the SciFi Stackexchange Dataset
Python
6
star
48

highlighting-pdf-viewer

A component (written in Vue) that supports highlighting of words in the PDF document.
Vue
6
star
49

elasticsearch-vagrant

An ubuntu 14.04 vagrant box running Elasticsearch
Shell
5
star
50

jackhanna

Simple CLI for Zookeeper
Java
5
star
51

keel

This gem provides a few easy to run rake tasks to deploy your Rails application to a Kubernetes cluster.
Ruby
5
star
52

bad-libs

📝 Automatically converts any book into a Mad-Libs style game of silliness using spaCy. Free Charles Dickens included!
Jupyter Notebook
4
star
53

elasticsearch-query-builder-example

Basic Elasticsearch Query Builder Plugin
Java
4
star
54

natural-language-search

Colaboratory notebooks for OSC's Natural Language Search training
Jupyter Notebook
4
star
55

opensearch-ubi

OpenSearch plugin for User Behavior Insights
Java
4
star
56

word2vec-experiments

Some experimentation with word2vec
Jupyter Notebook
3
star
57

trec-news-index

Index for the TREC Washington Post corpus
Jupyter Notebook
3
star
58

twittalytics

Twitter Analytics with Cassandra
Python
3
star
59

tlre-nlp

Materials for "Think Like A Relevance Engineer - NLP" Training
Jupyter Notebook
3
star
60

solr-monitor

Java
2
star
61

search-viz

Various experiments demonstrating pairing realtime visualizations with search results.
JavaScript
2
star
62

tm-import

Importing public domain Trademark XML from Google
Go
2
star
63

elasticsearch-heatmap

Java
2
star
64

o19s-blog-ltr

Using the Elasticsearch LTR demo w/ some hand-created judgments
Python
2
star
65

JodaTimeCodecs

A collection of Cassandra TypeCodecs for serializing and deserializing Joda Time objects.
Java
2
star
66

Spark-Cassandra-Demo

Demo code for loading data into Cassandra and Solr with Spark.
Java
2
star
67

trec-podcasts-index

Index Spotify's 100k podcasts dataset into Elasticsearch
Python
2
star
68

ispy_component

Relevance debugging component for Solr
Java
2
star
69

clustering-lowes-grouts

Code to support a blog post about extracting tags from Lowes.com for clustering unsanded grout search results
JavaScript
2
star
70

visualizing-signals

A Practical Introduction to Exploring and Visualizing E-Commerce Search Signal Data
Shell
2
star
71

solr-query-parser-demo

A "surround"-like and capitalization custom query parsers demo
Java
2
star
72

rated-ranking-evaluator

Search Quality Evaluation Tool for Apache Solr & Elasticsearch search-based infrastructures
Java
2
star
73

metric-plots

Plots for search metrics nDCG and ERR
JavaScript
1
star
74

jupyter-blogs

Drafts of Doug's Jupyter Notebook Blog Posts
Python
1
star
75

os-tmdb

TLRE OpenSearch
Python
1
star
76

quepid-jupyterlite

Jupyter notebooks to help with search relevancy measurements, optimized for Quepid.
Jupyter Notebook
1
star
77

ndoch-trademark-challenge

Applications built for the National Day of Civic Hacking's USPTO Trademarks Challenge
Ruby
1
star
78

movielens-judgments

experiments using movielens genome tags as an experimental ltr training set
Python
1
star
79

training_coms

R scripts to manage bulk training communications and certificate generation
R
1
star
80

jarjar

Joint Analysis Review of Judgements And Raters
Jupyter Notebook
1
star
81

puppet-modules

puppet modules for o19s
Puppet
1
star
82

thats-trackable

Running app for XC team.
Ruby
1
star
83

ggoodggraphics

The grammer of graphics is powerful and now in Python thanks for `plotnine`!
Jupyter Notebook
1
star