Discover nextapps-de/flexsearch-server Open Source project

High-performance FlexSearch Server for Node.js

A full documentation of FlexSearch is available here: https://github.com/nextapps-de/flexsearch

Cluster (Vertical Scaling)

FlexSearch Server actually use Node.js Clusters and a Connection Pool to balance all incoming load through the workers.

The performance gain of workers depends on the complexity of the index and how much time a query takes.

For most situations disabled workers performs a lot better, e.g.:

	Single Thread (Request per seconds)	Cluster (Request per seconds)
Query Time < 10 ms	6122	1215
Query Time > 1000 ms	114	1041

Note: A query time of 10 ms is already based on a large index.

Sharding / Replication (Horizontal Scaling)

Basically the current codebase has all requirements to provide also scaling in horizontal direction. If this project gains popularity I will made an implementation and also provide some docker bindings for an out-of-the-box installation.

Installation

npm install flexsearch-server

Run setup once when not installing via "npm install":

npm run setup

Run as a single web server:

npm start

Run as a server cluster:

npm run cluster

Run with a specified environment:

npm start production

npm run cluster test

The server is listening at:

Environment	Server Address
development	http://localhost
production	http://localhost:6780 https://localhost:6780
test	http://localhost

The default port for production is 6780 (over HTTP) and 6781 (over HTTPS), you can change the default ports in the configs.

When NODE_ENV is set, all manual passed environments will be overridden.

API Overview

RESTful

Request Type	Endpoint	Description
GET	/	Returns info about the index
POST	/add/:id/:content	Add content to the index
POST	/update/:id/:content	Update content of the index
GET	/search/:query	Search for query
POST	/remove/:id	Remove id from the index

JSON

Request Type	Endpoint	Body	Description
POST	/add	JSON	Add multiple contents to the index
POST	/update	JSON	Update multiple contents of the index
GET	/search	JSON	Search for query (custom search)
POST	/remove	JSON	Remove multiple ids from the index

Server Environment Variables

Variable	Values	Default	Description
PORT	integer	80 (development) 6780 (production)	Server listening port
PORT_SSL	integer	443 (development) 6781 (production)	Server listening port
SSL	true false	false	Starts a HTTPS server (additionally to HTTP)
FORCE_SSL	true false	false	Accept HTTPS connections only
COMPRESS	true false	true	Enable/Disable response compression (gzip)
WORKER	integer "auto"	"auto"	Sets the number of workers or automatically determine available cpus
AUTOSAVE	integer false	10000	Enables persistent handler. Set the delay in milliseconds or disable by setting to false

Configuration

When you do not pass any custom options, the environment will be inherited from NODE_ENV by default.

Override the current environment (NODE_ENV) by adding a field in the root of package.json:

  "flexsearch": "production" // development, test

Or add a config payload in the root of package.json:

The field "client" holds the flexsearch options

  "flexsearch": {
    "server": {
      "debug": true,
      "port": 80,
      "port_ssl": 443,
      "force_ssl": false,
      "https": true,
      "compress": true,
      "autosave": 2000,
      "worker": "auto"
    },
    "client": {
      "async": false,
      "cache": 0,
      "worker": 4
    }
  }

Or provide a config json file for each environment respectively and pass file locations:

You can also provide a config json file to override the default settings globally.

  "flexsearch": {
    "development": "config/development.json",
    "production": "config/production.json",
    "test": "config/test",
    "default": "config/default.json"
}

Or just provide a flexsearch.json in the root of your project.

Persistence

Turn on persistence by setting a numeric value as delay on config.autosave. Flexsearch will automatically save the index to the local filesystem and also loads it when starting.

Disable persistence by setting this field to false.

Setup SSL for HTTPS Connections

Inside the folder cert add the certificate as a .crt file and the key as a .pem file for each environment respectively.

Development Environment:

cert/development.crt
cert/development.pem

Production Environment:

cert/production.crt
cert/production.pem

Important Notes

You can increase workers without any drawbacks, but when you decrease worker count, the old index doesnt restore completely from the filesystem. This will be fixed in an upcoming version. As long please do not decrease worker count when using persistent index.

nextapps-de/flexsearch-server

nextapps-de

Reviews

Repository Details