• Stars
    star
    241
  • Rank 167,643 (Top 4 %)
  • Language
    PHP
  • Created over 5 years ago
  • Updated 11 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

🔍 JoliCode's Elastica wrapper to bootstrap Elasticsearch PHP integrations

Elastically, Elastica based framework

CI

Opinionated Elastica based framework to bootstrap PHP and Elasticsearch / OpenSearch implementations.

Main features:

  • DTO are first class citizen, you send PHP object as documents, and get objects back on search results, like an ODM;
  • All indexes are versioned and aliased automatically;
  • Mappings are done via YAML files, PHP or custom via MappingProviderInterface;
  • Analysis is separated from mappings to ease reuse;
  • 100% compatibility with ruflin/elastica;
  • Mapping migration capabilities with ReIndex;
  • Symfony HttpClient compatible transport (optional);
  • Symfony support (optional):
    • See dedicated chapter;
    • Tested with Symfony 5.4 to 6;
    • Symfony Messenger Handler support (with or without spool);

Require PHP 8.0+ and Elasticsearch 7+.

Works with Elasticsearch 8+ but is not officially supported by Elastica yet. Use with caution.

Works with OpenSearch 1 and 2.

You can check the changelog and the upgrade documents.

Installation

composer require jolicode/elastically

Demo

If you are using Symfony, you can move to the Symfony chapter

Quick example of what the library do on top of Elastica:

// Your own DTO, or one generated by Jane (see below)
class Beer
{
    public string $foo;
    public string $bar;
}

use JoliCode\Elastically\Factory;
use JoliCode\Elastically\Model\Document;

// Factory object with Elastica options + new Elastically options in the same array
$factory = new Factory([
    // Where to find the mappings
    Factory::CONFIG_MAPPINGS_DIRECTORY => __DIR__.'/mappings',
    // What object to find in each index
    Factory::CONFIG_INDEX_CLASS_MAPPING => [
        'beers' => Beer::class,
    ],
]);

// Class to perform request, same as the Elastica Client
$client = $factory->buildClient();

// Class to build Indexes
$indexBuilder = $factory->buildIndexBuilder();

// Create the Index in Elasticsearch
$index = $indexBuilder->createIndex('beers');

// Set the proper aliases
$indexBuilder->markAsLive($index, 'beers');

// Class to index DTO(s) in an Index
$indexer = $factory->buildIndexer();

$dto = new Beer();
$dto->bar = 'American Pale Ale';
$dto->foo = 'Hops from Alsace, France';

// Add a document to the queue
$indexer->scheduleIndex('beers', new Document('123', $dto));
$indexer->flush();

// Set parameters on the Bulk
$indexer->setBulkRequestParams([
    'pipeline' => 'covfefe',
    'refresh' => 'wait_for'
]);

// Force index refresh if needed
$indexer->refresh('beers');

// Get the Document (new!)
$results = $client->getIndex('beers')->getDocument('123');

// Get the DTO (new!)
$results = $client->getIndex('beers')->getModel('123');

// Perform a search
$results = $client->getIndex('beers')->search('alsace');

// Get the Elastic Document
$results->getDocuments()[0];

// Get the Elastica compatible Result
$results->getResults()[0];

// Get the DTO 🎉 (new!)
$results->getResults()[0]->getModel();

// Create a new version of the Index "beers"
$index = $indexBuilder->createIndex('beers');

// Slow down the Refresh Interval of the new Index to speed up indexation
$indexBuilder->slowDownRefresh($index);
$indexBuilder->speedUpRefresh($index);

// Set proper aliases
$indexBuilder->markAsLive($index, 'beers');

// Clean the old indices (close the previous one and delete the older)
$indexBuilder->purgeOldIndices('beers');

// Mapping change? Just call migrate and enjoy a full reindex (use the Task API internally to avoid timeout)
$newIndex = $indexBuilder->migrate($index);
$indexBuilder->speedUpRefresh($newIndex);
$indexBuilder->markAsLive($newIndex, 'beers');

mappings/beers_mapping.yaml

# Anything you want, no validation
settings:
    number_of_replicas: 1
    number_of_shards: 1
    refresh_interval: 60s
mappings:
    dynamic: false
    properties:
        foo:
            type: text
            analyzer: english
            fields:
                keyword:
                    type: keyword

Configuration

This library add custom configurations on top of Elastica's:

Factory::CONFIG_MAPPINGS_DIRECTORY (required with default configuration)

The directory Elastically is going to look for YAML.

When creating a foobar index, a foobar_mapping.yaml file is expected.

If an analyzers.yaml file is present, all the indices will get it.

Factory::CONFIG_INDEX_CLASS_MAPPING (required)

An array of index name to class FQN.

[
  'indexName' => My\AwesomeDTO::class,
]

Factory::CONFIG_MAPPINGS_PROVIDER

An instance of MappingProviderInterface.

If this option is not defined, the factory will fallback to YamlProvider and will use Factory::CONFIG_MAPPINGS_DIRECTORY option.

There are two providers available in Elastically: YamlProvider and PhpProvider.

Factory::CONFIG_SERIALIZER (optional)

A SerializerInterface compatible object that will by used on indexation.

Default to Symfony Serializer with Object Normalizer.

A faster alternative is to use Jane to generate plain PHP Normalizer, see below. Also we recommend customization to handle things like Date.

Factory::CONFIG_DENORMALIZER (optional)

A DenormalizerInterface compatible object that will by used on search results to build your objects back.

If this option is not defined, the factory will fallback to Factory::CONFIG_SERIALIZER option.

Factory::CONFIG_SERIALIZER_CONTEXT_BUILDER (optional)

An instance of ContextBuilderInterface that build a serializer context from a class name.

If it is not defined, Elastically, will use a StaticContextBuilder with the configuration from Factory::CONFIG_SERIALIZER_CONTEXT_PER_CLASS.

Factory::CONFIG_SERIALIZER_CONTEXT_PER_CLASS (optional)

Allow to specify the Serializer context for normalization and denormalization.

[
    Beer::class => ['attributes' => ['title']],
];

Default to [].

Factory::CONFIG_BULK_SIZE (optional)

When running indexation of lots of documents, this setting allow you to fine-tune the number of document threshold.

Default to 100.

Factory::CONFIG_INDEX_PREFIX (optional)

Add a prefix to all indexes and aliases created via Elastically.

Default to null.

Usage in Symfony

Configuration

You'll need to add the bundle in bundles.php:

// config/bundles.php
return [
    // ...
    JoliCode\Elastically\Bridge\Symfony\ElasticallyBundle::class => ['all' => true],
];

Then configure the bundle:

# config/packages/elastically.yaml
elastically:
    connections:
        default:
            client:
                host:                '%env(ELASTICSEARCH_HOST)%'
                # If you want to use the Symfony HttpClient (you MUST create this service)
                #transport:           'JoliCode\Elastically\Transport\HttpClientTransport'

            # Path to the mapping directory (in YAML)
            mapping_directory:       '%kernel.project_dir%/config/elasticsearch'

            # Size of the bulk sent to Elasticsearch (default to 100)
            bulk_size:               100

            # Mapping between an index name and a FQCN
            index_class_mapping:
                my-foobar-index:     App\Dto\Foobar

            # Configuration for the serializer
            serializer:
                # Fill a static context
                context_mapping:
                    foo:                 bar

            # If you want to add a prefix for your index in elasticsearch (you can still call it by its base name everywhere!)
            # prefix: '%kernel.environment%'

Finally, inject one of those service (autowirable) in you code where you need it:

JoliCode\Elastically\Client (elastically.default.client)
JoliCode\Elastically\IndexBuilder (elastically.default.index_builder)
JoliCode\Elastically\Indexer (elastically.default.indexer)

Advanced Configuration

Multiple Connections and Autowiring

If you define multiple connections, you can define a default one. This will be useful for autowiring:

elastically:
    default_connection: default
    connections:
        default: # ...
        another: # ...

To use class for other connection, you can use Autowirable Types. To discover them, run:

bin/console debug:autowiring elastically
Use a Custom Serializer Context Builder
elastically:
    default_connection: default
    connections:
        default:
            serializer:
                context_builder_service: App\Elastically\Serializer\ContextBuilder
                # Do not defined "context_mapping" option anymore
Use a Custom Mapping provider
elastically:
    default_connection: default
    connections:
        default:
            mapping_provider_service: App\Elastically\MappingProvider
            # Do not defined "index_class_mapping" option anymore
Using HttpClient as Transport

You can also use the Symfony HttpClient for all Elastica communications:

JoliCode\Elastically\Transport\HttpClientTransport: ~

JoliCode\Elastically\Client:
    arguments:
        $config:
            host: '%env(ELASTICSEARCH_HOST)%'
            transport: 'JoliCode\Elastically\Transport\HttpClientTransport'
            ...

Reference

You can run the following command to get the default configuration reference:

bin/console config:dump elastically

Using Messenger for async indexing

Elastically ships with a default Message and Handler for Symfony Messenger.

Register the message in your configuration:

framework:
    messenger:
        transports:
            async: "%env(MESSENGER_TRANSPORT_DSN)%"

        routing:
            # async is whatever name you gave your transport above
            'JoliCode\Elastically\Messenger\IndexationRequest':  async

services:
    JoliCode\Elastically\Messenger\IndexationRequestHandler: ~

The IndexationRequestHandler service depends on an implementation of JoliCode\Elastically\Messenger\DocumentExchangerInterface, which isn't provided by this library. You must provide a service that implements this interface, so you can plug your database or any other source of truth.

Then from your code you have to call:

use JoliCode\Elastically\Messenger\IndexationRequest;
use JoliCode\Elastically\Messenger\IndexationRequestHandler;

$bus->dispatch(new IndexationRequest(Product::class, '1234567890'));

// Third argument is the operation, so for a delete:
// new IndexationRequest(Product::class, 'ref9999', IndexationRequestHandler::OP_DELETE);

And then consume the messages:

php bin/console messenger:consume async

Grouping IndexationRequest in a spool

Sending multiple IndexationRequest during the same Symfony Request is not always appropriate, it will trigger multiple Bulk operations. Elastically provides a Kernel listener to group all the IndexationRequest in a single MultipleIndexationRequest message.

To use this mechanism, we send the IndexationRequest in a memory transport to be consumed and grouped in a really async transport:

messenger:
    transports:
        async: "%env(MESSENGER_TRANSPORT_DSN)%"
        queuing: 'in-memory:///'

    routing:
        'JoliCode\Elastically\Messenger\MultipleIndexationRequest': async
        'JoliCode\Elastically\Messenger\IndexationRequest': queuing

You also need to register the subscriber:

services:
    JoliCode\Elastically\Messenger\IndexationRequestSpoolSubscriber:
        arguments:
            - '@messenger.transport.queuing' # should be the name of the memory transport
            - '@messenger.default_bus'
        tags:
            - { name: kernel.event_subscriber }

Using Jane to build PHP DTO and fast Normalizers

Install JanePHP json-schema tools to build your own DTO and Normalizers. All you have to do is setting the Jane-completed Serializer on the Factory:

$factory = new Factory([
    Factory::CONFIG_SERIALIZER => $serializer,
]);

Not compatible with Jane < 6.

To be done

  • some "todo" in the code
  • optional Doctrine connector
  • better logger - maybe via a processor? extending _log is supposed to be deprecated :(
  • extra commands to monitor, update mapping, reindex... Commonly implemented tasks
  • optional Symfony integration:
    • web debug toolbar!
  • scripts / commands for common tasks:
    • auto-reindex when the mapping change, handle the aliases and everything
    • micro monitoring for cluster / indexes
    • health-check method

Sponsors

JoliCode

Open Source time sponsored by JoliCode.

More Repositories

1

JoliNotif

💻 Send notifications to your desktop directly from your PHP script
PHP
1,324
star
2

JoliCi

✅ JoliCi - Run your TravisCi builds locally
PHP
657
star
3

castor

🦫 DX oriented task runner and command launcher built with PHP.
PHP
406
star
4

elasticsearch-cheatsheet

🔎 Elasticsearch is awesome, here is a cheatsheet for it.
HTML
339
star
5

docker-starter

🏗️ A skeleton to start a new web project with PHP, Docker and Castor
PHP
335
star
6

JoliTypo

🔤 Microtypography fixer for the web
PHP
318
star
7

slack-php-api

#️⃣ PHP Slack Client based on the official OpenAPI specification
PHP
219
star
8

emoji-search

😄 Emoji synonyms to build your own emoji-capable search engine (elasticsearch, solr, OpenSearch)
PHP
215
star
9

secret-santa

🎅 The code behind Secret Santa, the holiday bot for Slack / Discord / Webex
PHP
213
star
10

GifExceptionBundle

😛 The GhostBuster of your exception page!
PHP
205
star
11

php-ar-drone

🚁 Port of node-ar-drone which allows user to control a Parrot AR Drone over PHP
PHP
204
star
12

composer-cheatsheet

📋 Everything you have to know about composer.json in one page.
HTML
146
star
13

automapper

🚀 Very FAST 🚀 PHP AutoMapper with on the fly code generation
PHP
130
star
14

generator-joli-symfony

👨 Yeoman Generator for Symfony2 projects with sensible defaults and frontend tools.
JavaScript
100
star
15

asynit

🌠 Asynchronous HTTP Request Testing Library for API or more...
PHP
77
star
16

Alloy-PullToRefresh

Pull to refresh widget for Titanium Alloy applications.
JavaScript
55
star
17

Badass-Pageflow

📲 A simple Alloy "pageflow" widget, which allows to open windows w/ styles back buttons. Features a complete a simple API.
JavaScript
44
star
18

ffi-uuid

Binding of the libuuid library with PHP thanks to PHP/FFI.
PHP
39
star
19

docker-images

🚢 Basic images for different usages
Makefile
33
star
20

pomdok

🍏 Simple wrapper to Symfony Go Binary for multi-app
Go
27
star
21

qotd

PHP
25
star
22

codingstyle

💅 JoliCode's base dotfile for PHP / JS projects
PHP
24
star
23

chef-cookbook-php

A Chef Cookbook for PHP, does not depend on apache and will not use or install pear like the official one.
Ruby
23
star
24

harvest-php-api

🌾 A Harvest API PHP Client
PHP
22
star
25

starfleet

🚀 Share your conferences activity to your buddies
PHP
20
star
26

php7-checker

☑️ PHP7 checker
PHP
18
star
27

Harvest-Forecast-tools

📅 Some useful additions to https://www.getharvest.com/ and https://www.getharvest.com/forecast
JavaScript
16
star
28

symfony-jwt-article

PHP
16
star
29

seo-override

🏁 Override your SEO related markup on the fly
PHP
14
star
30

harvest-openapi-generator

🔮 Transforms Harvest API HTML documentation pages into a valid Swagger / OpenAPI 3.0 specification
PHP
13
star
31

JoliTypoBundle

🔤 Integration of JoliTypo for Symfony2 (deprecated, use the provided bridge instead)
PHP
13
star
32

forecast-tools

⛅ Tools to get the most out of Harvest Forecast. Get Slack notifications, schedule Slack stand-up meetings, share public Forecasts, and more
PHP
13
star
33

JoliMarkdown

✍ A syntax fixer for markdown content
PHP
12
star
34

Symfony2BackboneDemo

JavaScript
12
star
35

symfony-security-article

PHP
12
star
36

php-os-helper

Provides helpers to detect OS of the machine where PHP is running.
PHP
12
star
37

monologue

A bot to manage "monologue" rule in a Slack Channel
PHP
10
star
38

ApacheTikaBundle

📁 Symfony Bundle for https://github.com/vaites/php-apache-tika
PHP
10
star
39

elasticsearch-php-benchmark

Benchmark of some PHP Clients for Elasticsearch
PHP
10
star
40

Reepo

Abstraction of repository providers (github, gitlab, redmine, ...)
PHP
9
star
41

symfony2-eventdispatcher-extension

Symfony2 Event Dispatcher as PHP extension written with Zephir
C
9
star
42

JoliToken

🔍 Elasticsearch plugin to visualize field tokens as analyzed by Lucene
JavaScript
9
star
43

isready-symfony2

Symfony2 production checklist for http://isready.org
7
star
44

best-bundle-conf

Symfony Live Paris 2013 talk about the best and unknown Symfony2 Bundles (in French)
JavaScript
7
star
45

JoliSnap

JavaScript
6
star
46

100-async-0-callback-conf

✨ https://jolicode.github.io/100-async-0-callback-conf/#/0
JavaScript
6
star
47

webhook-demo

PHP
6
star
48

unicode-conf

😋 Unicode, PHP et la sécurité
JavaScript
5
star
49

forecast-php-api

⛅ A Forecastapp API PHP Client
PHP
5
star
50

GouvCamp-mobile

Mobile app for the GouvCamp 2012
JavaScript
5
star
51

SecurityBundle-avec-de-l-aspirine

JavaScript
4
star
52

react-et-symfony-conf

💍 Marier React et Symfony
HTML
4
star
53

value-object-conf

💻 Slides from Forum PHP 2015 talk about Value Object
HTML
4
star
54

everything-titanium-using-the-cli-talk

Slides of the TiConf Amsterdam 2014 about using CLI tools in the Titanium world.
JavaScript
4
star
55

symfony-htmx-demo

Small Symfony project with HTMX and AssetMapper
PHP
3
star
56

lab-webgl-home

Jolicode website homepage in 3D
JavaScript
3
star
57

jolitypo-website

✍️ Sample app to testdrive JoliTypo
Twig
3
star
58

tabto

➡️ Auto-tabulation for your inputs. Tiny helper that focus user on the next field when a field reach maxLength.
JavaScript
3
star
59

php-the-wrong-way-conf

⛔ PHP The Wrong Way
PHP
3
star
60

docker-drupal-meetup-conf

CSS
2
star
61

php7cc-conf

💻 A la recherche d'incompatibilités avec PHP 7, Apéro PHP conf by @pyrech
JavaScript
2
star
62

http-cache-conf

JavaScript
2
star
63

smile-php

🙂
PHP
2
star
64

fosuserbundle-conf

💻 Do Not Use FOSUserBundle conference
JavaScript
2
star
65

zephir-conf

Introduction about Zephir in french
JavaScript
2
star
66

typography-conf

Sud Web 2013 - Damien ALEXANDRE slides about space and french typography
JavaScript
1
star
67

drupal-rache-conf

Drupal Meetup Paris mai 2013 – Bastien Jaillot slides "Drupal au secours de la méthode R.A.C.H.E"
JavaScript
1
star
68

Badass-Pageflow-Demo

Demo App for the Badass-Pageflow alloy widget
JavaScript
1
star
69

phptour-2014--dette-technique--conf

Conférence "Prévenez la dette technique de vos projets" au PHP Tour Lyon 2014
CSS
1
star
70

phptour2014-drink

1
star
71

phptour2014-burger

PHP
1
star
72

ca-marche-chez-moi

Sources des slides de la conférence "Chez moi ça marche" - PHP Tour 2012
CSS
1
star
73

sud-web--2014--dette-technique

Conférence sur la dette technique à SudWeb 2014
CSS
1
star