Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add index for all filterable, sortable and searchable fields #68

Open
wants to merge 3 commits into
base: 0.6
Choose a base branch
from
Open

Add index for all filterable, sortable and searchable fields #68

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
fa3384f
Add index for all filterable, sortable and searchable fields
alexander-schranz Jan 10, 2023
c478906
Update not searchable field test to fail on elastic and opensearch co…
alexander-schranz Jan 10, 2023
7f8d101
Add doc about elasticsearch mapping
alexander-schranz Jan 11, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 16 additions & 7 deletions packages/seal-elasticsearch-adapter/ElasticsearchSchemaManager.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,36 +68,45 @@ private function createPropertiesMapping(array $fields): array
$properties = [];

foreach ($fields as $name => $field) {
// TODO recheck doc_values https://github.com/schranz-search/schranz-search/issues/65
$index = $field->searchable || $field->filterable || $field->sortable;

match (true) {
$field instanceof Field\IdentifierField => $properties[$name] = [
'type' => 'keyword',
'index' => $field->searchable,
'index' => $index,
'doc_values' => $field->filterable,
],
$field instanceof Field\TextField => $properties[$name] = \array_replace([
'type' => 'text',
'index' => $field->searchable,
'index' => $index,
], ($field->filterable || $field->sortable) ? [
'fields' => [
'raw' => ['type' => 'keyword'],
'raw' => [
'type' => 'keyword',
'index' => $field->searchable || $field->filterable,
'doc_values' => $field->filterable,
],
],
] : []),
$field instanceof Field\BooleanField => $properties[$name] = [
'type' => 'boolean',
'index' => $field->searchable,
'index' => $index,
'doc_values' => $field->filterable,
],
$field instanceof Field\DateTimeField => $properties[$name] = [
'type' => 'date',
'index' => $field->searchable,
'index' => $index,
'doc_values' => $field->filterable,
],
$field instanceof Field\IntegerField => $properties[$name] = [
'type' => 'integer',
'index' => $field->searchable,
'index' => $index,
'doc_values' => $field->filterable,
],
$field instanceof Field\FloatField => $properties[$name] = [
'type' => 'float',
'index' => $field->searchable,
'index' => $index,
'doc_values' => $field->filterable,
],
$field instanceof Field\ObjectField => $properties[$name] = [
Expand Down
225 changes: 225 additions & 0 deletions packages/seal-elasticsearch-adapter/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,3 +49,228 @@ $engine = new Engine(
$schema,
);
```

## Schema Mapping

This shows how the [Seal Schema](../seal/README.md#schema) is mapped to Elasticsearch.

There is no diff between sortable and filterable fields so that cases are not specific listed.

| searchable | filterable / sortable | description |
|------------|-----------------------|-----------------------------------------------------------------------------|
| yes | no | Searchable but can not be used for filter or sorting |
| no | yes | Is not keep in mind for search score but can be used for filter and sorting |
| yes | yes | Is keep in mind for search score and can be used for filter and sorting |
| no | no | Just stored no index, analyzing required |

### IdentifierField

Field:

```php
new IdentifierField('id');
```

Mapping:

```json
{
"id": {
"type": "keyword",
"index": true,
"doc_values": true
}
}
```

Also the identifier is used as elasticsearch internal `_id` field.

### TextField

Field:

```php
new TextField('text_a', searchable: true, filterable: false, sortable: false), // default
new TextField('text_b', searchable: false, filterable: true, sortable: true),
```

Mapping:

```json
{
"text_a": {
"type": "text",
"index": true,
"doc_values": false
},
"text_b": {
"type": "text",
"index": true,
"doc_values": true,
"field": {
"raw": {
"type": "keyword",
"index": true,
"doc_values": true
}
}
}
}
```

### BooleanField

Field:

```php
new BooleanField('bool_a', searchable: true, filterable: false, sortable: false), // default
new BooleanField('bool_b', searchable: false, filterable: true, sortable: true),
```

Mapping:

```json
{
"bool_a": {
"type": "boolean",
"index": true,
"doc_values": false
},
"bool_b": {
"type": "boolean",
"index": true,
"doc_values": true
}
}
```

### DateTimeField

Field:

```php
new DateTimeField('date_a', searchable: true, filterable: false, sortable: false), // default
new DateTimeField('date_b', searchable: false, filterable: true, sortable: true),
```

Mapping:

```json
{
"date_a": {
"type": "date",
"index": true,
"doc_values": false
},
"date_b": {
"type": "date",
"index": true,
"doc_values": true
}
}
```

### IntegerField

Field:

```php
new IntegerField('int_a', searchable: true, filterable: false, sortable: false), // default
new IntegerField('int_b', searchable: false, filterable: true, sortable: true),
```

Mapping:

```json
{
"int_a": {
"type": "integer",
"index": true,
"doc_values": false
},
"int_b": {
"type": "integer",
"index": true,
"doc_values": true
}
}
```

### FloatField

Field:

```php
new FloatField('float_a', searchable: true, filterable: false, sortable: false), // default
new FloatField('float_b', searchable: false, filterable: true, sortable: true),
```

Mapping:

```json
{
"float_a": {
"type": "float",
"index": true,
"doc_values": false
},
"float_b": {
"type": "float",
"index": true,
"doc_values": true
}
}
```

### ObjectField

Field:

```php
new ObjectField('object', /* ... */);
```

Mapping:

```json
{
"object": {
"type": "object",
"properties": {
/* ... */
}
}
}
```

### TypedField

Field:

```php
new TypedField('typed', 'type', ['type_a' => /* ... */, 'type_b' => /* ... */]);
```

Mapping:

```json
{
"typed": {
"type": "object",
"properties": {
"type_a": {
"type": "object",
"properties": {
/* ... */
}
},
"type_b": {
"type": "object",
"properties": {
/* ... */
}
}
}
}
}
```
16 changes: 11 additions & 5 deletions packages/seal-elasticsearch-adapter/Tests/ElasticsearchSchemaManagerTest.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,6 @@ public function testSimpleElasticsearchMapping(): void
$this->assertSame([
'id' => [
'type' => 'keyword',
'index' => false,
],
'title' => [
'type' => 'text',
Expand Down Expand Up @@ -94,7 +93,6 @@ public function testComplexElasticsearchMapping(): void
],
'categoryIds' => [
'type' => 'integer',
'index' => false,
],
'comments' => [
'properties' => [
Expand All @@ -109,7 +107,6 @@ public function testComplexElasticsearchMapping(): void
],
'commentsCount' => [
'type' => 'integer',
'index' => false,
],
'created' => [
'type' => 'date',
Expand Down Expand Up @@ -142,9 +139,19 @@ public function testComplexElasticsearchMapping(): void
],
],
],
'internalTags' => [
'type' => 'text',
'fields' => [
'raw' => [
'type' => 'keyword',
],
],
],
'published' => [
'type' => 'date',
],
'rating' => [
'type' => 'float',
'index' => false,
],
'tags' => [
'type' => 'text',
Expand All @@ -159,7 +166,6 @@ public function testComplexElasticsearchMapping(): void
],
'uuid' => [
'type' => 'keyword',
'index' => false,
],
], $mapping[$index->name]['mappings']['properties']);
}
Expand Down
30 changes: 17 additions & 13 deletions packages/seal-opensearch-adapter/OpensearchSchemaManager.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,43 +66,47 @@ private function createPropertiesMapping(array $fields): array
$properties = [];

foreach ($fields as $name => $field) {
// TODO recheck doc_values https://github.com/schranz-search/schranz-search/issues/65
$index = $field->searchable || $field->filterable || $field->sortable;
$doc_values = $field->filterable || $field->sortable;

match (true) {
$field instanceof Field\IdentifierField => $properties[$name] = [
'type' => 'keyword',
'index' => $field->searchable || $field->filterable, // TODO recheck doc_values https://github.com/schranz-search/schranz-search/issues/65
'doc_values' => $field->filterable,
'index' => $index,
'doc_values' => $doc_values,
],
$field instanceof Field\TextField => $properties[$name] = \array_replace([
'type' => 'text',
'index' => $field->searchable || $field->filterable, // TODO recheck doc_values https://github.com/schranz-search/schranz-search/issues/65
'index' => $index,
], ($field->filterable || $field->sortable) ? [
'fields' => [
'raw' => [
'type' => 'keyword',
'index' => $field->searchable || $field->filterable, // TODO recheck doc_values https://github.com/schranz-search/schranz-search/issues/65
'doc_values' => $field->filterable,
'index' => true,
'doc_values' => true,
],
],
] : []),
$field instanceof Field\BooleanField => $properties[$name] = [
'type' => 'boolean',
'index' => $field->searchable || $field->filterable, // TODO recheck doc_values https://github.com/schranz-search/schranz-search/issues/65
'doc_values' => $field->filterable,
'index' => $index,
'doc_values' => $doc_values,
],
$field instanceof Field\DateTimeField => $properties[$name] = [
'type' => 'date',
'index' => $field->searchable || $field->filterable, // TODO recheck doc_values https://github.com/schranz-search/schranz-search/issues/65
'doc_values' => $field->filterable,
'index' => $index,
'doc_values' => $doc_values,
],
$field instanceof Field\IntegerField => $properties[$name] = [
'type' => 'integer',
'index' => $field->searchable || $field->filterable, // TODO recheck doc_values https://github.com/schranz-search/schranz-search/issues/65
'doc_values' => $field->filterable,
'index' => $index,
'doc_values' => $doc_values,
],
$field instanceof Field\FloatField => $properties[$name] = [
'type' => 'float',
'index' => $field->searchable || $field->filterable, // TODO recheck doc_values https://github.com/schranz-search/schranz-search/issues/65
'doc_values' => $field->filterable,
'index' => $index,
'doc_values' => $doc_values,
],
$field instanceof Field\ObjectField => $properties[$name] = [
'type' => 'object',
Expand Down
Loading