Comparing Elasticsearch and Database
Compare the objects stored in the database and Elasticsearch index.
Unforeseen circumstances might occur in any system and might lead to differences between database and Elasticsearch index. As an example, a deleted object was correctly removed from the database, but for whatever reason not from the Elasticsearch index. As of yuuvis® Momentum version 2020 Winter, it is possible to check the consistency of the stored objects by means of the maintain compare command provided by the Commander service.
The command determines the objects that were last modified (or created) within a specified period of time and for both storage locations, database and Elasticsearch index. For every single day and for each single object, its properties are compared between database and Elasticsearch index. Differences are summarized and returned in a JSON list as shown in the example on the bottom of the article.
The following parameters can be given to the maintain compare command:
| Parameter | Description | Example |
|---|---|---|
-h | --help | Displays the help for the command. | maintain compare -h |
-t | --tenant <arg> | Specifies the target tenant. If not specified, the command is executed for all tenants. |
|
-s | --start-date <arg> | Specifies the start date for the comparison in If not specified, the earliest |
|
-e | --end-date <arg> | Specifies the end date for the comparison in yyyy-MM-dd format. Document objects with a system:lastModificationDate (or system:creationDate) before and inclusive of the specified date at 23:59:59.999 are taken into account for the comparison.If not specified, the current date is used as end date. |
|
-c | --useCreationDate | If the parameter is set, the system:creationDate is used instead of the system:lastModificationDate to decide whether the corresponding object is included in the comparison. |
|
-m | --mode <arg> | Specifies the mode of the comparison determining which object properties are included in the comparison. The available modes are listed and described in the table below. |
|
The following modes can be selected for the maintain compare command:
| Mode | Description |
|---|---|
COUNT | The command determines and compares the number of objects modified (or created) on every single day in the specified time range. |
BASIC | The command determines and compares the following properties of all objects that are included in the comparison:
|
ADVANCED | The command determines and compares the properties listed for the
|
The following code block shows an example response. The job section contains information on the parameters that were specified for the command. The next sections are named after the dates for which differences in the analyzed object properties were found. The MissingInES list contains the object IDs specifying the objects that were found in the database but not in the Elasticsearch index. Objects specified by IDs listed in Differences were found in both storage locations, but at least one of the analyzed properties has a different value. The MissingInDb list contains the object IDs specifying the objects that were found in the Elasticsearch index but not in the database.
{
"job": {
"tenant": "default",
"mode": "ADVANCED",
"startDate": "2020-10-25T23:00:00.000+0000",
"endDate": "2020-10-31T23:00:00.000+0000",
"useCreationDate": false,
"jobFields": [
"OBJECTID",
"CREATEDBY",
"CREATIONDATE",
"LASTMODIFIEDBY",
"LASTMODIFICATIONDATE",
"TENANT",
"VERSIONNUMBER",
"DELETED",
"BASETYPEID",
"OBJECTTYPEID",
"TRACEID",
"TAGS"
]
},
"2020-10-26": {
"MissingInES": [],
"Differences": [
"b6ef10db-8e46-43f6-9a3f-89eab4b7265d",
"b1786ae8-5fd1-4425-8974-ac6eb435e1c2",
"a64bb85f-87ab-47a9-8b5c-a816008a6691"
],
"MissingInDb": []
},
"2020-10-29": {
"MissingInES": [],
"Differences": [
"53508531-1913-4452-a4a1-c4b7c6a0d063"
],
"MissingInDb": []
}
}