Search Service API

This documentation is for developers and describes how to use the search API to retrieve results that can be used in individual codings such as third-party components to be represented and to be used in other yuuvis® RAD API.

This search-service API supports client developers to use the Elasticsearch capabilities more easily. It offers two specific main parts: one to find objects by determining filters and one to get statistics via aggregations. Another part is to support data export with resolved object references and user names. 

Endpoint Method Overview for Search

PropertyValue
Full path
[Gateway-URL]/search
HTTP method
POST
Success code
200: OK
Failure codes
401: Unauthorized
422: Unprocessable Entity
Types consumed
application/json
Types produced
application/json;charset=UTF-8

Endpoint Request Parameter

NameCommentType
size
optional parameter; default is 100
RequestParam
Authorization
optional parameter
RequestHeader 
Accept-Language
optional parameter; default is Localdefault of the server.
RequestHeader

Request (HTTP body) 

For this request, the data is sent in JSON format.

Notes:

  • Names of object types and fields are the technical names, as used in yuuvis® RAD designer.


{
  "term": "optimal", // optional: a query string is parsed into a series of terms and operators. A term can be a single word  or a phrase.
					 // The default operator is AND. For example, the query 'capital of Hungary' is translated to 'capital AND of AND Hungary' 
  "from": 0,         // optional: parameter sets the offset for result list, default is 0
  
 "filters": {        // optional: can be used to restrict the search in respect to special object attributes (baseparams), 
                     // index data and/or object types. Different filter groups are combined with AND.
    "creator": {     // object attribute, similar is "modifier" 
      "o": "eq",     // operator: eq = "term" is "eq"ual field value
					 // 		  in = "term" is "in" field value
					 //           rg = predefined range, "v1" is one of these: 
                     //                "today", "yesterday", "thisweek", "lastweek", "thismonth", "lastmonth", "thisyear", "lastyear"
                     //           gt = greater than "v1", lt = lower than "v1", gte = greater than equal "v1", lte = lower than equal "v1" 
                     //           gtlt = greater than "v1" and lower than "v2" , etc for gtlte,  gtelt, gtelte, 
      "v1": "smith" 
    },
    "mimetypegroup": {        // object attribute
      "o": "in",
      "v1": [
        "pdf",
        "word"
      ]
    },
    "modified": {             // object attribute
      "o": "rg",
      "v1": "thismonth"
    },
    "created": {             // object attribute
      "o": "gtelte",
      "v1": "2016-01-01T00:00:00.000Z",
      "v2": "2016-01-10T00:00:00.000Z"
    },
    "personalfile.name": {   // <technical name of object type>.<technical name of field>
      "o": "eq",
      "v1": "idpic"
    },
    "basisdocument.active": {
      "o": "eq",
      "v1": "true"           // field of type 'boolean' can also be "false" or "null"
    },
    "personalfile.employee": {
      "o": "in",
      "v1": [
        "Applicant",         // list of terms combined with OR. The wildcards '*' and '?' can be used in string fields. For catalog fields, the term must be an exact match of the catalog value.
        "Manager"
      ]
    },
    "personalfile.cv.position": {	// <technical name of objecttype>.<technical name of field>.<technical name of column>
        "o": "eq",
        "v1": "Developer"      // can also be "null" to find objects with no value for this attribute
    };
    "basisdocument.age": {
      "o": "lte",
      "v1": 50                // for numeric fields
    },
    "basisdocument.status": {
      "o": "eq",
      "v1": null                // For any field, this query returns documents that have no value in the named field.
    },
    "personalfile.activesince": {
      "o": "eq",
      "v1": "2016-01-01T00:00:00.000Z"  // for date and datetime fields
    },
	"personalfile.path": {			// 
      "o": "match",					// For string fields with classification 'Path', match all documents with paths that exist within a particular path
      "v1": "/Europe/Switzerland" 	// '/Europe/Switzerland', for example /Europe/Switzerland/Zürich, /Europe/Switzerland/Zürich (ZH), /Europe/Switzerland/Zürich/Winterthur
								   
    }
  },
  "types": [                 // optional: list of object types combined with OR. In the case of index data search, only one object type is allowed
    "picture"
  ],
  "contextfoldertypes": [    // optional: list of context folders (technical names) combined with OR
    "personalfile"           // find all objects that are saved in the general location with 'sysroot'
  ],
  "fields": [         // optional: list of general object attributes and index fields which should be returned in the result
                      // if this section is not specified, a default set of general attributes as configured is delivered
    "type"                   // default: technical name of object type as configured in the designer
    "title",                 // default: individually defined for each object type
    "description",           // default: individually defined for each object type
    "created",               // default: the date the object was created 
    "creator",               // login name of the user who created the object
    "creatorfullname",       // default: full name of user who created the object; format: <first name> <name>
    "creatortitle",          // title of user who modified the object; format: <name>,<first name> (<login name>)
    "modified",              // default: the date the object was modified 
    "modifier",              // login name of the user who modified the object
    "modifierfullname"       // default: full name of user who modified the object; format: <first name> <name>
    "modifiertitle",         // title of user who modified the object; format: <name>,<first name> (<login name>)
    "finalized"              // finalization state of an object showing 'true' or 'false'
    "mimetype",              // mime type of a file as given in the operating system
    "mimetypegroup",         // same as 'filetype' of the document file which group mime types, e.g., all Microsoft Word mime types are grouped as 'word', and all image type as 'image'
    "filename",              // of the document file
    "filetype",              // of the document file
    "filesize",              // of the document file
    "idpicture.width,        // any object field must be given by its qualified technical name in the format: <technical name of object type>_<technical name of field>
    "myobject.mytable.mycolumn",  // the column of a table of an object
	"contextfolderid",
	"personalfile.name"
  ],
  "options": {               // optional: defines some special search behaviors
    "tracktotalhits": false  // wiht default 'true' 
    "searchmode": "idxs",     // (default: "idxs")optional,  index data search; case insensitive with wildcards '*' and '?' allowed
                             // "fts" = full-text search
	"resolvereference":true, // (default: false) resolves ID references of users and objects
	"resolveorgaddon" :false,// (default: true) resolves user ID, if false the login name will be returned
	"withcontext: true,		 // (default: false) returns additionally title, description and ID of the context folder  for every hit
	"expertmode": false,     // (default: false) if true, the "term" is interpreted in the syntax of the Elasticsearch engine
    "sort": {                // optional: defines the sort order
      "created": {
         "order": "asc",       // control the result order; possible settings are 'asc' and 'desc'
         "missing": "_first"   // control how to handle missing field values: possible settings are "_last" and "_first" in the list
      }
    }
    "scope" : "all"          // "all" as default searches in indexdata and content, "indexdata" only in indexdata, and "content" only in contents (document files).
    }
  }
}

Result (response body)

For this request, the data is sent in JSON format, as seen in the following example.


{
    "took": 31,                     // time the response needed (in milliseconds)
    "timed_out": false,             // OK, no timeout was reached
    "hits": {
        "total": 5,                 // 5 hits are shown in the following hit list !! use this code in versions 6.0 or earlier
        "total": {					// 5 hits are shown in the following hit list !! use this code for versions 6.0 or later !! Breaking change, see comment below this code block !!
            "value": 5,				
            "relation": "eq"
        },
        "max_score": 2.1420739,     // for each hit, a score is given which can be used for sorting purposes
        "hits": [{
             "_index": "enaiored",    // name of search index database 
             "_type": "dmsobject",      // internal notation only
             "_id": "7A241428D3164044AB98533A70A382D4",        
             "_score": 2.1420739,  
             "_source": {               // list of result list parameters as requested
                   "idpicture.firstname": "Martin",
                   "created": "2016-01-21T02:47:40Z",
                   "modifiertitle": "Roth, Rudger",
                   "creatortitle": "Roth, Rudger",
                   "modified": "2017-02-16T06:39:40Z",
                   "id": "7A241428D3164044AB98533A70A382D4",    // ID of the object, which must be used in the client URL to show the object as described in the following section
                   "type": "idpicture",                         
                   "idpicture.name": "Schriddel",
				   "idpicture.participant": "FEFA1C490EAE4EBDBBFC100C10627013",
				   "idpicture.participant_meta":{
        				"title": "Bartonitz, Martin",
        				"id": "2FF19AE615174A4188E2BB3AB3881FF3",
       					 "type": "personalakte"
      				}
             },
			"context": { 								// context folder of this dms object 
				"title": "Akte", 						// title of the context folder
				"description": "Personalakte", 			// description of the context folder
				"id": "F6483A09F580420D86950BB4722D02A2"// id of the context folder
			},
			...
        }]
    },
	"config": {								// a result list configuration by type and optional context type.			
    "type": "dokument",
    "elements": [                           // list of field parameters depending on the use of "type"
      {
        "name": "type",
        "qname": "sysobject.type",
        "hitname": "type",
        "label": "Type",
        "description": "Type of the object.",
        "type": "STRING",
        "readonly": true,
        "baseparameter": true,
        "system": true,
        "selectedforenrichment": false,
        "required": true,
        "maxlen": 0,
        "minlen": 0,
        "scale": 0,
        "precision": 0,
        "withtime": false
      },
      {
        "name": "mimetypegroup",
        "qname": "sysdocument.mimetypegroup",
        "hitname": "mimetypegroup",
        "label": "File group",
        "description": "File group of the content.",
        "type": "STRING",
        "readonly": true,
        "baseparameter": true,
        "system": true,
        "selectedforenrichment": false,
        "required": true,
        "maxlen": 0,
        "minlen": 0,
        "scale": 0,
        "precision": 0,
        "withtime": false
      },
      {
        "name": "titel",
        "qname": "dokument.titel",
        "hitname": "dokument.titel",
        "label": "titel",
        "description": "titel",
        "type": "STRING",
        "readonly": false,
        "baseparameter": false,
        "system": false,
        "selectedforenrichment": false,
        "required": true,
        "maxlen": 50,
        "minlen": 5,
        "scale": 0,
        "precision": 0,
        "withtime": false
      },
      {
        "name": "bemerkung",
        "qname": "dokument.bemerkung",
        "hitname": "dokument.bemerkung",
        "label": "bemerkung",
        "description": "bemerkung",
        "type": "STRING",
        "readonly": false,
        "baseparameter": false,
        "system": false,
        "selectedforenrichment": false,
        "required": false,
        "maxlen": 200,
        "minlen": 0,
        "scale": 0,
        "precision": 0,
        "withtime": false
      } ,
	...    
}

Search Response for hits

In the search response, hits.total is an object. The total count of hits that match the search request is now returned as an object with a value and a relation.

The value specifies the number of hits that match. The relation specifies whether the value is accurate (eq) or a lower bound (gte):

{
    "hits": {
        "total": {
            "value": 1000,
            "relation": "eq"
        },
        ...
    }
}

The total object count in the response specifies that the query matches exactly 1.000 documents ("eq"). The value is always accurate ("relation": "eq") when track_total_hits is set to true (default) in the request.

Example: Find all documents without a file

Beginning with version 9.4, the following example requests only objects that are no folders and that do not contain a document file.

Documents without a file
"filters": {
  "filesize": {
     "o": "rg",         // operator for range
     "v1": "none"       // none stands for no content (document file)
  },
  "folder": {           // this determines to exclude folders
     "o": "eq",
     "v1": false
  }
}


Example: Searching a Term in Several Fields

The following example requests objects where the string "irina@qs.optimal-systems.de" is part of the fields "systo" OR "syscc" in the object type "sysemail". The example uses Elasticsearch engine syntax, so you must append ".fts" to the field specifier.

{
	"term": "txt_sysemail_systo:irina@qs.optimal-systems.de OR txt_sysemail_syscc:irina@qs.optimal-systems.de",
	"options": 
	{
		"expertmode": true           
	}
}

Endpoint Method Overview for Aggregations

PropertyValue
Full path
[Gateway-URL]/search/aggregate
HTTP method
POST
Success code
200: OK
Failure codes
404: Not Found
412: Precondition Failed
500: Internal Server Error
Types consumed
application/json
Types produced
application/json;charset=UTF-8

Endpoint Request Parameter

NameCommentType
Authorization
optional parameter
RequestHeader 
Accept-Language
optional parameter; default is local of the server.
RequestHeader

Aggregations

The following examples show how to request aggregations and what the response looks like.

Example: Aggregation for Object Types

The following example requests aggregations over the objects that were modified today with a specified time zone regarding the requesting user:

{
	"aggs": {  				// The aggregations object (the key aggs can also be used) in the JSON holds the aggregations to be computed
		"type": {			// Property name of an object for that the aggregation should be calculated, here the system property "type" that is used for the object types
			"size" : 30,				// This key is optional with the default 10. The terms aggregation returns the buckets for the set terms ordered. 
										// Set "size" for a different number of returned buckets
            "order" : "desc",           // This key is optional with the default "desc" for descanding order in respect with the key "sort". Set "asc" for ascanding.
            "sort" : "_count",          // This key is optional with the default "_count" for 
            "min_doc_count" : 1
		},
	"filters": {              // The "filter" objects configures how to reduce the hits
		"modified": {        // The system property "modified" contains the date of the last modification of an object
				"o": "rg",      // The operator "o" is set to "rg" for a date range configuration
				"v1":"today"    // The filter value "v1" for the range is set to "today" to get only objects that were modified today.
		}
	},
	"options": 
	{
		"timezone": "+02:00"   // The timezone can be used if users are working in different timezones.
	}
}

The response to the above request will look like the following JSON example:

{
	"aggregations": {
		"type": {
			"doc_count_error_upper_bound": 0,
			"sum_other_doc_count": 0,
			"buckets": [
				{
					"key": "personalfile",          // technical name of the object type
					"doc_count": 2,                 // number of found objects of the object type "personalfile"
				},
				{
					"key": "qsfolder",
					"doc_count": 1,
				}
			]
		}
	}
}

Beginning with version 9.16 LTS, the bucket keys for the system property "type" and for codesystems (catalogues) are localized in the language of the calling user.

Example: Use of Sub-Aggregations

This example shows how to configure the aggregation request for counting document objects sub-aggregated regarding the object type of the folder the document objects are saved to:

{
	"aggs": {  							
		"type": {						
			"sub": {
				"contextfoldertype": {       
				}
			}
		}
    },
	"filters": {
		"folder": {           / this determines to exclude folders
			"o": "eq",
			"v1": false
        }
	}
}

The response looks like this:

{
    ...
    "aggregations": {
        "type": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
                {
                    "key": "letterofconcent",           // technical document object type nam
                    "doc_count": 3,                     // summed number of his for "letterofcontent
                    "contextfoldertype": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "key": "personalfile",    // technichal name of folder object type name
                                "doc_count": 2            // number of "letterofcontent" found in "personalfile"
                            },
                            {
                                "key": "project",
                                "doc_count": 1            // number of "letterofcontent" found in "project"
                            }
                        ]
                    }
                }
            ]
        }
	}
}

Example: Aggregate with ranges for Number Properties

The following example shows how to use the range "rg" parameter that defines a list of three ranges starting from "o1" to "o2".

Aggregation with ranges for numbers properties
{
    "aggs": {
        "invoice.amountbeforetax": {
            "rg": [
                {
                    "o1": 0.0,
                    "o2": 2000.0
                },
                {
                    "o1": 2001.0,
                    "o2": 4000.0
                },
                {
                    "o1": 4001.0,
                    "o2": 6000.0
                }
            ]
        }
    },
    "types": ["invoice"]
}

The response looks like:

Response of an aggregation with range for a number property
{
    ...,
    "aggregations": {
        "invoice.amountbeforetax": {
            "buckets": [
                {
                    "key": "0.0_2000.0",
                    "from": 0.0,
                    "to": 2000.0,
                    "doc_count": 2
                },
                {
                    "key": "2001.0_4000.0",
                    "from": 2001.0,
                    "to": 4000.0,
                    "doc_count": 7
                },
                {
                    "key": "4001.0_6000.0",
                    "from": 4001.0,
                    "to": 6000.0,
                    "doc_count": 10
                }
            ]
        }
    }
}

Example: Aggregate with ranges for Date Properties

The following example shows how to use the range "rg" parameter that defines a list of three ranges starting from "o1" to "o2".

Aggregation with ranges for numbers properties
{
    "aggs": {
        "qdoc.effectivefrom": {
            "rg": [
                {
                    "o2": "2010-12-31"
                },
                {
                    "o1": "2011-01-01",
                    "o2": "2016-12-31"
                },
                {
                    "o1": "2017-01-01"
                }
            ]
        }
    }
} 

The response looks like:

Response of an aggregation with range for a date property
{
    ...,
    "aggregations": {
        "qdoc.effectivefrom": {
            "buckets": [
                {
                    "key": "to_2010-12-31",
                    "to": 1.2937536E12,
                    "to_as_string": "2010-12-31T00:00:00.000Z",
                    "doc_count": 7669
                },
                {
                    "key": "2011-01-01_2016-12-31",
                    "from": 1.29384E12,
                    "from_as_string": "2011-01-01T00:00:00.000Z",
                    "to": 1.4831424E12,
                    "to_as_string": "2016-12-31T00:00:00.000Z",
                    "doc_count": 11220
                },
                {
                    "key": "from_2017-01-01",
                    "from": 1.4832288E12,
                    "from_as_string": "2017-01-01T00:00:00.000Z",
                    "doc_count": 65
                }
            ]
        }
    }
}


Example: Use of Metrics for Number Properties

The following configuration shows how to use the aggregation attribute metrics for properties of type numbers like the amount of an invoice:

{
	"aggs": {			
		"invoice.amount": {                  // technical name of an object property
			"metrics" : "<metrics type>"	 // see legend below	    
		}
	}
}

The metrics types are:

metrics typeDescription
avgcomputes the average of numeric values that are extracted from the aggregated objects.
maxreturns the maximum value among the numeric values extracted from the aggregated objects.
minreturns the minimum value among numeric values.
sumsums up numeric values that are extracted from the aggregated objects
medianapproximates the median absolute deviation of its search results

The response for a single metrics request is:

Response for a single metrics request
{
   ...
    "aggregations": {
        "min#invoice.amount_min": {
            "value": 123.0
    }
} 

Beginning with version 10.0, it is possible to request for more than one value in the metrics. They must be comma-separated like "min,max". The response looks a bit different regarding the key with an additional extension:

Response for a single metrics request
{
   ...
    "aggregations": {
        "min#invoice.amount_min": {
            "value": 123.0,
        "max#invoice.amount_max": {
            "value": 155.0
        }
    }
} 

Example: Use of sub-aggregation combined with metrics

Beginning with version 10.0, the following request shows how to get the sum for a number attribute like the turnover for customers instead of the document counts:

Request
{
  "aggs" : {
    "invoice.customer" : {
      "size" : 10,
      "oder" : "asc",
      "sort" : "invoice.turnover.value",
      "sub": {
        "invoice.turnover" : {
          "metrics" : "sum"
        }
      }
    }
  }
}

The response looks like this:

Response
{
    ...
    "aggregations": {
        "invoice.customer": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
                {
                    "key": "Sunshine Inc.",
                    "doc_count": 1,
                    "sum#invoice.turnover": {
                        "value": 43.2
                    }
                },
                {
                    "key": "BuyMe Org",
                    "doc_count": 1,
                    "sum#einvoice.turnover": {
                        "value": 235.62
                    }
                },
                ...
            ]
        }
    }
}

And if you want to get more than one metric you can request such this with the sorting one specifically:

Request
{
  "aggs" : {
    "invoice.customer" : {
      "size" : 10,
      "oder" : "asc",
      "sort" : "invoice.turnover_max.value",
      "sub": {
        "invoice.turnover" : {
          "metrics" : ["sum","min","max"]
        }
      }
    }
  }
}

The response looks like this:

Response
}
    ...
    "aggregations": {
        "etlainvoice.etlasupplierno": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
                {
                    "key": "Sunshine Inc.",
                    "doc_count": 1,
                    "sum#invoice.turnover_sum": {
                        "value": 43.2
                    },
                    "max#invoice.turnover_max": {
                        "value": 43.2
                    },
                    "min#invoice.turnover_min": {
                        "value": 43.2
                    }
                },
                ...
            ]
        }
   }
}

Example: Use of Histograms for DateTime Properties

The following configuration shows how to use the aggregation attribute histogram for object properties of type datetime. Possible histograms can be used for days, weeks, month, and years.

{
	"aggs": {  							
		"project.datefield1": {        // Beginning with version 9.16 LTS, the following histogram based aggregations for days, months, weeks, and years are supported:
			"histogram": {
				"calendar_interval" : "year",   // This single-value aggregation responds buckets per year
				"format" : "yyyy",              // The syntax of the format is that of Elasticsearch. It is important that calendar_interval and format must fit together to optain meaningful results.
                "size": 10,                     // Return maximal 10 buckets
			}                                   
		},
		"project.datefield2": {
			"histogram": {                           
				"calendar_interval" : "month",  // This single-value aggregation responds buckets per month in the format year-month like 2023-01 for January of 2023
				"format" : "yyyy-MM"
			}
		},
		"document.datefield3": {
			"histogram": {
				"calendar_interval" : "week",  // This single-value aggregation responds buckets per week in the format year-week like 2023-47 for th 47th week of 2023
				"format" : "yyyy-ww"
			}
		},
		"document.datefield4": {
			"histogram": {
				"calendar_interval" : "day",   // This single-value aggregation responds buckets per day in the format year-month-day like 2023-12-31 for the day 31st of December 2023
				"format" : "yyyy-MM-dd",
                "size" : 5,                     // In this case of a histogram aggregation use this parameter for the number of partial values (buckets) within "histogram"!
                "min_doc_count": 1,             // This parameter forces to return only buckets with a least one document found.
                "sort": "_count",               // Returns the buckets sorted by number of documents per bucket.
                "order": "desc"                 // The order can be "desc" or "asc" (default).
			}
		}
	},
}


Endpoint Method Overview for CSV Exports

PropertyValue
Full path
[Gateway-URL]/search/export
HTTP method
POST
Success code
200: OK
Failure codes
404: Not Found
412: Precondition Failed
500: Internal Server Error
Types consumed
application/json
Types produced
text/csv;charset=UTF-8

Endpoint Request Parameter

NameCommentType
Authorization
optional parameter
RequestHeader 
Accept-Language
optional parameter; default is locale of the server.
RequestHeader

Request (HTTP body) 

For this request, the data is sent in JSON format.

Notes:

  • The names of the object types and fields are the technical names, as used in yuuvis® RAD designer.
  • The delimiter character (separates each cell in a row) is configured in the search-prod.yml of the search service:
    csv.delimiter : ';'
    Default delimiter is  ';'.
  • Important: sort order is not considered

Example: Exporting Objects Created Within a Time Range

The following example exports the objects created yesterday as a CSV file.

{
	"filters":
	{
		"created":
		{
			"o": "rg",
			"v1":"yesterday"
		}
	}
}

Remark: Beginning with version 10.6, the column labels are only given in English. If other labels should be contained in the CSV file, they must be part of the payload as shown below:

The list of fields controls the column sequence in the CSV file. The value of the fields contains the qualified name of the field (object type property) followed by the label (all characters after the first space). If no label is given, the technical name is used for it.

{
  "types": [
    "personaldossier"
  ],
  "options": {
    ...
  },
  "fields": [
    "type Type",
    "personaldossier.dateofdeployment Date of employment",
    "personaldossier.class Class",
    "personaldossier.firstname First name",
    "personaldossier.surname Family name",
    ...
    "created Created",
    "creatortitle Creator",
    "modified Edited",
    "modifiertitle Editor"
  ]
}