Detailing the available schema, object type definitions as well as property definitions.

Introduction

In yuuvis® Momentum, documents are stored as document objects. The business integrator defines one or more object types and their properties according to specific needs. Any document imported will need to be classified as exactly one of the object types defined in the schema of the system. 

The schema defines a set of object types and a set of properties. The object type classifies the object and defines the properties that the object must have or is allowed to have (properties may be optional). There are furthermore some metadata like 'system:versionNumber' or 'system:lastModificationDate', whose values are provided by the system.

<schema xmlns="http://optimal-systems.org/ns/dmscloud/schema/v5.0/" 
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
        xsi:schemaLocation="http://optimal-systems.org/ns/dmscloud/schema/v5.0/ dmsCloud-schema.xsd">
        <propertyStringDefinition>
            <id>from</id>
            <propertyType>string</propertyType>
            <cardinality>single</cardinality>
            <required>true</required>
        </propertyStringDefinition>
        <propertyStringDefinition>
            <id>to</id>
            <propertyType>string</propertyType>
            <cardinality>multi</cardinality>
            <required>true</required>
        </propertyStringDefinition>
		<propertyStringDefinition>
            <id>subject</id>
            <propertyType>string</propertyType>
            <cardinality>single</cardinality>
            <required>true</required>
        	<defaultValue>hello</defaultValue>
        	<maxLength>20</maxLength>
	        <minLength>4</minLength>
		</propertyStringDefinition>
		<propertyDateTimeDefinition>
            <id>received</id>
            <propertyType>datetime</propertyType>
            <cardinality>single</cardinality>
            <required>true</required>
        </propertyDateTimeDefinition>
		<typeDocumentDefinition>
            <id>email</id>
            <baseId>system:document</baseId>
            <propertyReference>from</propertyReference>
            <propertyReference>to</propertyReference>
            <propertyReference>received</propertyReference>
			<contentStreamAllowed>required</contentStreamAllowed>
        </typeDocumentDefinition>
</schema>

By default, the maximum number of property definitions in a tenant-specific schema is 20. A system integrator can change this limit via the schema.tenant.properties.limit parameter in the system service. If you want to increase this limit, it is recommended to increase the maximum number of fields in your elasticsearch index as well.

Type IDs

In every property definition and every object type definition the id attribute is required. It is used to identify the object type or property. An ID is a string with a maximum of 63 characters and it must match the regular expression
([a...zA...Z][a...zA...Z0...9]*:)?[a...zA...Z][a...zA...Z0...9]*.

Type IDs are also used as the name of the type, e.g., in query operations. Hence, it is recommended to choose meaningful values for type IDs.

The part before the : character is the prefix. In tenant-specific (app-specific) types the prefix is always "ten"+<tenant name> ("app"+<app name>). In most cases, you can omit the prefix when designing a schema or importing objects or creating search queries. The prefix is added automatically.

An exception are column names in Table Property Definitions, where prefixes are prohibited as of version 2020 Winter.

Property Definitions

General Attributes

All property definitions have the following attributes:

id

localNamespace

description

propertyType

cardinality

required

queryable

defaultValue

propertyType

Specific Attributes

Depending on the property type a property can have specific attributes.

Integer Property Definitions

maxValue

minValue

DateTime Property Definitions

resolution

Decimal Property Definitions

Decimal properties support values of 64-bit precision (IEEE 754). The values have to be specified in decimal notation. However in the table below, the technical limits are provided in base 2 scientific notation in order to display the values in a condensed and comfortable format. This format cannot be used to specify the value for a maxValue or minValue attribute in a decimal property definition.

maxValue

minValue

Please note the additional limit of precision: values of magnitude smaller than 2 ^-1074 will be rounded to 0.

String Property Definitions

maxLength

minLength

In general, the length of a string is limited to 8192. Hence, the maximum value for maxLength and minLength is 8192.

Table Property Definitions

The column types of a table property are defined by a list of property definitions inside the table property definition. Each column property definition has its own attributes, such as required or default value. The values are applied to each row entry. The cardinality of a column property definition must be single.

As of version 2020 Winter, the column names specified via <id>examplecolumn</id> must not contain a prefix. They have to follow the convention [a...zA...Z][a...zA...Z0...9]*. Otherwise, the schema containing the corresponding property definition will not pass the validation.

Table properties differ depending on the value of the property queryable. If queryable is false, the table must not appear in WHERE clauses in search queries. However, you can still find objects using full-text conditions on values stored in a table (query keyword 'CONTAINS'). If queryable is true, you can apply more precise search queries to a table, but you will need more disk space to store objects.

The number of rows and columns of a table property definition is limited to a maximum of 512 columns and 1024 rows.

<schema xmlns="http://optimal-systems.org/ns/dmscloud/schema/v5.0/" 
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
        xsi:schemaLocation="http://optimal-systems.org/ns/dmscloud/schema/v5.0/ dmsCloud-schema.xsd">
        <propertyTableDefinition>
            <id>aTableProperty</id>
            <propertyType>table</propertyType>
            <cardinality>single</cardinality>
            <required>false</required>
            <propertyStringDefinition>
                <id>col0</id>
                <propertyType>string</propertyType>
                <cardinality>single</cardinality>
                <required>false</required>
            </propertyStringDefinition>
            <propertyIntegerDefinition>
                <id>col1</id>
                <propertyType>integer</propertyType>
                <cardinality>single</cardinality>
                <required>false</required>
            </propertyIntegerDefinition>
            <propertyDateTimeDefinition>
                <id>col2</id>
                <propertyType>datetime</propertyType>
                <cardinality>single</cardinality>
                <required>false</required>
            </propertyDateTimeDefinition>
            <propertyDecimalDefinition>
                <id>col3</id>
                <propertyType>decimal</propertyType>
                <cardinality>single</cardinality>
                <required>false</required>
            </propertyDecimalDefinition>
		</propertyTableDefinition>
</schema>

Object Type Definitions

There are different groups of object type definitions:

• document object types
• folder object types
• secondary object types

All object type definitions have the following attributes:

id

localNamespace

description

baseId

propertyReference 

A tenant-specific object type can have references to both tenant-specific and global properties.

Document Object Type Definitions

Document object types are the elementary object types. To store objects with content, the objects' type must be a document type.

Document object type definitions have the following specific attributes:

contentStreamAllowed

secondaryObjectTypeId

Folder Object Type Definitions

You define folder object types as structuring elements either in your global, tenant-specific or application schema. They do not have their own content files in contrast to document object types. In yuuvis® Momentum versions 2020 Autumn and older, folders cannot be set up in a hierarchical structure – a folder inside a folder is not allowed. As of version 2020 Winter, a folder hierarchy is possible. Folders allow the grouping of multiple objects and have their own metadata which will not be inherited by the assigned objects. They act as object parent being referred to by its system:objectId property value in the individual metadata of the child objects as system:parentId and thus are also taken into account during the search. Similar to a document object type a folder can reference a secondary object type's property group. The properties can be used as regular properties on folder level. Folders with objects assigned to them cannot be deleted.

The following code block shows the example folder object type definition "dossier" that can be integrated into any given schema. Note that folder object type definitions need to be defined after all document type definitions, yet before any secondary object type definitions.

<schema xmlns="http://optimal-systems.org/ns/dmscloud/schema/v5.0/" 
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
        xsi:schemaLocation="http://optimal-systems.org/ns/dmscloud/schema/v5.0/ dmsCloud-schema.xsd">

        <propertyStringDefinition>
            <id>str1</id>
            <propertyType>string</propertyType>
            <cardinality>single</cardinality>
            <required>true</required>
        </propertyStringDefinition>
      
        <typeDocumentDefinition>
            <id>documentType1</id>
            <baseId>system:document</baseId>
            <propertyReference>str1</propertyReference>
            <contentStreamAllowed>required</contentStreamAllowed>
            <secondaryObjectTypeId>secondaryA</secondaryObjectTypeId>
        </typeDocumentDefinition>
       
		<typeFolderDefinition>
        	<id>dossier</id>
       		<baseId>system:folder</baseId>
        	<propertyReference>title</propertyReference>
        	<propertyReference>type</propertyReference>
        	<propertyReference>description</propertyReference>
    	</typeFolderDefinition>
</schema>

The document object types are the elementary object types of your schema in which the "real" content is stored. During import or update of an object the object ID of the target folder is provided as the new object's parent ID (system:parentId). Actions such as assigning objects, moving them to or removing them from a folder can easily be carried out in this way. A validation takes place and checks whether the given ID is a folder and does exist. Furthermore, as of version 2020 Winter, folders in a folder will be allowed. The folder hierarchy will represent a tree structure where each folder cannot appear as a parent in different levels of the structure. Hierarchies that do not adher to this structure will not pass validation.

By assigning documents to a configured folder you set an exact filing location and ensure a "tidy" storage of documents within your repository – if needed for your use case.

After schema modification, a folder object can be created by importing the following metadata:

{
	"objects": [
	{
		"properties": {
			"system:objectTypeId": {
				"value": "dossier"
			},
			"title": {
				"value": "My E-mail Folder #1"
			},
			"type": {
				"value": "E-mails"
			},
			"description": {
				"value": "This folder holds all the e-mails authored by me"
			}
		}
	}
	]
}

After extracting the object ID of the folder from the import response, you can start populating the folder with documents:

{
	"objects": [
	{
		"properties": {
			"system:objectTypeId": {
				"value": "documentType1"
			},
			"system:parentId": {
				"value": "<the Folder object ID from the import response"
			},
			"str1": {
				"value": "Some important business documents"
			},
			"description": {
				"value": "This folder holds all the e-mails authored by me"
			}
		}
	}
	]
}

Folder object type definitions have the following specific attributes:

secondaryObjectTypeId

Secondary Object Type Definitions

Secondary object types are abstract. This means that they cannot be instantiated. They allow you to design a more complex schema. In a way the concept of secondary object types is similar to the concept of inheritance. 

Secondary object types can be used to group properties and then assign these property groups to object types (e.g., documents or folders). Like other object types, a secondary object type can have references to properties. They can also have no properties at all which can be understood as a way of categorizing document types (tagging). Document or folder object types can in turn reference secondary object types, which give them their properties. There are two ways for secondary object types to be referenced by an object in the schema definition: as static or as floating.

• <secondaryObjectTypeId static="true">INV</secondaryObjectTypeId>.
• <secondaryObjectTypeId static="false">INV</secondaryObjectTypeId>.

The property groups of static referenced secondary object types are automatically available in all instances of the object type. Floating secondary object types can be handled in a flexible way during the import (POST /api/dms/objects endpoint) or at runtime for already existing instances of an object type with an update (POST /api/dms/objects/{objectId} / PATCH /api/dms/objects/{objectId}).  The keywords "add":, "value": or "remove": can be used in the "system:secondaryObjecttypeIds": property area of the metadata.json filecan.

"value":

{
    "objects": [{
        "properties": {
            "system:objectTypeId": {
                "value": "appSot:document"
            }
            "system:secondaryObjectTypeIds": {
                "value": ["INV","SUP"]
            }
...

"add":

{
    "objects": [{
        "properties": {
            "system:secondaryObjecttypeIds": {
                "add": "INV"
            },
...

"remove":

{
    "objects": [{
        "properties": {
            "system:secondaryObjecttypeIds": {
                "remove": "INV"
            },
...

Document object types have a system:secondaryObjectTypeIds property that contains the secondary object types associated with the document object type in the schema. This allows secondary object types to be taken into account during the search. The secondary object type must be explicitly specified in the FROM clause: select * from appSot:INV.

The system:secondaryObjectTypeIds property is set by the repository using the schema.

<schema xmlns="http://optimal-systems.org/ns/dmscloud/schema/v5.0/"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://optimal-systems.org/ns/dmscloud/schema/v5.0/ dmsCloud-schema.xsd">
 
        <propertyStringDefinition>
            <id>appSot:dateOfReceipt</id>
            <propertyType>date</propertyType>
            <cardinality>single</cardinality>
            <required>true</required>
        </propertyStringDefinition>
        <propertyStringDefinition>
            <id>appSot:comment</id>
            <propertyType>date</propertyType>
            <cardinality>single</cardinality>
            <required>true</required>
        </propertyStringDefinition>
        <propertyStringDefinition>
            <id>appSot:invoiceNo</id>
            <propertyType>string</propertyType>
            <cardinality>single</cardinality>
            <required>true</required>
        </propertyStringDefinition>
        <propertyStringDefinition>
            <id>appSot:paymentTerm</id>
            <propertyType>string</propertyType>
            <cardinality>single</cardinality>
            <required>true</required>
        </propertyStringDefinition>
         
        <typeDocumentDefinition>
            <id>appSot:document</id>
            <baseId>system:document</baseId>
            <propertyReference>appSot:dateOfReceipt</propertyReference>
            <contentStreamAllowed>required</contentStreamAllowed>
            <secondaryObjectTypeId>appSot:basicInfo</secondaryObjectTypeId>
        </typeDocumentDefinition>
         
        <typeSecondaryDefinition>
            <id>appSot:basicInfo</id>
            <baseId>system:secondary</baseId>
            <propertyReference>appSot:comment</propertyReference>
        </typeSecondaryDefinition>
        <typeSecondaryDefinition>
            <id>appSot:invoice</id>
            <baseId>system:secondary</baseId>
            <propertyReference>appSot:invoiceNo</propertyReference>
            <propertyReference>appSot:paymentTerm</propertyReference>
        </typeSecondaryDefinition>
</schema>

Consider the example schema. If a document of the type appSot:document is created, it may have values for the properties appSot:dateOfReceipt and appSot:comment. For the appSot:dateOfReceipt property definition this is obvious, because there is a direct reference in the document type definition of appSot:document. 

The appSot:comment property definition is not directly referenced but the definition of appSot:document has a reference to the secondary object type appSot:basicInfo which references the appSot:comment property definition. Thus, both properties are available for creating documents.

Furthermore, properties passed on by secondary object types are treated like "regular" properties in a document. The attributes of a property are taken into account. For example, a document of the type appSot:document not only may have a value for appSot:comment, it must have a value, because it is a required property. It makes no difference, whether a document type references the property definitions directly or indirectly via a secondary object type reference.

You cannot tell from the metadata of a document any longer if a property is referenced directly or indirectly in the schema. All properties are plain in the properties list. For example, the metadata of a document based on the appSot:document document type definition may look like this:

{
    "objects": [
        {
            "properties": {
                "system:objectId": {
                    "value": "7bce6618-2b0e-4abf-af26-e4137e6b0461"
                },
                "system:baseTypeId": {
                    "value": "system:document"
                },
                "system:objectTypeId": {
                    "value": "appSot:document"
                },
                "system:secondaryObjectTypeIds": {
                    "value": [
                        " appSot:basicInfo"
                    ]
                },
                <...>
                " appSot:dateOfReceipt": {
                    "value": "2020-02-20T02:02:20.220Z"
                },
                "appSot:comment": {
                    "value": "Yearly invoice - payment 1 out of 12 monthly payments."
                }
            },
            "contentStreams": [
                <...>
            ]
        }
    ]
}

Secondary object type definitions have the following attributes:

System Properties

General Metadata Properties

In addition to the properties assigned to the object types in the schema, each instantiated object has a set of general system properties. Some system properties are set by the system and some system properties can be set by the user.

Content Stream Properties

The contentStreams group of properties comprises system properties of document objects containing a binary content. Thus, their contentStreamAllowed attribute has to be required or allowed. The properties in contentStreams contain all information necessary to be able to handle binary content. Depending on the type of operation, different properties are required for import/update requests or are displayed in a response.

Prefixes

The IDs of many object type and property definitions have prefixes. System types have the prefix system. Types defined in a tenant schema have the prefix ten followed by the name of the tenant. Types defined in an app schema have the prefix app followed by the name of the app. If you post a tenant schema or an app schema, all IDs in the schema must either match these rules or be missing altogether. If they are missing, the prefix is added to the IDs for the applied schema.

IDs of types defined in the global schema can have any prefix, as long as they do not start with ten or app or equal system. Alternatively, they can also have no prefix.

App Schemata

In the multi-tenant landscape of yuuvis® Momentum, any object types or properties that need to be available for multiple or all tenants, need to be introduced to the system schema using the system schema endpoints. To prevent cluttering the system schema, avoid dependencies and allow for duplicate names, the system schema can be structured into applications, which provide a namespace for properties and object types pertaining to a particular use case.

Applications are defined as smaller schema files that are integrated into the global schema to be available for every tenant. This allows modular usage of application schemata across multiple yuuvis® systems.

Within the schema, applications are defined by a prefix followed by the application's name:  app<app name>. The app name is case insensitive when used as a path parameter to interact with the application schema or within the search engine.

Depending on the situation, the prefix can be omitted, for example to broaden a search query across multiple application schemata.

The app schema endpoints are:

• POST /api/system/apps/{app}/schema - Introduces provided schema as app within tenant schema, overwrites previous app schema
• GET /api/system/apps/{app}/schema - Retrieves the specified app portion of the tenant schema
• POST /api/system/apps/{app}/schema/validate - Validates a schema based on app schema rules

When uploading an app schema, all properties that do not specify a prefix will have that prefix generated as app<app name> where <app name> is equal to the path parameter {app}.

It is allowed to specify this prefix ahead of time, meaning that both "<property name>" and "app<app name>:<property name>" are acceptable names for a property within a schema posted to {base URL}/api/system/apps/<App name>/schema. Uploading a schema using prefixes that do not match the {app} path parameter or any other known apps or tenants will result in a validation error message.

Resolve conflicts that result from multiple similar app/property combinations will also lead to schema validation errors.

Tenant Schemata

Each tenant can define tenant-specific object types in a separate tenant schema. The object types defined in a tenant schema are only available for the corresponding tenant.

Each tenant can define exactly one tenant schema. It can be customized via the following endpoints:

• GET /api/admin/schema - Retrieve the tenant's schema
• POST /api/admin/schema - Update the tenant's schema
• POST /api/admin/schema/validate - Validate the tenant's schema

Each object type ID and property type ID has the prefix ten + <tenant name>. Thus, the same object type name can occur in multiple tenant schemata.

If any prefix is used not equal to ten + <tenant name>, the tenant schema will not pass the validation.

If the prefix is missing in the request body for a tenant schema update, it will be added automatically in the applied schema that is used in use cases like search, import or update of objects.

Summary

In this article, you have reviewed the tools available for creating yuuvis® Momentum schemata. Now you can get started implementing your own custom schema to solve your information management problems.