HCMS Schema - Custom Keys and Views

Describes how to define and use custom keys and global views.

Each entity is always identified by its automatically assigned id. This is handled by censhare server, which ensures that this id (asset id) is always present and unique in that particular server instance. This means that each entity has unique endpoint to access it.

There are some practical disadvantages of auto-assigned asset id:

Values are different for different servers, even if the entities themselves are otherwise equal.
- This is especially problematic when using several environments (development, testing, production) with the same data.
The value has no intrinsic meaning and there is no way to influence its value.
- There is no reliable way to know id of new entity; it must be created first.
- On the other hand, it is possible to predict assigned id value based on previous ones (because they are always assigned from increased sequence). This is not reliable, but it might be abused in some cases where random id is required.

For these reasons, each schema can define custom keys to identify entities by some other values. These keys can be used to construct alternative URL to access the entity.

Any number of these keys can be defined.
Each key is identified by its name. This name is used as part of the access URL.
- /entity/{schemaName}/key/{keyName}
Key can be simple (one value) or composite (several values at once).
- In the access URL, each key part has its own path segment (that is, they are separated by forward slash /).
- Examples:
  - /entity/{schemaName}/key/{keyName}/{simpleKeyValue} (simple key, one part)
  - /entity/{schemaName}/key/{keyName}/{keyPart1}/{keyPart2} (two parts)
  - /entity/{schemaName}/key/{keyName}/{keyPart1}/{keyPart2}/{keyPart3} (three parts)
Key parts have their own declaration, independent of the entity mapping itself. It is recommended to include key values in the entity itself, but it is not strictly required.
Each key part must be searchable by string value:
- Only scalar mapping is allowed (string, number, boolean).
- Any mapping that can be used as part of query in form property="value" is acceptable.
- Features are the most usual ones. Relation mappings are also possible, but less useful. Storage items, storage item attributes, etc are not searchable.
- null is not valid value for key part!
It is expected that each entity has key present (that is, all key parts are present and non-null), but not strictly required.
- Entities without key are omitted from key/view listings.
It is expected that each key value (tuple of values) is unique among applicable entities.
- Any attempt to access entity by key that is not unique results in failed request (http code 519).
Each key can also define additional query condition to filter out undesirable entities.
- This can (and should) be used to ensure that the key itself is really unique. See example below.
Key can be also declared as global view, making it accessible in a way that mimics another entity (hence the name "view").
- The URLs based on /view/{viewName} are actually just alias to /entity/{entityName}/{keyName}.
- View name can be different to the key name; it must be also globally unique.

Limitations and notable pitfalls

There is no mechanism ensuring that the keys are unique.
- There are some exceptions, features whose values are ensured to be unique by server:
  - censhare:asset.id - this one is probably never useful as part of custom key)
  - censhare:asset.id_extern - this one is also always ensured to be present, making it an ideal custom key
  - censhare:resource-key - good candidate for custom key, although its presence on asset is optional
- Uniqueness of any natural key must be ensured by application logic (for example, server-side automation).
- Note that the uniqueness is needed only among assets visible to HCMS, that is the assets with proper output channel.
  - In many cases, it is perfectly acceptable that the key is unique among this particular subset of assets, but there are many duplicates among all assets on server.
There is no mechanism to ensure that each entity has key value.
- This might actually be a desired behavior. Entities without key values are just inaccessible and automatically omitted from listings.
- When generating links, entity without valid key will be represented by its canonical id-based link (/entity/{name}/{id}).
- There are some exceptions, features that are always present on asset (enforced by censhare server).
  - censhare:asset.id_extern is always present (and unique) -censhare:asset.name is always present, but not unique in general -censhare:asset.domain and censhare:asset.domain2 are always present, but they are definitely not unique (there is a limited number of values available) making them ill-suited for custom key
Values of key parts are part of the url path, which means some severe limits:
- Special characters (including space) must be url-encoded. This should, in theory, make all strings acceptable; in reality, many values are mangled by various clients (browsers, libraries):
  - Empty string is theoretically acceptable, but some clients and proxies "correct" // to /.
  - Forward slash / can be, in theory, url-encoded; in reality, however, some clients thread %2f as path separator just like unescaped /.
  - Characters outside ASCII range (accented characters, non-latin alphabets) must be represented in some encoding. UTF-8 is the most common standard, but far for universally accepted.
    - Values outside Unicode BMP (for example emoji) are particularly problematic.
All key-based URLs are just aliases for the canonical id-based urls and behave the same.
- Request headers, query parameters, request body (in case of PUT and POST), response headers and response body are the same.
- The only difference is that generated links (media links, storage item links) are using key-based url prefix based on the request.
- Modification (PUT) and creation (POST) accept any valid entity, regardless of the key value.
  - PUT to key-based entity URL can invalidate that URL; the entity might be accessible by new URL with new key, but it might become inaccessible by this key at all.
  - POST to global view URL might create entity that is not accessible via this view.

Generated links

Links inside serialized entities are generated in key or view form only if all following conditions are met:

The request used view or key endpoint.
The link entity is of the same schema.
The link entity has valid key and conform to condition query (if present).

When these conditions are not met, standard link (/entity/{schemaName}/{id}) is created.

Note that these conditions are always met when the link entity is the serialized entity. This means that all storage item download links and media links automatically uses the link base from request.

Declaration in schema

Custom keys are defined in property cs:keys; its value must be JSON object. Each sub-property defines one key (property name is the key name).

Key definition is json object with one to three properties:

"key": defines key part(s); required
- Array of JSON objects, or single JSON object for simple key (equal to single-item array).
- Each value part is defined by JSON object that represents single property mapping. Usually, it's feature mapping.
  - Only single scalar mappings are accepted.
- Note that order is very important: key values must be provided in the same order they are defined here.
- It is recommended that the these mappings are also present in the entity schema itself, to provide convenient access to key values in serialized entity.
  - This is not, however, strict requirement; key values might be completely missing from the entity itself, or they might have different mapping.
  - There is one special case when different mapping is actually recommended: reference features (or relations) that are mapped as link in entity should be mapped as simple integer (asset id) in the key instead.
  - Note that the key parts might be located in various parts of the schema, with any name.
"condition": query expression used to filter entities (string); optional
"global": name of the global view (string); optional, no global view is defined by default
- Value must be globally unique; if some other schema already defines global view with the same name, schema creation/update will be refused.

Example: simple id_extern key

JSON

{
    "cs:keys": {
        "id_extern": {
            "key": {
                "type": "string",
                "cs:feature.key": "censhare:asset.id_extern"
            }
        }
    }
}

Example: global views for latest product and history

This example models (simplified) product information case: each product (identified by its unique global number) exists in many different versions. Only one version is considered valid ("latest") and these versions are accessible by convenience endpoint /view/product. All versions are available as another view /view/history, but this one uses composite key (product number and revision number).

Note that the latest key contains only subset of all entities, those with latest_revision=true. This means that older product versions are not listed nor accessible, but more importantly it makes the key unique.

JSON

{
    "cs:keys": {
        "latest": {
            "global": "product_latest",
            "key": {
                "cs:feature.key": "pim:product-number",
                "type": "string"
            },
            "query": "latest_revision=true"
        },
        "history": {
            "global": "product_history",
            "key": [
                {
                    "cs:feature.key": "pim:product-number",
                    "type": "string"
                },
                {
                    "cs:feature.key": "pim:revision-no",
                    "type": "integer"
                }
            ]
        }
    },
    "properties": {
        "product_no": {
            "cs:feature.key": "pim:product-number",
            "type": "string"
        },
        "revision": {
            "cs:feature.key": "pim:revision-no",
            "type": "integer"
        },
        "latest_revision": {
            "cs:feature.key": "pim:revision-latest",
            "type": "boolean"
        }
    }
}