Configuring Services using Profiles

This article explains the usage of service profiles in yuuvis® Momentumhow to add and use custom profiles.

Table of Contents

Introduction

The yuuvis® Momentum service architecture makes each service retrieve configuration files from a central location managed by the CONFIG service. These YAML configuration files extend the base application.yml configuration file inherent to any Spring boot microservice. Each of the additional configuration files corresponds to a profile, denoted by the suffix of the file, which the service of interest will need to be configured to run in.

The architecture of services and profiles follows the basic concepts of Spring Boot. For more detailed information, please refer to the Spring Boot profiles documentation.

So how is this helpful? There are two main benefits: 

  • They act as globally reusable configuration elements
    Configuration profiles allow for specific bits of configuration to be used by multiple services across the system. Any service can load the information from existing configuration files using its SPRING_PROFILES_ACTIVE environement variable. For example, a configuration file containing Elasticsearch connection parameters are loaded by both the SEARCH and INDEX services. 
  • Profiles also allow for the customization of specific service instances.
    For example, several instances of the REPOSITORY service could be used to separate data into different data sources with a configuration profile containing access parameters for each of the available data sources.


Names of Profiles

The name of the profile file defines whether it will be available to all services or to only one specific service: there are two types of profiles that are distighuished by their naming scheme:

  • Profiles containing information relevant to multiple services must adhere to the application-<profile type>.yml naming scheme in order to be referable to by any service in the system. 
  • Profiles specific to a single service have to be called <service-identifier>-<profile type>.yml

The individual services do not reference a specific profile file name but only the profile type. It is important to consider the strict hierarchy implemented in Spring Boot profile-based configurations. A configuration specific to a single service (i.e., system-prod.yml) always takes precedence over any global configuration (i.e., application-prod.yml), which means that overriding a global configuration is entirely possible.

Available Profiles

The following profiles are always installed. Click a profile name to open a more detailed description and find the available parameters.

Profile TypeProfile File Name(s)Referenced byDescription

prod, dev

service-specific configurations:

all services except CONFIG service

Separation of development and production environments. A service with prod (dev) as active profile is running in the production (development) environment.

Per default, all services are deployed in the production environment. If you want to build a test environment, you need to manually create the corresponding profiles.

The CONFIG service will always try to load parameters from an application-prod.yml file at the start even if no profile prod is used. 

kubernetesapplication-kubernetes.yml

Used in Kubernetes systems.
Services will use a Kubernetes-specific internal configuration.

Furthermore, the API gateway and SYSTEM service query the Kubernetes API Server for other services in the namespace.
The list of services is filtered with the condition from the kubernetes profile file.

redisapplication-redis.yml
  • API Gateway
  • INDEX Service
  • REGISTRY Service
  • REPOSITORY Service
  • SYSTEM Service
Redis connection parameters.
mqapplication-mq.ymlMessaging queue connection parameters.
dbsapplication-dbs.yml
  • AUDIT Service
  • REGISTRY Service
Database connection parameters.
esapplication-es.ymlElasticsearch connection parameters
Index configuration used to create Elasticsearch index.
storageapplication-storage.ymlBinary data storage connection parameters.
lcapplication-lc.ymlLifecycle configuration for asynchronous operations.
keycloak-
  • ORGANIZATION Service
Enables user-role-mapping in Keycloak.
dockerapplication-docker.yml

The profile docker is used to ensure that the services have read their profiles.
If the service is started with the xdocker profile, the docker.enabled application property must have the value true.

oauth2application-oauth2.ymlContains authentication related parameters.
metricsapplication-metrics.ymlProfile under development. In the future, metrics should provide a possibility to monitor responses from the service instance.
jpapostgres-
  • AUDIT Service
  • REGISTRY Service
Decides whether PostgreSQL wire protocol or MS SQL should be used for the database connection of the AUDIT and/or REGISTRY service.
If jpapostgres is used, the corresponding service uses the PostgreSQL driver that can connect either a PostgreSQL database or a database which implements the PostgreSQL wire protocol, e.g., CockroachDB.

noamqp-
  • API Gateway
  • COMMANDER Service

Decides whether a messaging provider should be used by the API gateway and the COMMANDER service or not.

If noamqp is NOT acitvated,

  • the API gateway connects to the messaging provider during the start up. Thus, a messaging provider is required to operate the API Gateway.
  • the COMMANDER service uses the messaging provider during reindexing processes. The messaging queue is used to request the full-text analysis via CONTROLLER and TEXTEXTRACTOR service.

If noamqp is activated,

  • the API gateway does not connect to the messaging provider. Thus, the API gateway cannot send new messages to CONTROLLER and TEXTEXTRACTOR service. Configuration parameters from the profile mq are ignored.
  • the reindexing of the Elasticsearch index via COMMANDER service is performed only for the objects' metadata. The full-text is removed from the reindexed objects. Configuration parameters from the profile mq are ignored.

Profiles in Kubernetes

Configuration during Installation

You can set the configuration parameter values in advance during installation.

The configuration parameters are set in the values.yaml file of the yuuvis Helm Chart. Modify a value of the listed parameters or add one of the available parameters listed in the above linked fact sheet of the corresponding profile with your desired value to overwrite its default value.

During the deployment, the settings defined in the values.yaml file will be written into the profile files that are read by the services when they are started. 

Changing the Configuration of a Running System

For changes on a running system, you need to access your Git server.

Navigate to the root directory of the configuration git repositories' appropriate branch and find a list of the configuration files including the above listed profiles. Open the profile, modify it, and check in your changes.

After the modification of a profile, you need to restart the services that reference the profile in order to apply the changes.

Adding Custom Profiles

New profiles can be added by committing their configuration YAML files to the root directory of the configuration git repositories' appropriate branch.

To activate any custom profile-based configuration, the services that are meant to use them need to set them as their active Spring Profiles.   

In Kubernetes, this can be accomplished by modifying the deployment file of the service. Using kubectl edit -n <namespace> deployment <service identifier>, the active profiles of a given service can be changed. The active profiles are listed under the SPRING_PROFILES_ACTIVE environment variable. Any changes will incur a restart of the services.

Outsourcing Passwords from Profiles as Kubernetes Secrets

Passwords can be outsourced from profile files as Kubernetes Secrets. In the configuration file, placeholders are included instead that are replaced by the actual passwords during the pod's runtime. Please note that passwords as Kubernetes Secrets can be displayed in plain text format in the environment variables of the pods and contained services. The following guideline explains the configuration procedure for the password corresponding to the database connection. The same procedure can also be applied to any other password contained in a profile file.

  • Convert the password in the application-dbs.yml file to Base64 format.
    Example command for Windows Powershell: [Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes("changeme"))
    Example command for Linux: echo -n "changeme" | base64 -w0
    Both example commands lead to the output Y2hhbmdlbWU=.
  • Create a yuuvis-secrets.yml file with the following content:

    apiVersion: v1
    kind: Secret
    metadata:
      name: yuuvis-secrets
      namespace: yuuvis
    type: Opaque
    data:
      POSTGRES_PASSWORD: Y2hhbmdlbWU=

    The section data contains a list of key-value pairs that can contain multiple entries. As spacers within the string key name, only underscores are allowed. The value is the password in Base64 format.

  • Create a Kubernetes Secret by running the command kubectl apply -f yuuvis-secrets.yml.
  • Replace the password in the application-dbs.yml file by a placeholder:

    yuuvis.db.password: ${POSTGRES_PASSWORD}
  • Adjust the deployments of all yuuvis® services using the profile dbs by extending the section env:

    env:
    - name: POSTGRE_PASSWORD
      valueFrom:
        secretKeyRef:
          name: yuuvis-secrets
          key: POSTGRE_PASSWORD

    The name and the key given in secretKeyRef have to correspond to the name and the data value defined in the file yuuvis-secrets.yml.

Summary

Profiles act as globally reusable configuration elements or allow for the customization of specific service instances. The file name defines whether the configured parameters are available to multiple services or only one specific service. The profiles can be modified directly before the deployment or in a running system. Custom profiles can be added and referenced as well in addition to the always installed profiles. Sensitive data like passwords can be outsourced from the profiles and stored in Kubernetes Secrets instead.

Read on

yuuvis® Momentum Services

The services yuuvis® Momentum Core is made of. Keep reading

Basic Use Case Flows

Graphical overviews describing the interaction of the yuuvis® Momentum core services in exemplary basic use case flows.  Keep reading

Login to the Core API (Java)

This tutorial will give you a brief overview of the most important login procedures for the yuuvis® API system and how to use them in a Java Client application Keep reading