Beginning with yuuvis yuuvis® RAD version 9.0, the Extraction Service extraction-service can be extended by plugins.
This is based on the framework PF4J: https://pf4j.org/
Inside the extractionservice the plugins are loaded via PF4J-Spring:
https://github.com/pf4j/pf4j-spring
...
plug-ins.
Table of Contents | ||||
---|---|---|---|---|
|
Including an Existing Plug-in in the Extraction-Service
The deployment of an existing plugin plug-in is explained by setting up the an example plugin plug-in for processing PGN files: https://en.wikipedia.org/wiki/Portable_Game_Notation
...
PGN files.
Configuring the Extraction-Service
By default, the use of plugins plug-ins is disabled. In the extraction-prod.yml
file, these settings can be adjusted:
Code Block | ||
---|---|---|
| ||
plugins: enabled: true directory: "'../../data/plugins"' |
The jar file of the plugin plug-in must then be stored in the specified directory:
View file | ||
---|---|---|
|
The plugin is activated when the Extraction Service is restartedextraction-service must be restarted to enable the plug-in.
Any number of plugin plug-in files can be stored in this directory.
...
Checking the
...
Logging of the
...
Plug-in
As soons as the extraction-service has been restarted, available plug-ins are read indetected. Information about this is output on these plug-ins can be found in the log.
If the "plugins.enabled
" switch has is not been set to "true
", this the following will be indicated protocolled in the log:
Code Block | ||
---|---|---|
| ||
... c.o.s.e.routing.PluginProcessor : plugins not enabled |
If the plugins have been activated but no plugins have been put plug-ins are enabled, but not stored in the directory, the log will look as follows:
Code Block | ||
---|---|---|
| ||
... c.o.s.e.routing.PluginProcessor : plugins enabled: ../../data/plugins ... c.o.s.e.routing.PluginProcessor : plugins found: 0 |
If plugins plug-ins are found, they are will be listed with their names and "namespaces
":
Code Block | ||
---|---|---|
| ||
... c.o.s.e.routing.PluginProcessor : plugins enabled: ./../data/plugins ... c.o.s.e.routing.PluginProcessor : plugins found: 2 ... c.o.s.e.routing.PluginProcessor : plugin 'PgnFile': PGNFILE ... c.o.s.e.routing.PluginProcessor : plugin 'CAD-File': CAD |
When the plugins become If plug-ins are active for a specific file, the number of each extracted is plug-in will be in the log:
Code Block | ||
---|---|---|
| ||
... c.o.s.e.routing.PluginProcessor : extracted by plugin 'PgnFile': 7 |
Result of the
...
Extraction
View file | ||
---|---|---|
|
...
Code Block | ||
---|---|---|
| ||
"OS:FileName": "schach_PGN.pgn", "OS:FileSize": "740", "OS:ModifyDate": "2023-02-10T09:42:24+01:00", "OS:CreateDate": "2023-02-10T09:42:24+01:00", "OS:FileType": "TXT", "OS:MimeType": "text/plain", "ExifTool:ExifToolVersion": "12.52", "System:FileName": "schach_PGN.pgn", "System:Directory": "C:/Temp/Local/Temp/20230210.094224.115-1/389a8d00-25f8-4f84-9d08-238f0e81594c", "System:FileSize": "740", "System:FileModifyDate": "2023:02:10 09:42:24+01:00", "System:FileAccessDate": "2023:02:10 09:42:24+01:00", "System:FileCreateDate": "2023:02:10 09:42:24+01:00", "System:FilePermissions": "100666", "File:FileType": "TXT", "File:FileTypeExtension": "TXT", "File:MIMEType": "text/plain", "File:MIMEEncoding": "us-ascii", "File:Newlines": "\n", "File:LineCount": "16", "File:WordCount": "158", "PGNFILE:Site": "Belgrade, Serbia JUG", "PGNFILE:White": "Fischer, Robert J.", "PGNFILE:Event": "F/S Return Match", "PGNFILE:Round": "29", "PGNFILE:Black": "Spassky, Boris V.", "PGNFILE:Date": "1992-11-04T12:00:00+01:00", "PGNFILE:Result": "1/2-1/2", |
Creating
...
It follows the description of what to set up and configure for creating a new plugin, deploying, and checking it.
...
a Plug-in
This section describes how to set up and configure a new plug-in including its deployment and check.
The plug-in capability of the extraction-service is based on the PF4J framework.
The extraction-service loads plug-ins via PF4J-Spring.
The plug-in must comply with the interface defined by PF4J-Spring. In particular, this is a constructor with a PluginWrapper as a parameter:
Code Block | ||
---|---|---|
| ||
public PgnFilePlugin(PluginWrapper wrapper) { super(wrapper); } |
1. Include the library provided by Optimal Systems for the definition of the plugin interface
...
Including the Library for the Definition of the Plug-in Interface
OPTIMAL SYSTEMS provides a library for the defintion of the plug-in interface. The extractionservice-plugin-interface
" .jar
file is available as a jar file for download below and must be added to the project. This It defines the interface for the Extraction Service extraction-service and also brings the plugin plug-in capability of PF4J.
View file | ||
---|---|---|
|
It is important that the dependency of the plugin on plug-in to the interface library is defined as "provided
" (in gradle "compileOnly
"):
Code Block | ||
---|---|---|
| ||
<dependency> <groupId>com.os.enaio.services.extraction</groupId> <artifactId>extractionservice-plugin-interface</artifactId> <version>9.0.2</version> <scope>provided</scope> </dependency> |
...
Implementing the Plug-in
As an example of a plugin the library "plug-in, the extractionservice-plugin-demo
" .jar
file is also available as a jar-fileavaible for download:
View file | ||
---|---|---|
|
This Example The following example operates on PGN-Files: https://en.wikipedia.org/wiki/Portable_Game_Notation Plugin files.
Plug-in example code:
Code Block | ||
---|---|---|
| ||
package com.os.services.extraction.plugin.demo; import com.os.services.extraction.plugin.ExtractionDriver; import com.os.services.extraction.plugin.ExtractionInfo; import org.apache.commons.io.IOUtils; import org.pf4j.Extension; import org.pf4j.Plugin; import org.pf4j.PluginWrapper; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import java.io.File; import java.io.FileInputStream; import java.util.HashMap; import java.util.Map; public class PgnFilePlugin extends Plugin { /** * Constructor to be used by plugin manager for plugin instantiation. * Your plugins have to provide constructor with this exact signature to * be successfully loaded by manager. * * @param wrapper */ public PgnFilePlugin(PluginWrapper wrapper) { super(wrapper); } private final static Logger LOGGER = LoggerFactory.getLogger(PgnFilePlugin.class); @Override public void start() { LOGGER.info("Start"); } @Override public void stop() { LOGGER.info("Stop"); } @Extension public static class PgnFileProcessor implements ExtractionDriver { private final static Logger LOGGER = LoggerFactory.getLogger(PgnFileProcessor.class); private static final String PREFIX = "PGNFILE"; @Override public ExtractionInfo info() { return null; } @Override public String getExtractionDriverName() { return "PgnFile"; } @Override public String getExtractionNamespace() { return PREFIX; } @Override public boolean isApplicable (File file) { if (file.getName().endsWith(".pgn")) { return true; } return false; } @Override public Map<String, String> extractData( File file ) { Map<String, String> result = new HashMap<>(); try { String str = IOUtils.toString(new FileInputStream(file), "UTF-8"); String[] tokens = str.split("]"); if (tokens != null) { for (String token : tokens) { token = token.trim(); if (token.startsWith("[")) { String[] tag = token.substring(1).split("\""); if (tag.length >= 2) { //do not add prefix to key //it will be set by extractionservice String key = tag[0].trim(); String value = tag[1].trim(); if (key.endsWith("Date")) { value = value.replaceAll("\\?{2}", "01") .replaceAll("\\.", "\\-") + "T12:00:00+01:00"; } result.put(key, value); } if (tag.length == 1) { //do not add prefix to key //it will be set by extractionservice String key = tag[0].trim(); result.put(key, ""); } } } } } catch (Exception e) { LOGGER.error(e.getMessage()); } return result; } } } |
...
Building the jar
...
File
If the plugin plug-in is packaged in a jar
file, there are important points to note:
The following attributes must be specified in the MANIFEST.MF
file (in the example pluginplug-in):
Code Block | ||
---|---|---|
| ||
Plugin-Class: com.os.services.extraction.plugin.demo.PgnFilePlugin Plugin-Dependencies: Plugin-Id: PgnFileProcessor-Plugin Plugin-Provider: os Plugin-Version: 9.0.0 |
To have for these values to be automatically inserted into the jar
file, the "maven-assembly-plugin
" can be configured accordingly:
Code Block | ||
---|---|---|
| ||
<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-assembly-plugin</artifactId> <version>3.1.0</version> <configuration> <descriptorRefs> <descriptorRef>jar-with-dependencies</descriptorRef> </descriptorRefs> <finalName>${project.artifactId}-${project.version}-all</finalName> <appendAssemblyId>false</appendAssemblyId> <attach>false</attach> <archive> <manifest> <addDefaultImplementationEntries>true</addDefaultImplementationEntries> <addDefaultSpecificationEntries>true</addDefaultSpecificationEntries> </manifest> <manifestEntries> <Plugin-Id>${plugin.id}</Plugin-Id> <Plugin-Version>${plugin.version}</Plugin-Version> <Plugin-Provider>${plugin.provider}</Plugin-Provider> <Plugin-Class>${plugin.class}</Plugin-Class> <Plugin-Dependencies>${plugin.dependencies}</Plugin-Dependencies> </manifestEntries> </archive> </configuration> <executions> <execution> <id>make-assembly</id> <phase>package</phase> <goals> <goal>single</goal> </goals> </execution> </executions> </plugin> |
The jar
file must contain all required libraries (except extractionservice-plugin-interface
and thus also pf4j).
The MANIFEST.MF
file must have the appropriate entries.
The extensions.idx
file must be included. This It is automatically generated by PF4J during build.
This generated jar
file can now be placed in the appropriately configured directory of the Extractionserviceextraction-service.
...
Checking the
...
Plug-in
To check the new plugin, the extractionservice extraction-service must be configured like as described in Including an existing plugin in the extractionservice.