Beginning with yuuvis® RAD 9.0, the extraction-service can be extended by plug-ins.
This is based on the PF4J framework. https://pf4j.org/
Inside the extractionservice the plugins are loaded via PF4J-Spring:
https://github.com/pf4j/pf4j-spring
| Table of Contents | ||||
|---|---|---|---|---|
|
Including an
...
Existing Plug-in in the Extraction-Service
The deployment of an existing plugin plug-in is explained by setting up the an example plugin plug-in for processing PGN files: https://en.wikipedia.org/wiki/Portable_Game_Notation
...
Configuring the Extraction-Service
By default, the use of plugins plug-ins is disabled. In the extraction-prod.yml file, these settings can be adjusted:
| Code Block | ||
|---|---|---|
| ||
plugins: enabled: true directory: "'../../data/plugins"' |
The jar file of the plugin plug-in must then be stored in the specified directory:
| View file | ||
|---|---|---|
|
The plugin is activated when the Extraction Service is restartedextraction-service must be restarted to enable the plug-in.
Any number of plugin plug-in files can be stored in this directory.
...
Checking the
...
Logging of the
...
Plug-in
As soons as the extraction-service has been restarted, available plug-ins are read indetected. Information about this is output on these plug-ins can be found in the log.
If the "plugins.enabled" switch has is not been set to "true", this the following will be indicated protocolled in the log:
| Code Block | ||
|---|---|---|
| ||
... c.o.s.e.routing.PluginProcessor : plugins not enabled |
If the plugins have been activated but no plugins have been put plug-ins are enabled, but not stored in the directory, the log will look as follows:
| Code Block | ||
|---|---|---|
| ||
... c.o.s.e.routing.PluginProcessor : plugins enabled: ../../data/plugins ... c.o.s.e.routing.PluginProcessor : plugins found: 0 |
If plugins plug-ins are found, they are will be listed with their names and "namespaces":
| Code Block | ||
|---|---|---|
| ||
... c.o.s.e.routing.PluginProcessor : plugins enabled: ./../data/plugins ... c.o.s.e.routing.PluginProcessor : plugins found: 2 ... c.o.s.e.routing.PluginProcessor : plugin 'PgnFile': PGNFILE ... c.o.s.e.routing.PluginProcessor : plugin 'CAD-File': CAD |
When the plugins become If plug-ins are active for a specific file, the number of each extracted is plug-in will be in the log:
| Code Block | ||
|---|---|---|
| ||
... c.o.s.e.routing.PluginProcessor : extracted by plugin 'PgnFile': 7 |
...
Result of the
...
Extraction
| View file | ||
|---|---|---|
|
...
| Code Block | ||
|---|---|---|
| ||
"OS:FileName": "schach_PGN.pgn", "OS:FileSize": "740", "OS:ModifyDate": "2023-02-10T09:42:24+01:00", "OS:CreateDate": "2023-02-10T09:42:24+01:00", "OS:FileType": "TXT", "OS:MimeType": "text/plain", "ExifTool:ExifToolVersion": "12.52", "System:FileName": "schach_PGN.pgn", "System:Directory": "C:/Temp/Local/Temp/20230210.094224.115-1/389a8d00-25f8-4f84-9d08-238f0e81594c", "System:FileSize": "740", "System:FileModifyDate": "2023:02:10 09:42:24+01:00", "System:FileAccessDate": "2023:02:10 09:42:24+01:00", "System:FileCreateDate": "2023:02:10 09:42:24+01:00", "System:FilePermissions": "100666", "File:FileType": "TXT", "File:FileTypeExtension": "TXT", "File:MIMEType": "text/plain", "File:MIMEEncoding": "us-ascii", "File:Newlines": "\n", "File:LineCount": "16", "File:WordCount": "158", "PGNFILE:Site": "Belgrade, Serbia JUG", "PGNFILE:White": "Fischer, Robert J.", "PGNFILE:Event": "F/S Return Match", "PGNFILE:Round": "29", "PGNFILE:Black": "Spassky, Boris V.", "PGNFILE:Date": "1992-11-04T12:00:00+01:00", "PGNFILE:Result": "1/2-1/2", |
Creating a
...
Plug-in
This section describes how to set up and configure for creating a new plugin, deploying, and checking itplug-in including its deployment and check.
The plugin plug-in capability of the Extraction Service extraction-service is based on the PF4J framework PF4J: https://pf4j.org/
The Extraction Service loads the plugins extraction-service loads plug-ins via PF4J-Spring:
https://github.com/pf4j/pf4j-spring
The plugin plug-in must comply with the interface defined by PF4J-Spring. In particular, this is a constructor with a PluginWrapper as a parameter:
| Code Block | ||
|---|---|---|
| ||
public PgnFilePlugin(PluginWrapper wrapper) {
super(wrapper);
} |
Include the library provided by Optimal Systems for the definition of the plugin interface
...
Including the Library for the Definition of the Plug-in Interface
OPTIMAL SYSTEMS provides a library for the defintion of the plug-in interface. The extractionservice-plugin-interface" .jar file is available as a jar file for download below and must be added to the project. This It defines the interface for the Extraction Service extraction-service and also brings the plugin plug-in capability of PF4J.
| View file | ||
|---|---|---|
|
It is important that the dependency of the plugin on plug-in to the interface library is defined as "provided" (in gradle "compileOnly"):
| Code Block | ||
|---|---|---|
| ||
<dependency> <groupId>com.os.enaio.services.extraction</groupId> <artifactId>extractionservice-plugin-interface</artifactId> <version>9.0.2</version> <scope>provided</scope> </dependency> |
...
Implementing the
...
Plug-in
As an example of a plugin the library "plug-in, the extractionservice-plugin-demo" .jar file is also available as a jar-fileavaible for download:
| View file | ||
|---|---|---|
|
This Example The following example operates on PGN-Files: https://en.wikipedia.org/wiki/Portable_Game_Notation Plugin files.
Plug-in example code:
| Code Block | ||
|---|---|---|
| ||
package com.os.services.extraction.plugin.demo;
import com.os.services.extraction.plugin.ExtractionDriver;
import com.os.services.extraction.plugin.ExtractionInfo;
import org.apache.commons.io.IOUtils;
import org.pf4j.Extension;
import org.pf4j.Plugin;
import org.pf4j.PluginWrapper;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.File;
import java.io.FileInputStream;
import java.util.HashMap;
import java.util.Map;
public class PgnFilePlugin extends Plugin {
/**
* Constructor to be used by plugin manager for plugin instantiation.
* Your plugins have to provide constructor with this exact signature to
* be successfully loaded by manager.
*
* @param wrapper
*/
public PgnFilePlugin(PluginWrapper wrapper) {
super(wrapper);
}
private final static Logger LOGGER = LoggerFactory.getLogger(PgnFilePlugin.class);
@Override
public void start() {
LOGGER.info("Start");
}
@Override
public void stop() {
LOGGER.info("Stop");
}
@Extension
public static class PgnFileProcessor implements ExtractionDriver {
private final static Logger LOGGER = LoggerFactory.getLogger(PgnFileProcessor.class);
private static final String PREFIX = "PGNFILE";
@Override
public ExtractionInfo info() {
return null;
}
@Override
public String getExtractionDriverName() {
return "PgnFile";
}
@Override
public String getExtractionNamespace() {
return PREFIX;
}
@Override
public boolean isApplicable (File file) {
if (file.getName().endsWith(".pgn")) {
return true;
}
return false;
}
@Override
public Map<String, String> extractData( File file ) {
Map<String, String> result = new HashMap<>();
try {
String str = IOUtils.toString(new FileInputStream(file), "UTF-8");
String[] tokens = str.split("]");
if (tokens != null) {
for (String token : tokens) {
token = token.trim();
if (token.startsWith("[")) {
String[] tag = token.substring(1).split("\"");
if (tag.length >= 2) {
//do not add prefix to key
//it will be set by extractionservice
String key = tag[0].trim();
String value = tag[1].trim();
if (key.endsWith("Date")) {
value = value.replaceAll("\\?{2}", "01")
.replaceAll("\\.", "\\-") + "T12:00:00+01:00";
}
result.put(key, value);
}
if (tag.length == 1) {
//do not add prefix to key
//it will be set by extractionservice
String key = tag[0].trim();
result.put(key, "");
}
}
}
}
} catch (Exception e) {
LOGGER.error(e.getMessage());
}
return result;
}
}
} |
...
Building the jar
...
File
If the plugin plug-in is packaged in a jar file, there are important points to note:
The following attributes must be specified in the MANIFEST.MF file (in the example pluginplug-in):
| Code Block | ||
|---|---|---|
| ||
Plugin-Class: com.os.services.extraction.plugin.demo.PgnFilePlugin Plugin-Dependencies: Plugin-Id: PgnFileProcessor-Plugin Plugin-Provider: os Plugin-Version: 9.0.0 |
To have for these values to be automatically inserted into the jar file, the "maven-assembly-plugin" can be configured accordingly:
| Code Block | ||
|---|---|---|
| ||
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.1.0</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<finalName>${project.artifactId}-${project.version}-all</finalName>
<appendAssemblyId>false</appendAssemblyId>
<attach>false</attach>
<archive>
<manifest>
<addDefaultImplementationEntries>true</addDefaultImplementationEntries>
<addDefaultSpecificationEntries>true</addDefaultSpecificationEntries>
</manifest>
<manifestEntries>
<Plugin-Id>${plugin.id}</Plugin-Id>
<Plugin-Version>${plugin.version}</Plugin-Version>
<Plugin-Provider>${plugin.provider}</Plugin-Provider>
<Plugin-Class>${plugin.class}</Plugin-Class>
<Plugin-Dependencies>${plugin.dependencies}</Plugin-Dependencies>
</manifestEntries>
</archive>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin> |
The jar file must contain all required libraries (except extractionservice-plugin-interface and thus also pf4j).
The MANIFEST.MF file must have the appropriate entries.
The extensions.idx file must be included. This It is automatically generated by PF4J during build.
This generated jar file can now be placed in the appropriately configured directory of the Extractionserviceextraction-service.
Checking the
...
Plug-in
To check the new plugin, the extractionservice extraction-service must be configured like as described in 1 Including an existing plugin in the extractionservice.