Skip to main content

Explaining the Core DPU Files Generated by the DPU Template

Abstract

Explaining the Core DPU Files Generated by the DPU Template

This section contains a short guide on files that are being created by the DPU archetype.

The previous section explained how the skeleton for a new DPU may be prepared.

This section describes the files generated by the DPU template. Let’s start with a short list to deliver the general idea, afterward more important files will be explained in detail.

Note

The discussion below assumes that you used the same value when setting up the creation of DPU from DPU template.

List of Generated Files

  • DPU source code files:

    • MyDpu.java - M ain DPU class. It contains the method 'innerExecute()' where all the business logic of the DPU goes and which is called when the DPU is executed.

    • MyDpuVaadinDialog - Vaadin configuration dialog. Every DPU can provide a Vaadin configuration dialog. The dialog is presented to pipeline designers in the frontend part of UnifiedViews, so that they can configure the DPU via a graphical interface.

    • MyDpuConfig_V1 - DPU's Configuration. The configuration class must have private fields with public getters/setters for each field. This class is used on the one hand by the main DPU class of the DPU (to configure the DPU as it is executed). On the other hand it used by Vaadin configuration dialog to configure the dialog or to store the configuration from the dialog.

  • DPU documentation files:

    • README.md

    • CHANGELOG.md

    • doc/About.md: English version of the user documentation

    • doc/About_{languageTag}.md (optional): here {languageTag} can be 'sk' for Slovak localization, for example.

  • DPU resource files for localization, localized messages, localized DPU's name:

    • src/main/resources/resources.properties: this file contains text that is presented to the user as a message or as a part of a configuration dialog. In our case the file contains two messages.

      Note

      It is highly recommended to put all the strings inside this file and not directly into code as it makes DPU localization much easier.

    • src/main/resources/resources_{languageTag}.properties, here, too, {languageTag} can be 'sk' for Slovak localization, for example.

DPU Source Code FilesMain DPU Class

Source code file, class: MyDpu.java, further referred to simply as 'MyDpu' class.

The generated MyDpu class looks as follows (imports are skipped):

@DPU.AsTransformer
public class MyDpu extends AbstractDpu<MyDpuConfig_V1> {
    private static final Logger LOG = LoggerFactory.getLogger(MyDpu.class);
                
    @ExtensionInitializer.Init
    public FaultTolerance faultTolerance;
        public MyDpu() {
                super(MyDpuVaadinDialog.class, ConfigHistory.noHistory(MyDpuConfig_V1.class));
        }
                
    @Override
    protected void innerExecute() throws DPUException {
        ContextUtils.sendShortInfo(ctx, "MyDpu.message");        
    }
}

  • Line 1 contains an annotation, which defines the DPU type, in our case it is a transformer. Other options are DPU.AsExtractor, DPU.AsLoader, DPU.asQuality, which corresponds with the available DPU types.

  • Line 2 shows that every DPU prepared should extend AbstractDpu class, base class for DPUs. The parameter of the AbstractDpu class is the configuration class, in this case MyDpuConfig_V1. The usage of configuration class as a template parameter ensures compatibility with the configuration dialog.

  • Line 3 defines the Logger object which should be used when the DPU developer wants to log certain information about the DPU's execution. Only reasonable size messages and reasonable amount of messages should be logged as all the logs are stored in the database.

  • Lines 10 - 12 contain the draft of the main execution method, which is called when the DPU is executed. This is the place where the DPU developer should put the business logic of the DPU.

AbstractDpu class, this DPU class inherits from, provides two important variables.

They can be used inside of the main execution method innerExecute():

  • config - points to DPU configuration - the instance of the MyDpuConfig_V1 class

  • ctx - holds instance of the execution context.

The innerExecute method can throw a DPUException.

This exception should be used in case the DPU developer would like to express failure of the DPU due to error in the DPU execution or due to cancellation of the DPU's execution. Some helpers and other classes also throw this exception. In general this exception is considered to be a non-recovery failure, which means, the DPU should fail.

  • In Line 14, the ContextUtils class is used to send an event. Events are another way of communication with the user, they present a high level overview of the DPU's execution. When compared with using Logger from Line 3, events should be send only in significant cases, so the rule of thumb is that every DPU should send just a couple of these events. For example, information about a number of input entries processed. Also another difference is if an ERROR level event is sent, then the DPU is considered to have failed. It finishes with ERROR.

    • In this case, in Line 11, an information message is send.

ContextUtils also provides other methods for message sending and a method for creation of DPUException.

Every string passed (besides args) to ContextUtils goes through the localization process. The second parameter of sendShortInfo() method in Line 11 actually points to the file with resources. Translations of the message, so the method sendShortInfo()automatically supports localization. For this reason, developers are encouraged to use ContextUtils for message sending and creation of DPUException.

Note

As a DPU developer you yourself must check for requests for cancellation of execution. This check should be implemented e.g. after/before every new entry (file, graph) is being processed by the DPU.

This can be done by using the following code fragment:

if (ctx.canceled()) {
    log.info("DPU execution was cancelled.");
    break;
}
Vaadin Configuration Dialog

Source code file, class: MyDpuVaadinDialog.java, further referred to simply as 'MyDpuVaadinDialog' class.

The generated MyDpuVaadinDialog class looks as follows, imports are skipped:

public class MyDpuVaadinDialog extends AbstractDialog<MyDpuConfig_V1> {

    public MyDpuVaadinDialog() {
        super(MyDpu.class);
    }
    
        @Override
    public void setConfiguration(MyDpuConfig_V1 c) throws DPUConfigException {
    }
    
        @Override
    public MyDpuConfig_V1 getConfiguration() throws DPUConfigException {
        final MyDpuConfig_V1 c = new MyDpuConfig_V1();
        return c;
    }
    
        @Override
    public void buildDialogLayout() {
        final VerticalLayout mainLayout = new VerticalLayout();
        mainLayout.setWidth("100%");
        mainLayout.setHeight("-1px");
        mainLayout.setMargin(true);
        
                mainLayout.addComponent(new Label(ctx.tr("MyDpu.dialog.label")));
        
                setCompositionRoot(mainLayout);
    }
}

  • Line 1 shows that the Vaadin configuration dialog class of the DPU always extends AbstractDialog class. We use the base dialog abstract class with our configuration as a template parameter. The template parameter ensures compatibility of configuration between Main DPU class and Vaadin configuration dialog class.

  • Lines 3 - 5 represent constructor for the Vaadin configuration class. In the constructor we need to pass the main DPU class to the AbstractDialog class. Thanks to this, AbstractDialog can scan the DPU class for configuration history and extensions.

  • In Lines 7- 9 the configuration dialog class is given a configuration object to load. So the DPU developer has to specify in this method how the values from the configuration object should be loaded to the Vaadin components. This method is called when the configuration dialog needs to be prepared for the pipeline designer.

  • In Lines 11 - 15, the DPU developer should specify how the values taken from the Vaadin components should be serialized to the configuration object. This is the last place where any validation of the configuration can be done. This method is called when the pipeline designer saves the configuration dialog. In case of invalid configuration DPUConfigException should be thrown by the DPU developer.

  • Lines 17 - 27 create Vaadin dialog layout. As specified in Line 20, the width of the dialog is set to '100%', so it takes the whole space in the horizontal level.

  • Line 21 specifies height to be ' -1px', which means that the layout takes only as much vertical space as needed.

  • In Line 24 the ctx.tr returns the value assigned to the given string MyDpu.dialog.label in resources.properties, thus, it returns the proper localized version of the label. If no value is presented in resource.properties, the string MyDpu.dialog.label is returned. It is highly recommended to externalize messages to resources.properties – out of the code.

    Additionally, Line 24 creates a Label with the text. The Label is a simple read-only Vaadin component for displaying text. Finally, mainLayout. addComponent() method in Line 24 adds the given component, the Label, to the layout. If we would not call this method, the Label would not be presented in the layout/dialog.

  • Line 26 sets the above defined layout mainLayout as the root layout of the dialog.

For more information about Vaadin component’s and layouts, please refer to the Vaadin homepage, where you can find the excellent Book of Vaadin.

DPU Configuration Class

Source code file, class: eu.mycompany.uv.dpu.transformer.mydpu.MyDpuConfig_V1.java, further referred to simply as 'MyDpuConfig_V' class.

The generated MyDpuConfig_V1 class looks as follows (imports are skipped):

public class MyDpuConfig_V1 {
    public MyDpuConfig_V1() {
    }
}

Note

As can be seen in Lines 1 - 4, this a normal Java class, which has non-parametric constructor. The configuration class must have private fields with public getters/setters for each configuration field. This class is used on one side by the main DPU class of the DPU (to configure the DPU as it is executed) and on the other side by the Vaadin configuration dialog to configure the dialog or to store the configuration from the dialog.

DPU Documentation Files:

The DPU template generates the following three files:

  • README.md: this file is the main file, which points to CHANGELOG.md file and user documentation files (doc/About.md). Further it includes technical documentation if needed.

  • CHANGELOG.md: this file holds the version history.

  • doc/About.md: this file contains the user documentation, which is also displayed to DPU users in UnifiedViews. Such user documentation may also be localized.

Note

UnifiedViews frontend shows the user documentation involved in doc/About.md - in the proper version based on the localization.

File README.md

The generated file looks as follows; the file follows Markdown syntax:

MyDpu
----------

### Documentation

* see [Plugin Documentation](./doc/About.md)

### Technical notes

* May be used instead of DPU X, it also replaces DPY Y.

### Version history

* see [Changelog](./CHANGELOG.md)

  • In Line 6, README file contains links to the user documentation (About.md, see below). Line 14 contains links to Changelog file (see below). Further, if you would like to specify certain technical notes, you may specify them on Line 10.

File CHANGELOG.md

The generated file looks as follows; the file follows Markdown syntax:

MyDpu
----------

v1.0.0-SNAPSHOT
---
* Initial version of the DPU

  • Lines 4-6 contains the description of the initial version.

Note

Whenever you change the version in the future, you should also specify the new version and features included in such version here.

Basically for each new version you have to add the following fragment to the start:

v1.1
---
* Description of the bugfix/feature 1
* Description of the bugfix/feature 2
File doc/About.md

The content of the About.md file contains the user documentation. As the DPU is built, it is converted to About.html file, which is then displayed in the UnifiedViews frontend in the DPU detail as user documentation.

The file generated by the template looks as follows; the file follows the Markdown syntax:

### Description

Description of the DPU intended for the user of the DPU. 

### Configuration parameters

| Name | Description |
|:----|:----|
|**Param** | Param which ...|

### Inputs and outputs

|Name |Type | DataUnit | Description | Mandatory |
|:--------|:------:|:------:|:-------------|:---------------------:|
|input |i |FilesDataUnit |Input files |x|
|output |o |RdfDataUnit |Produced RDF data |x|

  • Line 3 contains description of the DPU, which is then depicted to user. This should be quite short, but should explain to nontechnical users what the DPU does.

  • Lines 5-10 contain description of parameters which are available in Vaadin configuration dialog.

  • Lines 11-16 contains description of input/output data units which the DPU uses. Every data unit contains its name (as in Main DPU class), type (i = it is input data unit, o = it is output data unit), dataUnit (FilesDataUnit for files data unit, RdfDataUnit for rdf data unit), description and flag, whether it is mandatory or not (if it is mandatory, pipeline designer has to use such input/output data unit).

A sample file converted to HTML looks as follows:

24577650.png

If you want to support also user documentation in a language other than English, you can create more such user documentation files and store them as doc/About_{languageTag}.md, where {languageTag} is for example 'sk' for Slovak localization.

UnifiedViews automatically ensures that the proper user documentation is loaded based on the localization of the whole application.

DPU Resource Files for Localization - Localized Messages, Localized DPU Name

The file generated by the template looks as follows, it contains key=value pairs:

# Contains localization of used string in dialog and events. To avoid property collision all values
# must be prefixed by '{dpu name}.' for DPU execution and '{dpu name}.dialog' for configuration dialog!
# Additional prefix in form of dpu name is optional.

# DPU execution.
MyDpu.message = DPU is running ...

# DPU's dialog.
MyDpu.dialog.label = DPU's configuration

  • The generated file contains key-value pairs holding localized versions of strings used by the DPU .

  • You have to prefix message identifiers (keys) with ‘{dpu name}.’ or ‘{dpu name}.dialog’ based on their usage during DPU execution or in dialog respectively.

  • The generated file should contain English strings.

  • If you want to support also localizations in other languages than English, you may create more of such resource files and store them as resource_{languageTag}.properties, where {languageTag} is for example 'sk' for Slovak localization. UnifiedViews automatically ensures that the proper messages are used in the DPUs based on the localization of the whole application.

In this section: