Release Notes - PoolParty 7.1

This page contains the release notes for PoolParty release 7.1. In the release notes you can find major developments, improvements and changes made in the respective release.

Details on how to upgrade from your present version find here: PoolParty Upgrades

The release notes contain three chapters, highlights, improvements and fixes:

Highlights

Synchronize Ontology and Custom Scheme

The Ontology Management has seen a major improvement: you can now synchronize Custom Schemes and their Ontologies by a simple click. Thus any changes in an ontology are immediately propagated to the custom scheme.

In an opened custom scheme (1) you click the Add icon (2) beside Synchronized Ontologies to open the dialogue. You can start typing in the search field to search for ontologies.

Activate the check box in front of its name, multi-select is also possible. Afterwards in that section the synchronized ontologies will be listed.

27892024.png
27891978.png

This procedure is the same for Ontologies, in reverse, you select custom schemes to synchronize and they will be listed in the corresponding section afterwards.

Once the synchronization is established, synced elements are displayed in different color in the tree. These elements cannot be removed from the scheme.

27891979.png

When the connection between a scheme and an ontology is removed you can choose to keep the current status or remove all elements of the ontology from the custom scheme.

NLP Interchange Format (NIF)

The NLP Interchange Format is a set of tools that are now available for the PoolParty extractor in order to enable you to return annotations for texts in NIF-compliant RDF format.

In order to take advantage of it, you would use the two new PoolParty Extractor API annotation calls that have been implemented based on this format, namely extractor/api/annotate/nif , one providing the content type application/x-www-form-urlencoded for POST and GET requests. The second one can be used for POST requests in application/json format.

Multilingual Extraction

ThePoolParty Extractor APIhas been enhanced to allow for multilingual extraction. It is now possible to match labels in different languages in just one Extractor call. This not only is convenient because it reduces the number of calls for multilingual extraction, but is the solution for cases where concepts appear in different languages in the same piece of text.

RDF Export - Export Templates

PoolParty offers a heap of sophisticated settings and options to configure separately for anRDF data export.

To enable a quick and fast custom export at any time, we now implemented export templates you can save and reuse.

In the export menu a new item is available to open the Export Templates dialogue. It also lists all existing templates. It lets you choose a name for the template and define settings you can save for later.

Additionally, you can now exclude data from the export, when you select the option Concepts in the Thesaurus Data field. An additional drop down becomes available then and lets you choose from the list of all properties available in th project, which properties for concepts should be excluded.

To select an export template you created, start an RDF Project Export or RDF Subtree Export , and in the Template drop down, choose from the available templates. You can start exporting by clicking Export. You also can select additional options to extend the chosen template.

27891972.png

UnifedViews API Extensions

The UnifiedViews API has been extended. This implementation has a big advantage compared to previous procedures, since the execution of pipelines is simplified as parameters do not have to be uploaded and then read by the pipeline anymore. These new API calls let you execute pipelines while defining or replacing configuration parameters inside DPUs, additionally to the execution calls available so far.

Additionally, working with pipelines is made easier this way, since numerous copies of pipelines with different sets of parameters are now a matter of the past.

The following calls are now additionally available:

  • Execute a Pipeline with Parameters

  • Execute a Pipeline with String Parameters

  • Execute a Pipeline with String and Files Parameters

  • Execute a Pipeline with RDF Configuration Parameters

  • Execute a Pipeline with All Parameters and Dynamic Configuration

This way you can define individual string configuration parameters to replace existing ones in DPUs inside a pipeline, in part or all of them. Alternatively you can define one JSON file instead of individual strings to replace whole or part of the existing configuration. You can now also use an RDF file in Turtle format as replacement, in case the respective DPU supports dynamic configuration.

Improvements

Search Index Upgrade

Elasticsearch Version Upgrade: as of PoolParty 7.1, you have to update Elasticsearch to version 6.5.4. in order to work with PoolParty's search index functionalities, such as GraphSearch. Take care to upgrade your indices accordingly when updating PoolParty.

CSV Export and Import Function Implemented

You can now use the export menu on all levels of your PoolParty thesaurus to export to Excel as well as CSV. The corresponding menu entry is called Tabular Project Export and Tabular Subtree Export respectively.

In the Tabular Subtree Export dialogue shown below (1), similar to the Tabular Project Export dialogue, you find the additional option Format, in a drop down, where you can choose between Excel or CSV as target export format (2).

In all other respects, export options are similar to the Excel export available already.

27890810.png

Similarly you can import CSV or Excel, on project level for example, using the Tabular Import dialogue, as shown below. Other import levels work similarly, and details can be found on the respective documentation pages in the respective chapter: PoolParty RDF Import & Export

27891158.png

Drupal Modules Updated for Latest Drupal Versions

The following Drupal modules we offer for you to be able to seamlessly integrate PoolParty functionalities with Drupal. They have been updated to be compatible with the listed Drupal versions:

Subtree Export Update

The subtree export in PoolParty has been updated: you find the functions in two menu item entries and options have been added that were available in project export only, before, namely:

  • Template

  • Action

  • Exclude Concept Data

The additional functions associated with the new options are similar to the ones for project export, described above. That is, you can choose from custom templates, decide if you would rather download or store to a remote triple store configured in the Semantic Middleware Configurator .

Extended SKOS-Collections Functionality

In PoolParty you can create collections from inside your thesaurus, in order to group concepts independent from the hierarchical structure.

The interface has been overhauled and improved, yet well-known functions are still easily found in place, such as the expand arrows on the left of each concept entry (1).

As of version 7.1 an export function has been added here as well, accessible via the collection's context menu (2). Similarly to the RDF Subtree Export described above, in the Collection Export dialogue you can select custom export templates or use individual settings, such as the exclusion of data during export (3).

In ordered collections now use RDF lists as defined in the SKOS recommendation. Use theArrowicon at the end of each entry to re-order concepts in the collection.

In the subsequent Move to Index dialogue, enter a new number for the position of that concept, and clickMove(4).

PoolParty GraphSearch - Search Space Access Limitation Implemented

Search Spaces in GraphSearch let you define quickly how to display data in the GraphSearch interface once you have created your thesaurus and custom scheme or schemes.

Accessing the Admin Dashboard (1) in your GraphSearch interface that here shows an already configured Search Space, you can create a Search Space using the dialogue.

Search Spaces can now be have the status PUBLIC or PRIVATE.

By using theSearch Space Access Typedrop down (2) you can define which of the two options should be applied. That way a Search Space is either accessible with our without credentials.

API Changes and Improvements

PoolParty's APIs have seen a number of changes and improvements. The whole documentation of the APIs find here:

Thesaurus and Ontology Management API

Changes
  • /corpusmanagement/{project}/createTfidfIndex: error handling for corpusId changed: PoolParty will check, if the IDs provided are actually part of the project.

  • /PoolParty/api/projects/{project}/export: the content-type changed to application/json. The previous parameters are now part of the request body as attributes. The previous parameter properties as attribute is now optional. The parameter exportModules has changed to the attribute modules in the body.

  • /PoolParty/api/projects/{project}/remoteExport: the content-type changed to application/json. The previous parameters are now part of the request body as attributes. The previous parameters properties and graph as attributes are now optional (graph uses thesaurus graph by default). The parameter exportModules has changed to the attribute modules in the body.

  • /PoolParty/api/projects/{project}/store: the content-type changed to application/json. The previous parameters are now part of the request body as attributes. The previous parameter properties as attribute is now optional. The parameter exportModules has changed to the attribute modules in the body.

New
  • /PoolParty/api/projects/{project}/subtree/export: export subtrees of your project thesaurus as text/plain.

  • /PoolParty/api/projects/{project}/subtree/remoteExport: export subtree-RDF data to a remote server, based on the configuration int the Semantic Middleware Configurator .

  • /PoolParty/api/projects/{project}/subtree/store: export subtree-RDF data to the PoolParty server.

  • /PoolParty/api/projects/{project}/exportFromTemplate: export a PoolParty project based on templates you configured. Refer also to the new templates export function described above.

  • [POST]/[GET] /PoolParty/api/projects/{project}/template: use POST to create, and GET to retrieve information about a project export template.

  • [GET] /PoolParty/api/projects/{project}/templates: retrieve all existing project templates of a PoolParty project.

  • [DELETE] /PoolParty/api/projects/{project}/template: delete a project export template.

  • /PoolParty/api/projects/{project}/collection/export: export a PoolParty collection from a project as RDF data as a text/plain return.

  • /PoolParty/api/projects/{project}/collection/remoteExport: export a PoolParty collection from a project to a remote server, based on the configuration int the Semantic Middleware Configurator .

  • /PoolParty/api/projects/{project}/collection/store: export a PoolParty collection as RDF data to the PoolParty server.

GraphSearch API

  • [POST] /api/context/default: the parameter searchSpaceId is now mandatory.

  • The following calls now have additional error messaging included:

    Error messages

    The return to this request will display these errors in the following specific cases:

    • If the Search Space ID is wrong, you will receive a 404 'Bad Request' message, with specific information in a JSON message as to the value that was not found.

    • If the credentials were mistyped or wrong: a 403 'Forbidden', and a JSON message 'Access denied' will be returned.

    • If you didn't specify a Search Space ID: the return will be made for the default Search Space.

      • If the default Search Space is private and you are not authenticated, you will also receive a 403 'Forbidden' in a JSON message.

    /api/agents/create

    /api/agents/delete

    /api/agents/runAgent

    /api/agents/runAgents

    /api/agents/update

    /api/context/default

    /api/context/refresh

    /api/content/delete/id

    /api/content/delete/source

    /api/content/update/type

    /api/chart

    /api/trend/update

    GET /api/agents

    GET /api/agents/config

    GET /api/agents/status

    GET /api/chart/list

    GET /api/content/synchronize

    GET /api/content/update/documents

    GET /api/search

    GET /api/search/modes

    GET /api/context

    GET /api/context/fields

    GET /api/search/image

    GET /api/trend/histories

    GET /api/trend/history

    GET /api/similar

    GET /api/similar/similarities

    GET /api/suggest

    GET /api/suggest/all

    GET /api/suggest/multi

    GET /api/recommend/recommendations

  • GET /api/context/all: based on the new authentication option, returns the information on all or just the public search spaces of a GraphSearch instance.

  • GET & POST /api/recommend: based on the new authentication option, changes or returns will return an error, if the requested search space is private. If no plugin name is specified, the default search space is requested.

Extractor API

  • /extractor/api: every endpoint changed to explicitly either return the content-types application/json or text/plain.

  • [GET] / [POST] /extractor/api/annotate:

    Request Changes:

    New parameter: extraConceptLaguages, type: List

    New parameter for file based requests: charset, type Charset

    Response Changes:

    New parameter triple for extraConceptLanguages of string list type.

    Example:

    ppx:extraConceptLanguages "[es, en]";

    New prefLabel and altLabel triples for concepts with different languages.

    Example:

    skos:altLabel "DEM"@en, "DEM"@es;
    skos:prefLabel "Development Effectiveness Matrix"@en, "Matriz de Efectividad en el Desarrollo"@es

    When 'showing matching details' is enabled then a new language triple is added to signal in which language this match happened.

    Example:

    ppx:language "en";
  • extractor/api/extract:

    Request Changes:

    New parameter: extraConceptLaguages, type: List

    New parameter for file based requests: charset, type Charset

    Response Changes:

    Concept information regarding language, prefLabel, and AltLabel changes to

    "languages": [
    "en",
    "es"
    ],
    "prefLabels": {
    "en": "Development Effectiveness Matrix",
    "es": "Matriz de Efectividad en el Desarrollo"
    },
    "altLabels": {
    "en": [
    "DEM"
    ],
    "es": [
    "DEM"
    ]
    },

    Enabling matching labels will show a new language entry for each matched entry.

    Example:

    "language": "en",
  • /extractor/categorization:

    New parameter for file based requests: charset, type Charset

    Works again with the request parameters fileandurl.

  • /extractor/api/annotate/nif: allows to do annotations according to the NLP Interchange Format (NIF), as also described above, details find here: Web Service Method: Annotate from Text in NIF Format

  • extract/api/suggest: The extractor API Concept Suggest call (GET or PUT) extract/api/suggest provides a slightly different response as of this version:

    Previous Response

    "prefLabel": "Brazil"

    Response Formatting as of this Version

    "prefLabels": {
    "en": "Brazil"
    }

    Similarly, the properties altLabel and hiddenLabel, are now returned as JSON objects that depict the language, where the values are included as an array of strings:

    Example for the PrefLable and AltLabel Properties Changes

    "altLabels" :
    { "es" : [ "B.I.D.", "I.D.B.", "IADB", "IDB", "I.A.D.B", "BID" ] }

UnifiedViews API

The following calls were added, as also described in little more detail above:

  • POST http://{server}/master/api/1/pipelines/{pipelineId}/execution: execute a pipeline with parameters

  • POST http://{server}/master/api/1/pipelines/{pipelineId}/execution: execute pipeline with string parameters

  • POST http://{server}/master/api/1/pipelines/{pipelineId}/execution: execute pipeline with string and files parameters

  • POST http://{server}/master/api/1/pipelines/{pipelineId}/executions: demo: execute pipeline with all parameters

  • POST http://{server}/master/api/1/pipelines/{pipelineId}/execution: execute pipeline with all parameters and dynamic configuration

Bug Fixes

  • An issue with the display and download of the MultiThes format has been fixed. The display in the Linked Data Frontend and download from there work as expected again.

  • Due to default .zip size unpack limitations on some operating systems the PoolParty project .zip file could not be unpacked, after export, with the default tool. This has been changed for the more common file sizes.

  • The display of Top Concept of a Concept Scheme labels was at issue, Delete icons being visible, but labels missing; this has been fixed.

  • Due to a faulty code entry, lower case language tags were considered in the autocomplete search results only. This has been fixed and all entries are available now.

  • On importing and creating a project from a .ppar and changing the default URI pattern,this changed URI pattern would not be considered for new resources, which were created later. The issue has been fixed.

  • An issue with displaying and extracting concept and concept scheme labels of varying language tags, that is: mixed lower and upper case, has been fixed as of this release.

  • The statistics regarding concepts' preferred labels were incorrect for upper case language tags. Fixed as of this release.

  • The API import modules were missing 6 modules, compared to those that are available for export. Fixed as of this release.

  • On creating a new concept a warning message for manually entered, existing IDs was missing. This message is now displayed correctly.

  • The Corpus Search autocomplete did not display preferred labels correctly when alternative labels would match the search string. Fixed as of this release.