Configuring the Recommender using the Workbench
This section will give you some initial insights regarding the recommender configuration using the Workbench including
the SEARCH SPACE, EXTRACTION, EXPANSION and RECOMMENDATION tabs
all configurable parameters for these calls on each of those tabs
The PoolParty Workbench supports subject matter experts in finding the optimal combination of parameters for their semantic recommender system. The expert can set multiple configurations reflecting their specific recommendation requirements.
Note
Since recommendation is a three-step process you will need not only to specify the search space but also the extraction, expansion and recommendation call. The Workbench is the ideal tool for writing and saving multiple configurations for your recommendation service. The Workbench path may look like this [your.poolparty.server]/RecommenderWorkbench/.
The Recommender interacts with the Workbench, the knowledge model and based on the underlying configuration with the recommender DB. This section will give you a brief introduction to the PoolParty Workbench where being a subject-matter expert you configure your PoolParty semantic recommender system.
Tip
The PoolParty Workbench is only accessible to authenticated and authorized users and relies on PoolParty for user management, meaning that you need an appropriate PoolParty user account grating you access to the Workbench.
Note
Keep in mind that the recommendation calls explicitly require Elasticsearch index to be in place. The other two calls (extract and expand) can but don't have to use Elasticsearch index)You need to set the correct indexing in the Semantic Middleware Configurator (also refer to Configuring the Recommender - Preliminary Steps). If you need to set up an Elasticsearch index from scratch refer to How to Setup a Connection to an Elasticsearch Index.
After launching the PoolParty Workbench you have the following options:
(1) - here you see the selected Search Space. Clicking on the pen symbol opens a new dialog where you can see the details of the search space along with other configurable parameters and where you can select a different search space.
(2) - use this field to enter your input text for which you want to test the existing recommender configuration or want to write a new one.
(3) - clicking on the magnifying glass symbol opens a new page where you will see potential recommendations for the specified query (i.e. your input) and the selected recommender configuration. Even if you have not set up your configuration yet, there is always the default configuration with all parameters set to default values. It can also be that your settings for the search space or expansion call need to be adjusted.
(4) - a click on the gear symbol opens the Settings window where you can write, modify, save, and load recommender configurations.
Tip
The gear symbol is accessible on all Workbench main pages.
Tip
You can of course also import and export an existing recommender configuration.
In this guide we want to verify an existing recommender configuration. To do so we click the gear symbol (4) and open the following dialog window:
The top row (2) lists the available tabs giving us access to the respective parameters of the recommender configuration. The first tab SEARCH SPACE allows us to select the desired search space we want to use for recommendations (1).
We could also load or save configuration (3) using the two buttons on the top right. Finally, in the bottom right corner we have the Apply and Cancel buttons. In our case we select the. Recommender onboarding Search Space (1).
Note
Click on LOAD CONFIGURATION (3) opens a dialog listing available configurations, whereas SAVE CONFIGURATION (3) prompt us to enter the name of the configuration and optionally a description under which we want to save the currently selected and possibly modified configuration. However keep in mind that loading a configuration file will override any settings you might have made, and saving the configuration will only save any changes made up to this point (i.e. for the search space).
Note
Do not yet click the Apply or Cancel button (4) as we want to continue with the other parameters of the recommender configuration on the remaining three tabs (2). Clicking one of the two buttons will always take you back to the initial Workbench screen - with Apply you apply any changes made, with Cancel you discard such changes.
After making sure that we have selected the desired SEARCH SPACE, we can move to the second tab EXTRACTION.
On the EXTRACTION tab you see both mandatory and optional parameters you can specify for the extraction call. The Extraction language (1) and Maximum number of concepts (3) are mandatory; if you want to use the scoring functionality you will also need to specify the Minimum concept score (2). By setting the minimum score you can exclude concepts which score is too low from being annotated. The other optional parameters have the following functionalities:
(4) - Concept scheme filter - If you want to narrow down the accuracy of extraction you can select one or more concept schemes to be used as a filter. Only concepts contained in those selected concept schemes will be then extracted;
(5) - the two checkboxes Filter nested concepts and Use disambiguation; ticking the first checkbox excludes the nested concepts from extraction, whereas ticking Use disambiguation will make the system use the thesaurus and local context to resolve any potential ambiguities. More details on how to handle disambiguities in PoolParty can be found in Thesaurus Based Disambiguation of Annotated Concepts. The Use corpus (6) parameter allows you to select a corpus for the extraction of shadow concepts. Furthermore, the corpus is also used for the corpusScoring parameter.
Tip
Both shadow concepts and corpus scoring can be enabled or disabled separately for both the Extractor and Recommender.
You will find more information on shadow concepts in the section Shadow Concepts in PoolParty and if you want to see further details related to corpus, refer to the section Corpus Management - Overview respectively to the section Check Scoring Results for Concepts and Free Terms for more information on corpus scoring.
Note
Keep in mind that loading a configuration will override any settings you might have made, and saving the configuration will only save any changes made up to this point (in our example on the tabs: Search Space and Extraction).
Clicking the Apply or Cancel button will always take you back to the initial Workbench screen - with Apply you apply any changes made, with Cancel you discard such changes.
After having set all the desired parameters for the extraction we can move to the third tab, the EXPANSION.
The EXPANSION tab is comprised of two sub-tabs - the Expansion Query Builder tab and the Switch to SPARQL tab.
After selecting the EXPANSION tab (2) you can write a new SPARQL query using one of the two sub-tabs. Furthermore, you can use the Save query respectively Load query buttons (6) to save new and load existing queries.
You can also save a new or load an existing configuration using the corresponding buttons (5) here.
As mentioned before the EXPANSION tab has two sub-tabs - the Expansion Query Builder (3) and the Switch to SPARQL (4) tabs. Users less familiar with SPARQL can use the Expansion Query Builder tab to enter all data required for formulating a correct SPARQL query intended for performing an expansion call, whereas the Switch to SPARQL tab is intended for users experienced in formulating SPARQL queries using a command line environment.
Note
Expansion calls are SPARQL queries.
The Expansion Query Builder tab is intended for users who are less familiar with SPARQL and contains all fields required for building a proper SPARQL expansion query.
The Expansion Query Builder user interface has all fields required for writing a well-formed SPARQL query even if you are less familiar with SPARQL syntax. This tab has the following sections Locale, Prefixes, Rules, and Test expansion query.
The Locale (1) section has a mandatory pulldown menu where you specify the language of the concepts and their labels . By specifying the language for your concepts you make sure that only concepts with preferred labels in this language will be fetched.
Underneath you have the Prefixes (2) section. where you can select one of the standard prefixes (3) or add your own prefix (4) using the Name and URI fields and clicking on the ADD button. Both standard and custom prefixes can be used in a SPARQL query, whereby a prefix is always a combination of a Name and URI.
The next section is intended for entering the expansion rules (Rules, (5). By default this window shows Rule #1, you can however enter as many rules as required. To add a new rule click on Add a new rule + on the right (6), underneath you see a bin icon (6) - click on this bin to delete an unwanted rule. In this context a rule is an ordered list of relations. You can specify as many relations as required. A relation is a combination of a URI and Type (7), however leave the field Type empty if a type check is not needed. This URI field contains the URI of the specific relation for which we want to obtain expanded concepts. This field supports prefixes. To add a new relation click on the + Add a new relation symbol (8). Each rule has a certain score. To set the score first select the Score Type (9)) which can be either a fixed score or a multiplier. Selecting a fixed score in the Score Type field requires input of any numeric within the range from 0 to 1 (e.g. 0.25), whereas selecting a multiplier as the score type supports input of any numeric (e.g. 0.8).
At the very bottom of this window, just below the Text expansion query section there are two buttons - Apply and Cancel; use the Apply button to add your query to the configuration or Cancel to discard any changes made. Keep in mind that you have to save your query if you want to reuse it or edit it later.
Just like on the Switch to SPARQL sub-tab you can text your query on this sub-tab. Enter at least one concept in the Concepts field in the Test expansion query section at the bottom of this window.
Tip
Remember to save your new query before testing it.
To test your saved query go to the Concept field and enter the first two letters of the concept you want to use for the test. A list with all available concepts matching your input appears where you can select the desired concept for testing. If the query is correctly configured you will receive a list of matching expanded concepts.
Note
To be able to test your expansion query you have to specify at least one concept. You can however also specify more concepts for testing an expansion query.
The screenshots above illustrate how we formed our SPARQL query using the Expansion Query Builder sub-tab.
We entered respectively selected the following parameters: (1) en as the Locale; skos (2) as our Prefix (it is one of the standard prefixes). Then we defined the first Rule (2) as skos:narrower, selected Fixed Score as the Score Type (4) and entered 0.8 as the Score (5). Next step was to test this query, to do so we selected two Concepts (6) - Steel and Tax credit, clicking on the TEST button (7) returned in our example three Results (8) listed along with their Preferred label, URI and Score.
Now we are ready to move on to the last tab, the RECOMMENDATION.
Note
Expansion calls are SPARQL queries.
When you click the Switch to SPARQL sub-tab you see an empty Expansion query field (1) where you can enter your expansion call. You can also save (2) a query you have just specified or load a previously set up one (3).
Note
When you save an expansion query it will only be visible to you and not to any other Workbench users.
Below the query input field you can specify the maximum number of concepts (4) to be returned in the course of the semantic expansion. You can also test your expansion query by specifying the concepts (5) you want to search for in this field followed by a click on the TEST button (6). This field supports the autocomplete function and will show you a selection list after you have entered at least 2 letters. Those concepts are based on the search space you have previously defined on the first tab. If you try testing without a query or without any concepts a corresponding message will appear in the bottom left corner of this window. This tab also has the load and save configuration buttons (8) and the Apply and Cancel buttons (7).
Note
Keep in mind that loading a configuration will override any settings you might have made, and saving the configuration will only save any changes made up to this point (in our example on the tabs: Search Space, Extraction and Expansion).
Clicking the Apply or Cancel button will always take you back to the initial Workbench screen - with Apply you apply any changes made, with Cancel you discard such changes.
In our case we will load a query with a click on the Load query (3) button.
A new window showing the available expansion queries opens. In our example we select the only available expansion query (1) with a single click on the query and then confirm with LOAD (2).
Note
If you don't select any query and click LOAD, this window will close but no expansion query will be loaded.
The selected expansion query has now been loaded into the Expansion Query field (1):
Underneath you can specify the maximum number of expanded concepts (2) and also test the query. To test your loaded query go to the Concept field (3) and enter the first two letters of the concept you want to use for the test. A list with all available concepts matching your input appears where you can select the desired concept for testing. In our example we use Receptors, Colony-Stimulating Factor as the concept used for testing. We also keep the default setting of maximum 20 concepts to be returned as expanded concepts.
Note
To be able to test your expansion query you have to specify at least one concept. You can however also specify more concepts for testing an expansion query.
If the query is correctly configured you will receive a list of matching expanded concepts which in our example looks like this:
The bottom section of the EXPANSION tab now shows the specified Concept URIs (1) (in our example: Receptors, Colony-Stimulating Factor ), specified number of Results (2) (in our example: 20) and the individual results listed in a tabular form showing for each expanded concept its Preferred label (3), URI (4) and Score (5).
Tip
Always remember to save your query. Swapping between the two sub-tabs will discard any inputs you might have made.
Now we are ready to move on to the last tab, the RECOMMENDATION.
On this tab you can specify the maximum number of recommendations (1) you want to receive as well whether you want to use the PoolParty concept-based search paired with the Elasticsearch full-text search function (2) or not.
Tip
It is recommended to tick the checkbox Use PoolParty... (2) to even further enhance the quality of recommendations.
Note
The PoolParty Semantic Recommender synergistically combines the advantages of concept-based search with the full-text search functionality of Elasticsearch.
Elasticsearch index is required for the last call, the recommendation.
Now we have finished defining all the relevant parameters and can confirm our configuration with a click on the Apply button (4). Clicking on the Cancel button (4) will discard all changes we made so far.
We also have the SAVE CONFIGURATION button at the top right (3) if we want to save the current configuration. However clicking on the LOAD CONFIGURATION button (3) and selecting a stored configuration to be loaded will override all our current settings with any configuration we might have decided to load.
If you decide to edit an existing configuration you will have two options available after clicking SAVE CONFIGURATION - the option Save which means that the configuration will be stored under its original name and the option Save as... - meaning that you will create a new configuration with a new name and description based on a previously stored one with some amendments, if applicable.
In our next step we will test the configuration.