HTTP API Request

Description

HTTP API Request (uv-e-httpRequest):

This DPU allows executing HTTP requests (GET, POST methods) to web services and passes the response in form of file data unit.

This DPU targets to enable consuming web services, both. REST and SOAP.

For POST HTTP requests, there are 4 possible modes (type of sent data)

  • multipart (form data) body
  • raw data (content type can be specified: XML, JSON, ...)
  • raw data with bodies from input file(s) - for each input file a separate HTTP request (raw data) is executed
  • multipart (form data) with bodies from an input RDF configuration - for each input set of form params a separate HTTP request is executed

If sent data is multipart or raw, this DPU offers possibility to preview the HTTP response in design time.

The DPU also supports HTTPS requests.

Configuration Parameters


ParameterDescriptionExample
HTTP methodSupported HTTP request methods: GET, POST, PUT, DELETE. Based on the method additional configuration options are shown.POST
URL addressURL address of the target web service, where the HTTP or HTTPS request will be sent.

http://localhost/PoolParty

https://vocabulary.semantic-web.at/PoolParty

Target file nameName of created file where the content of the HTTP response is stored.response.json
Target files suffix(POST / file mode) Suffix of created files containing the content of HTTP responses.001_suffix, 002_suffix
Basic authenticationSets BASIC authentication (user name, password) for HTTP requesttrue
User name(if authentication is on) User name for basic authenticationadmin
Password(if authentication is on) Password for basic authentication<password>
Data type(only for POST HTTP method) Type of sent data in HTTP request: Raw body (text), Form-data body (multipart), Raw bodies from input files, Form-data bodies from input RDF configurationForm-data bodies from input RDF configuration
Content-type(only for POST HTTP method) Type of sent raw data, set as HTTP header "Content-Type" (e.g. XML, JSON, SOAP, ...)text/html
Request body text encoding(only for POST HTTP method) Encoding of HTTP request body textThis%20is%20some%20sample%20encoded%20text
Request body(only for POST HTTP method - raw body) Text sent in HTTP request body"This is some sample text"
Form data(only for POST HTTP method - form-data body) Table of sent form data in the form of key - values


Inputs and Outputs

NameTypeDataUnitDescriptionRequired
requestOutputoutputFilesDataUnitFile(s) containing HTTP response(s)(tick)
requestFilesConfiginputFilesDataUnitFiles sent as content of raw HTTP POST request(error)
rdfConfiginputRDFDataUnitRDF configuration used to configure form-data bodies(error)


Notes

Advanced Configuration

It is also possible to dynamically configure the request body over the input config data unit using RDF data. This is available only for raw mode and you can configure only the request.


Configuration samples

# to dynamically configure request URL and request body (raw data mode)
<http://localhost/resource/config>
    <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://unifiedviews.eu/ontology/dpu/httpRequest/Config>;
    <http://unifiedviews.eu/ontology/dpu/httpRequest/requestBody> "..." ;
    <http://unifiedviews.eu/ontology/dpu/httpRequest/url> "http://semantic-web.com/service/x".


# two form-param bodies with the same set of three form params
<http://localhost/resource/config>
    <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://unifiedviews.eu/ontology/dpu/httpRequest/Config>;
    <http://unifiedviews.eu/ontology/dpu/httpRequest/formParamBody> <http://unifiedviews.eu/ontology/dpu/httpRequest/FormParamBody/1> ;
    <http://unifiedviews.eu/ontology/dpu/httpRequest/formParamBody> <http://unifiedviews.eu/ontology/dpu/httpRequest/FormParamBody/2> .

<http://unifiedviews.eu/ontology/dpu/httpRequest/FormParamBody/1>  a <http://unifiedviews.eu/ontology/dpu/httpRequest/FormParamBody> ;
    <http://unifiedviews.eu/ontology/dpu/httpRequest/formParam> <http://unifiedviews.eu/ontology/dpu/httpRequest/FormParam1> ;
    <http://unifiedviews.eu/ontology/dpu/httpRequest/formParam> <http://unifiedviews.eu/ontology/dpu/httpRequest/FormParam2> ;
    <http://unifiedviews.eu/ontology/dpu/httpRequest/formParam> <http://unifiedviews.eu/ontology/dpu/httpRequest/FormParam3> .

<http://unifiedviews.eu/ontology/dpu/httpRequest/FormParam1> a <http://unifiedviews.eu/ontology/dpu/httpRequest/FormParam> ;
    <http://unifiedviews.eu/ontology/dpu/httpRequest/param> "corpusId" ;
    <http://unifiedviews.eu/ontology/dpu/httpRequest/value>  "corpus:307b420d-43ad-4771-be41-308199da95b1" .

 <http://unifiedviews.eu/ontology/dpu/httpRequest/FormParam2> a <http://unifiedviews.eu/ontology/dpu/httpRequest/FormParam> ;
    <http://unifiedviews.eu/ontology/dpu/httpRequest/param> "text" ;
    <http://unifiedviews.eu/ontology/dpu/httpRequest/value>  "Test" .

 <http://unifiedviews.eu/ontology/dpu/httpRequest/FormParam3> a <http://unifiedviews.eu/ontology/dpu/httpRequest/FormParam> ;
    <http://unifiedviews.eu/ontology/dpu/httpRequest/param> "title" ;
    <http://unifiedviews.eu/ontology/dpu/httpRequest/value>  "Test title" .


 <http://unifiedviews.eu/ontology/dpu/httpRequest/FormParamBody/2>  a <http://unifiedviews.eu/ontology/dpu/httpRequest/FormParamBody> ;
    <http://unifiedviews.eu/ontology/dpu/httpRequest/formParam> <http://unifiedviews.eu/ontology/dpu/httpRequest/FormParam1> ;
    <http://unifiedviews.eu/ontology/dpu/httpRequest/formParam> <http://unifiedviews.eu/ontology/dpu/httpRequest/FormParam2> ;
    <http://unifiedviews.eu/ontology/dpu/httpRequest/formParam> <http://unifiedviews.eu/ontology/dpu/httpRequest/FormParam3> .

Useful Endpoints for Graph Databases and PoolParty

Many times this DPU is used to get RDF data from PoolParty or triple stores using a SPARQL CONSTRUCT query. The following table contains relevant endpoints for this:

Database

Path Variable

Service Path for SPARQL Query

Service Path for SPARQL Graph Store HTTP Protocol

Example

RDF4J$REPOSITORY: RDF4J repository name/$REPOSITORY/$REPOSITORY_ID/rdf-graphs/servicehttp://db-rdf4j-oews-poc.semantic-web.at:8080/rdf4j-server/repositories/test?query=...
Stardog$DATABASE: Stardog database name/$DATABASE/query/$DATABASEhttp://db-stardog-stardog-poc.semantic-web.at:5820/msmetrics/query?query=....
MarkLogicNone. "repository" is decided by port number/v1/graphs/sparql/v1/graphshttp://pp-sem-doc.demo.marklogic.com:8026/v1/graphs/sparql with Construct in Body (raw body, text (text/plain), UTF-8)
Allegrograph$REPOSITORY: RDF4J repository name/repositories/$REPOSITORYNot supportedhttps://db-allegrograph.poolparty.biz/repositories/tcg?query=Construct%20%7B%3Fs%20%3Fp%20%3Fo%7D%20WHERE%20%7B%3Fs%20%3Fp%20%3Fo%7D&queryLn=SPARQL
GraphDB$REPOSITORY: GraphDB repository name/$REPOSITORY/$REPOSITORY_ID/rdf-graphs/servicehttp://172.28.9.18:7200/repositories/requisition?query=....

Examples


GET: Create PoolParty project snapshot


The following image shows a fragment of a pipeline which first executes a SPARQL DELETE in a PoolParty project (SPARQL Endpoint Loader) and then creates a snapshot of the PoolParty project with a HTTP Request. The configuration for the snapshot API call can be seen in the image below. For more information on the used Web Service Method see here: https://help.poolparty.biz/x/ySiU





GET: Get Subtree Concepts from PoolParty


The following image shows another configuration example to make an API call to PoolParty.  For this example, the Web Service Method Request Subtree of Concept or Concept Scheme is used. For more information on this Web Service Method see here: https://help.poolparty.biz/x/AimU






POST: With Config Input Example


The following image shows a fragment of a pipeline which downloads a file, sends it to the PoolParty extractor to be annotated, from the response we will construct the configuration that will be sent as input the the HTTP Request API, and finally the result will be loaded into a SPARQL Endpoint.


The SPARQL Construct configuration can be seen below. This construct will be parsed into the HTTP API Request DPU as configuration parameters following a specific format. The construct creates a unique configuration for each subject, which is essentially the document URI. Attached to each config is the document text encoded to remove whitespaces.





POST: Simple GraphSearch Example


The following image shows a fragment of a pipeline which is used to populate a GraphSearch search space. What is not seen are files are downloaded from a folder on the server and are annotated using the PoolParty Concept Extractor. This fragment shows the start of the configuration based on the annotation results, the configuration is transferred to the PoolParty GraphSearch Content Indexing Request Constructor. Following this HTTP API Requests are sent to drop the current index, create the new content in GraphSearch Space, and finally a refresh of the newly created content's index.



POST: Refresh Search Index of GraphSearch Search Space


The following image is a configuration of the refresh search index for GraphSearch, this is usually attached to the pipeline where content is created (as seen above). 


For further information about this API please check https://help.poolparty.biz/x/FyqU