UnifiedViews API is available as a web application called master.war
. All API functions are reachable with prefix:
http://host:8080/master/api/1/
where host is the hostname
where UnifiedViews is installed, master
is the name of the webapp in the application server.
The api
keyword in URI is followed by API version (for future enhancements), for now, only version 1
will be developed.
All functions in this document are to be prefixed by this prefix. We write them in shorthand form, without this prefix.
Authentication
Please keep in mind that the web service is protected against unauthorized access by using basic authentication.
Username and password is stored in UV core (frontend) properties file:
# credentials for Master REST API master.api.user=master master.api.password=commander
Sample curl request to get all pipelines using default username and password is below:
curl http://master:commander@localhost:8080/master/api/1/pipelines
NOTE: Certain calls may require to also specify (either as GET or POST param) userExternalId
, which is by default equal to the user ID. Some older instances may have user John Admin (as the default user account), in that case userExternalId is equal to http://www.johnadmin.cz
List of methods
1. Pipelines
1.1 Get all Pipelines
Returns list of all pipelines for the given user (user is owner of those pipelines).
Request
GET <prefix>/pipelines Accept: application/json
Sample request:
curl http://master:commander@localhost:8080/master/api/1/pipelines?userExternalId=admin
Response
HTTP/1.1 200 OK Content-Type: application/json [ { "id": 98, "name": "[AutoTest] RDF from file - Merge RDF - Validate RDF - RDF to CKAN", "description": "", "userExternalId": "4AFA411A-476B-4B49-8F48-3274B838F460", "userActorExternalId": "8EEBA23B-6BFE-4F76-A3B5-D9B627823137" }, { "id": 24, "name": "Periodické sťahovanie meteo dát", "description": "", "userExternalId": "4AFA411A-476B-4B49-8F48-3274B838F460", "userActorExternalId": null }, { "id": 20, "name": "Priebežná aktualizácia dátového zdroja (MV)", "description": "", "userExternalId": "4AFA411A-476B-4B49-8F48-3274B838F460", "userActorExternalId": null } ]
Response, error
HTTP/1.1 500 Internal Server Error Content-Type: application/json { "errorMessage": "user readable message", "technicalMessage": "some technical message" }
1.2 Get one Pipeline
Request
GET <prefix>/pipelines/<id> Accept: application/json
Sample request:
http://admin:commander@localhost:8080/master/api/1/pipelines/33?userExternalId=admin
Response
HTTP/1.1 200 OK Content-Type: application/json { id: 1, name: "The pipeline", description: "The description" }
Response, invalid id
HTTP/1.1 404 Not Found Content-Type: application/json { "errorMessage": "Proces s ID='100' sa nepodarilo nájsť!", "technicalMessage": "Pipeline with id=100 doesn't exist!" }
Response, error
HTTP/1.1 500 Internal Server Error Content-Type: application/json { "errorMessage": "user readable message", "technicalMessage": "some technical message" }
1.3 Get all visible Pipelines
Returns list of all pipelines visible for the user specified in parameter userExternalId. This means pipelines of that user (he is owner) and public pipelines of others
Request
GET <prefix>/pipelines/visible?userExternalId=admin Accept: application/json
Response
HTTP/1.1 200 OK Content-Type: application/json [ { "id": 1, "name": "The pipeline", "description": "The description", "userActorExternalId": null, "userExternalId": "organization_ext_id" }, { "id": 2, "name": "The pipeline 2", "description": "The description 2", "userActorExternalId": null, "userExternalId": "another_organization_ext_id" } ]
Returns list of all pipelines visible for organization specified in parameter userExternalId. This means private and public pipelines for specified organization and public pipelines of other organizations.
Response, no pipelines in UV visible by organization
HTTP/1.1 200 OK Content-Type: application/json []
Response, error
HTTP/1.1 500 Internal Server Error Content-Type: application/json { "errorMessage": "user readable message", "technicalMessage": "some technical message" }
scheduledexecutions
1.4 Create Pipeline
Request
POST <prefix>/pipelines Accept: application/json { name: "The pipeline", description: "The description" }
Response
HTTP/1.1 200 OK Content-Type: application/json { id: 1, name: "The pipeline", description: "The description" }
Response, error
HTTP/1.1 500 Internal Server Error Content-Type: application/json { "errorMessage": "user readable message", "technicalMessage": "some technical message" }
1.5 Copy Pipeline
Request
POST <prefix>/pipelines/<pipeline_id>/clones/ Accept: application/json { name: "The pipeline", description: "The description" }
Response
HTTP/1.1 200 OK Content-Type: application/json { id: 1, name: "The pipeline", description: "The description" }
Response, invalid id
HTTP/1.1 404 Not Found Content-Type: application/json { "errorMessage": "Proces s ID='xyz' sa nepodarilo nájsť!", "technicalMessage": "Pipeline with id=xyz doesn't exist!" }
Response, error
HTTP/1.1 500 Internal Server Error Content-Type: application/json { "errorMessage": "user readable message", "technicalMessage": "some technical message" }
1.6 Import Pipeline
Why
Automatic testing and deployment.
Request
POST <prefix>/pipelines/import Content-Disposition:form-data; name="file"; filename="zipperPipeline.zip" Content-Disposition:form-data; name="importUserData" true Content-Disposition: form-data; name="importSchedule" true Content-Disposition: form-data; name="userExternalId" admin
- file: ZIP file with pipeline to import
- userExternalId: admin (user which is marked as pipeline author)
importUserData: optional, String, values: true / false, indicates if user data should be imported
importSchedule: optional, String, values: true / false, indicates if schedule information should be imported
Response
HTTP/1.1 200 OK Content-Type: application/json { "id": 60, "name": "The pipeline", "description": "The desciption", "userExternalId": "admin", "userActorExternalId": null }
2. Pipeline.Schedules
2.1 Get all Pipeline.Schedules events
Get all scheduled events for the given pipeline id
Request
GET <prefix>/pipelines/<pipeline_id>/schedules/ Accept: application/json
Response
HTTP/1.1 200 OK Content-Type: application/json [ { id: 1, description: "Runs it daily", enabled: true, scheduleType: "PERIODICALLY", firstExecution: "2014-08-08T15:15:15.555Z", lastExecution: "2014-08-08T15:15:15.555Z", nextExecution: "2014-08-08T15:15:15.555Z", afterPipelines: null, justOnce: false, period: 1, periodUnit: "DAY" }, { id: 2, description: "Runs the pipeline weekly", enabled: true, scheduleType: "PERIODICALLY", firstExecution: "2014-08-08T15:15:15.555Z", lastExecution: "2014-08-08T15:15:15.555Z", nextExecution: "2014-08-08T15:15:15.555Z", afterPipelines: null, justOnce: false, period: 1, periodUnit: "WEEK" }, { id: 3, description: "Runs it just once", enabled: false, scheduleType: "PERIODICALLY", firstExecution: "2014-08-08T15:15:15.555Z", lastExecution: "2014-08-08T15:15:15.555Z", nextExecution: null, afterPipelines: null, justOnce: true, period: null, periodUnit: null, }, { id: 4, description: "Runs after other pipelines", enabled: true, scheduleType: "AFTER_PIPELINE", firstExecution: null, lastExecution: "2014-08-08T15:15:15.555Z", nextExecution: null, afterPipelines: [15, 54], justOnce: null, period: null, periodUnit: null, }, { id: 5, description: "Runs every 6 minutes", enabled: true, scheduleType: "PERIODICALLY", firstExecution: "2014-08-08T15:15:15.555Z", lastExecution: "2014-08-08T15:15:15.555Z", nextExecution: "2014-08-08T15:15:15.555Z", afterPipelines: null, justOnce: false, period: 6, periodUnit: "MINUTE", }, { id: 6, description: "Runs it just once, had not run till now", enabled: false, scheduleType: "PERIODICALLY", firstExecution: "2014-08-08T15:15:15.555Z", lastExecution: null, nextExecution: "2014-08-08T15:15:15.555Z", afterPipelines: null, justOnce: true, period: null, periodUnit: null, }, ]
Returns list of all pipeline schedule entries.
firstExecution is the time when the pipeline will run first time based on this schedule
lastExecution is the time when the schedule was triggered last time, the pipeline was added to queue of waiting/running pipelines, it is not the time when the pipeline exec started, nor finished.
For Schedule there are consistency criteria:
- Schedule is PERIODICALLY type
- justOnce = true
- enabled = true (the pipeline will run in future)
- firstExecution = time when the pipeline will run in future
- lastExecution = null (had not run yet)
- nextExecution = null (had run already), or date - same as firstExecution (hadn't run)
- enabled = false (the schedule was either not enabled or the pipeline has been executed already)
- firtstExecution = time, when the pipeline was intended to start.
- nextExecution = null
- lastExecution = null (had not run yet), user had to disable schedule before it could start for the first time
- lastExecution = some date (had run), backend disabled the schedule to prevent running it again (to satisfy just once requirement)
- afterPipelines = null
- period = null
- periodUnit = null
- enabled = true (the pipeline will run in future)
- justOnce = false
- enabled = true | false (if schedule is not enabled, it is ignored by UV, pipeline execution is not triggered by this schedule)
- firstExecution = time, which user specified when the pipeline execution should happen by this schedule for the first time
- lastExecution = time, when the pipeline execution was triggered by this schedule (not by user, or other schedule)
- nextExecution = time, next execution should happen
- afterPipelines = undefined (null, empty array, ignore it)
- period = int - each period times
- periodUnit = MINUTE | HOUR | DAY | WEEK | MONTH | YEAR
- justOnce = true
- Schedule is AFTER_PIPELINE type
- justOnce = null
- enabled = true | false (if schedule is not enabled, it is ignored by UV, pipeline execution is not triggered by this schedule)
- firstExecution = null
- lastExecution = null (pipeline had not run yet) or date of last execution which was triggered by this schedule
- nextExecution = null
- afterPipelines = array of other pipelines IDs , pipeline is run immediately after any of the pipelines in the array
- period = null
- periodUnit = null
Response, no schedule for given pipeline
HTTP/1.1 200 OK Content-Type: application/json []
Response, error
HTTP/1.1 500 Internal Server Error Content-Type: application/json { "errorMessage": "user readable message", "technicalMessage": "some technical message" }
2.2 Get one Pipeline.Schedule event
Get one scheduled events (with the given id) for the given pipeline id
Request
GET <prefix>/pipelines/<pipeline_id>/schedules/<id> Accept: application/json
Response
HTTP/1.1 200 OK Content-Type: application/json { id: 6, description: "Runs it just once, had not run till now", enabled: false, scheduleType: "PERIODICALLY", firstExecution: "2014-08-08T15:15:15.555Z", lastExecution: null, nextExecution: null, afterPipelines: null, justOnce: true, period: null, periodUnit: null, }
Response, no entity for given id
HTTP/1.1 404 Not Found Content-Type: application/json { "errorMessage": "Proces s ID='xyz' sa nepodarilo nájsť!", "technicalMessage": "Pipeline with id=xyz doesn't exist!" }
Response, error
HTTP/1.1 500 Internal Server Error Content-Type: application/json { "errorMessage": "user readable message", "technicalMessage": "some technical message" }
2.3 Edit Pipeline.Schedule
Request
PUT <prefix>/pipelines/<pipeline_id>/schedules/<schedule_id> Accept: application/json { id: 4, description: "Runs after other pipelines", justOnce: false, enabled: false, firstExecution: "2014-08-08T15:15:15.555Z", lastExecution: null, afterPipelines: [15, 54], period: null, periodUnit: null, }
Response
HTTP/1.1 200 OK Content-Type: application/json { id: 4, description: "Runs after other pipelines", justOnce: false, enabled: false, firstExecution: "2014-08-08T15:15:15.555Z", lastExecution: null, nextExecution: null, afterPipelines: [15, 54], period: null, periodUnit: null, }
Returns the entity in its new version.
Response, invalid entity ID (someone deleted it meanwhile)
HTTP/1.1 404 Not Found Content-Type: application/json { "errorMessage": "Proces s ID='xyz' sa nepodarilo nájsť!", "technicalMessage": "Pipeline with id=xyz doesn't exist!" }
Response, error
HTTP/1.1 500 Internal Server Error Content-Type: application/json { "errorMessage": "user readable message", "technicalMessage": "some technical message" }
2.4 Create Pipeline.Schedule
Request
POST <prefix>/pipelines/<pipeline_id>/schedules Accept: application/json { description: "Runs after other pipelines", justOnce: false, enabled: false, firstExecution: "2014-08-08T15:15:15.555Z", afterPipelines: [15, 54], period: null, periodUnit: null, }
Response
HTTP/1.1 200 OK Content-Type: application/json { id: 4, description: "Runs after other pipelines", justOnce: false, enabled: false, firstExecution: "2014-08-08T15:15:15.555Z", lastExecution: null, nextExecution: null, afterPipelines: [15, 54], period: null, periodUnit: null, }
Returns the entity in its new version.
Response, error
HTTP/1.1 500 Internal Server Error Content-Type: application/json { "errorMessage": "user readable message", "technicalMessage": "some technical message" }
3. Pipeline.Executions
3.1 Get all Pipeline.Executions
Request
GET <prefix>/pipelines/<pipeline_id>/executions/ Accept: application/json
Response
HTTP/1.1 200 OK Content-Type: application/json [ { id: 1, status: "FAILED", isDebugging: false, orderNumber: 55, start: "2014-08-08T15:15:15.555Z", end: "2014-08-08T15:15:15.555Z", schedule: 15, stop: false, lastChange: "2014-08-08T15:15:15.555Z" }, { id: 2, status: "RUNNING", isDebugging: true, orderNumber: 98, start: "2014-08-08T15:15:15.555Z", end: "2014-08-08T15:15:15.555Z", schedule: null, stop: false, lastChange: "2014-08-08T15:15:15.555Z" }, { id: 3, status: "FINISHED_SUCCESS", isDebugging: false, orderNumber: 1, start: "2014-08-08T15:15:15.555Z", end: "2014-08-08T15:15:15.555Z", schedule: 12, stop: false, lastChange: "2014-08-08T15:15:15.555Z" } ]
Returns execution entity:
- status: QUEUED, RUNNING, CANCELLING, CANCELLED, FAILED, FINISHED_SUCCESS, FINISHED_WARNING
- orderNumber: the priority in queue
- start , end : real times of pipeline execution start and end
- schedule: id of schedule entity which triggered the execution, if any (null when pipeline execution was created manually by user)
- silentMode: (from javadoc:It true pipeline run in silent mode and the end of the execution can't be used to fire schedule.)
- stop: (from javadoc: True if pipeline should or has been stopped on user request.)
- lastChange: (from javadoc: Timestamp when this execution was last changed.)
Response, no entity for given id
HTTP/1.1 404 Not Found Content-Type: application/json { "errorMessage": "Proces s ID='xyz' sa nepodarilo nájsť!", "technicalMessage": "Pipeline with id=xyz doesn't exist!" }
Response, error
HTTP/1.1 500 Internal Server Error Content-Type: application/json { "errorMessage": "user readable message", "technicalMessage": "some technical message" }
3.2 Get one Pipeline.Execution
Request
GET <prefix>/pipelines/<pipeline_id>/executions/<execution_id> Accept: application/json
Response
HTTP/1.1 200 OK Content-Type: application/json { id: 3, status: "FINISHED_SUCCESS", isDebugging: false, orderNumber: 1, start: "2014-08-08T15:15:15.555Z", end: "2014-08-08T15:15:15.555Z", schedule: 12, stop: false, lastChange: "2014-08-08T15:15:15.555Z" }
Returns single pipeline execution entry based on its id.
Response, no entity for given id
HTTP/1.1 404 Not Found Content-Type: application/json { "errorMessage": "Proces s ID='xyz' sa nepodarilo nájsť!", "technicalMessage": "Pipeline with id=xyz doesn't exist!" }
Response, error
HTTP/1.1 500 Internal Server Error Content-Type: application/json { "errorMessage": "user readable message", "technicalMessage": "some technical message" }
3.3 Get last finished Pipeline.Execution
This function will be removed in future and replaced by correct where and orderBy handling in standard GET method
Request
GET <prefix>/pipelines/<pipeline_id>/executions/last Accept: application/json
Response
HTTP/1.1 200 OK Content-Type: application/json { id: 3, status: "FINISHED_SUCCESS", isDebugging: false, orderNumber: 1, start: "2014-08-08T15:15:15.555Z", end: "2014-08-08T15:15:15.555Z", schedule: 12, stop: false, lastChange: "2014-08-08T15:15:15.555Z" }
Returns single pipeline execution entry, last one according to end time, only statuses CANCELLED FINISHED_SUCCESS FINISHED_WARNING FAILED are considered.
Response, no entity found
HTTP/1.1 404 Not Found Content-Type: application/json { "errorMessage": "Proces s ID='xyz' sa nepodarilo nájsť!", "technicalMessage": "Pipeline with id=xyz doesn't exist!" }
Response, error
HTTP/1.1 500 Internal Server Error Content-Type: application/json { "errorMessage": "user readable message", "technicalMessage": "some technical message" }
3.4 Get last running/queued Pipeline.Execution
This function will be removed in future and replaced by correct where and orderBy handling in standard GET method
Request
GET <prefix>/pipelines/<pipeline_id>/executions/pending Accept: application/json
Response
HTTP/1.1 200 OK Content-Type: application/json [{ id: 3, status: "QUEUED", isDebugging: false, orderNumber: 1, start: "2014-08-08T15:15:15.555Z", end: "2014-08-08T15:15:15.555Z", schedule: 12, stop: false, lastChange: "2014-08-08T15:15:15.555Z" }]
Returns single pipeline execution entry, last one according to end time, only statuses QUEUED RUNNING(CANCELLING are selected but presented as RUNNING) are considered.
Response, no entity found
HTTP/1.1 404 Not Found Content-Type: application/json { "errorMessage": "Proces s ID='xyz' sa nepodarilo nájsť!", "technicalMessage": "Pipeline with id=xyz doesn't exist!" }
Response, error
HTTP/1.1 500 Internal Server Error Content-Type: application/json { "errorMessage": "user readable message", "technicalMessage": "some technical message" }
3.5 Create Pipeline.Execution
Request
POST <prefix>/pipelines/<pipeline_id>/executions Accept: application/json Content-type: application/json
Sample request to run in normal mode:
curl -H "Accept: application/json" -H "Content-type: application/json" -X POST --data '{ "userExternalId" : "http://www.johnadmin.cz" }' http://master:commander@localhost:8080/master/api/1/pipelines/33/executions
Sample request to run in debug mode:
curl -H "Accept: application/json" -H "Content-type: application/json" -X POST --data '{ "debugging" : "true", "userExternalId" : "http://www.johnadmin.cz" }' http://master:commander@localhost:8080/master/api/1/pipelines/33/executions
For UnifiedViews <2.3
For UnifiedViews < 2.3, please use "isDebugging" instead of "debugging"
Response
HTTP/1.1 200 OK Content-Type: application/json {"id":321,"status":"QUEUED","orderNumber":1,"start":null,"end":null,"schedule":null,"stop":false,"lastChange":"2017-03-08T18:12:31.535+0100","userExternalId":"http://www.johnadmin.cz","userActorExternalId":null,"debugging":false}
From this response, client obtains id of PipelineExecution, which he can then check for modifications periodically to update status.
Response, error
HTTP/1.1 500 Internal Server Error Content-Type: application/json { "errorMessage": "user readable message", "technicalMessage": "some technical message" }
4 Pipeline.Execution.Events
4.1 Get all Pipeline.Execution.Events
Request
GET <prefix>/pipelines/<pipeline_id>/executions/<execution_id>/events Accept: application/json
Response
HTTP/1.1 200 OK Content-Type: application/json [ { id: 1, time: "2015-15-15T15:15:15.658Z", type: "DPU_DEBUG", shortMessage: "Everything is OK!", fullMessage: "As expected this is very long message on many lines", dpuInstance: { id: 546, name: "CSV tool", description: "mastering CSV is good, this dpu does that", serializedConfiguration: "<conf> </conf>" } }, { id: 2, time: "2015-15-15T15:15:16.658Z", type: "PIPELINE_ERROR", shortMessage: "Bad things happend", fullMessage: "As expected this is very long message on many lines", dpuInstance: null } ]
- type: DPU_DEBUG, DPU_INFO, DPU_WARNING, DPU_ERROR, DPU_TERMINATION_REQUEST, PIPELINE_ERROR, PIPELINE_INFO. Last two can have null dpuInstance field (produced by UV not DPU).
- dpuInstance: basic info about DPU instance that emitted the message.
Response, error
HTTP/1.1 500 Internal Server Error Content-Type: application/json { "errorMessage": "user readable message", "technicalMessage": "some technical message" }
5. Pipeline.Schedule.ScheduledExecutions
5.1 Get next Pipeline.Schedule.ScheduledExecution
For the given pipeline, get next scheduled executions
Request
GET <prefix>/pipelines/<pipeline_id>/schedules/~all/scheduledexecutions Accept: application/json
Response for schedule with starting time
HTTP/1.1 200 OK Content-Type: application/json [{ "start": "2014-08-08T15:15:15.555Z", "schedule": 12, "afterPipelines": null }]
Response for pipeline that is scheduled after another pipeline
HTTP/1.1 200 OK Content-Type: application/json [{ "start": null, "schedule": 12, "afterPipelines": [ {"1": "pipeline name for pipe with id 1"}, {"2": "pipeline name for pipe with id 2"} ] }]
Response, no entity found
HTTP/1.1 404 Not Found Content-Type: application/json []
Response, error
HTTP/1.1 500 Internal Server Error Content-Type: application/json { "errorMessage": "user readable message", "technicalMessage": "some technical message" }
6.DPUs
6.1 Import DPU
Why
Deployment of DPU by script.
Request
POST <prefix>/import/dpu/jar?name=&description=&visibility=&force= form-data; name="file"; filename="name_of_dpu.jar"
- name: optional, type - String, if set, this name will override DPU's default name.
- decription: optional, type - String, if set, this description will override DPU's default description.
- visibility: optional, type - enum {cz.cuni.mff.xrg.odcs.commons.app.auth.ShareType}, visibility of imported DPU, if not present, default value 'PUBLIC_RO' is used.
- force: optional, type - boolean, if dpu already exists, and flag is set to true, API will try to replace it. If dpu is already used in any pipeline, import will fail.
Response
HTTP/1.1 200 OK Content-Type: application/json [OK]
Response, DPU exists
HTTP/1.1 400 Bad Request Content-Type: application/json { "error": "DPU already exists!" }
Response, DPU used in pipelines
HTTP/1.1 409 Conflict Content-Type: application/json { "error": "DPU is already used in pipelines!." }
Response, visibility invalid value
HTTP/1.1 400 Bad Request Content-Type: application/json { "error":"No enum constant cz.cuni.mff.xrg.odcs.commons.app.auth.ShareType.foo" }
Response, name invalid value
HTTP/1.1 400 Bad Request Content-Type: application/json { "error": "Cannot process DPU name!" }
7.USERs
7.1 Create User
Why
For external tool that will handle user management, e.g. provisioning by Midpoint. Note: Will work reasonably for UV + CAS
Request
POST <prefix>/users/create Accept: application/json { "Subject.UPVSIdentityID":"1AE84104-6E73-4907-AF8B-ADC9BCD9AE37", "Subject.FormattedName":"NASES", "Actor.UPVSIdentityID":"1DC6DFC0-5499-45D3-A833-B18E942D4EC2", "Actor.FormattedName":"Andrej SprávcaTransformáciíMOD", "SPR.Roles":"MOD-R-TRANSA" }
Warning! This function works as UPSERT. Create user if none, update if exists.
Required JSON fields are parametrized in frontend.properties file. E.g. this is part of frontend properties, which will accept given request.
#CAS attribute representing full name cas.attributeName.userName = Subject.UPVSIdentityID #CAS attribute representing full name cas.attributeName.fullName = Subject.FormattedName #CAS attribute representing actor external identifier cas.attributeName.actorId = Actor.UPVSIdentityID #CAS attribute representing actor name cas.attributeName.actorName = Actor.FormattedName #CAS attribute representing roles cas.attributeName.role = SPR.Roles
Response
{ "success": true }