Skip to content

API

Jump to bottom
Bart Geesink edited this page Sep 23, 2024 · 34 revisions

Manage API's

Manage has three API's:

  • Client API for the Manage JavaScript GUI application which is secured using federative authentication (using shibboleth)
  • Internal Metadata READ / WRITE API for other applications secured by basic authentication and scopes (using ansible)
  • Recent activity secured by basic authentication (using ansible)

Client API

The first API is self-explaining and only interesting for Manage developers.

READ API

The read API is very simple. It has two endpoints: one for searching and one for retrieval of entire documents by primary key.

Search API

You can specify which fields you want to fetch with the REQUESTED_ATTRIBUTES key and optionally which fields to search. Example:

curl -H 'Content-Type: application/json' -u pdp:secret  -X POST -d \
'{"metaDataFields.AssertionConsumerService:0:Location":".*","metaDataFields.coin:policy_enforcement_decision_required":"1","REQUESTED_ATTRIBUTES":["metaDataFields.AssertionConsumerService:0:Location","metaDataFields.coin:policy_enforcement_decision_required"]}' \
'http://localhost:8080/manage/api/internal/search/saml20_sp'

Or against a deployed application on a test environment:

curl -H 'Content-Type: application/json' -u pdp:secret  -X POST -d '{"metaDataFields.AssertionConsumerService:0:Location":".*","REQUESTED_ATTRIBUTES":["entityid"]}' 'https://manage.example.org/manage/api/internal/search/saml20_sp'

The query will default AND the different inputs. To query with the logical OR operator you need to specify the optional LOGICAL_OPERATOR_IS_AND post parameter to false.

curl -H 'Content-Type: application/json' -u pdp:secret  -X POST -d '{"metaDataFields.contacts:0:contactType":"technical","metaDataFields.contacts:1:contactType":"technical","metaDataFields.contacts:2:contactType":"technical","REQUESTED_ATTRIBUTES":["metaDataFields.contacts:0:emailAddress", "metaDataFields.contacts:0:contactType","metaDataFields.contacts:1:emailAddress", "metaDataFields.contacts:1:contactType","metaDataFields.contacts:2:emailAddress", "metaDataFields.contacts:2:contactType"],"LOGICAL_OPERATOR_IS_AND": false }' 'http://localhost:8080/manage/api/internal/search/saml20_sp' | jq .

Wildcards like .*shib.* are translated to a regular expression search. Specify booleans with 0 or 1 and leave the value empty for a does not exists query. The result will always - depending on the type of Metadata specified as a path variable - return some defaults fields. For SP / IdP Metadata the default fields are:

  • entityid
  • state
  • metaDataFields.name:en
  • metaDataFields.name:nl

The following example will return the default fields and the ARP for the SP with the specified entityid:

curl -H 'Content-Type: application/json' -u pdp:secret  -X POST -d \
'{"entityid":"https://dmsonline.uvt.nl/nl/home","REQUESTED_ATTRIBUTES":["arp"]}' \
'http://localhost:8080/manage/api/internal/search/saml20_sp'

And will return the following JSON:

[{
	"_id": "362c928f-e80c-4318-a04e-c7dc072c131c",
	"data": {
		"entityid": "https://example.org/nl/home",
		"state": "testaccepted",
		"arp": {
			"attributes": {
				"urn:mace:dir:attribute-def:displayName": [{
					"source": "idp",
					"value": "*"
				}],
				"urn:mace:dir:attribute-def:eduPersonAffiliation": [{
					"source": "idp",
					"value": "*"
				}],
				"urn:mace:dir:attribute-def:cn": [{
					"source": "idp",
					"value": "*"
				}],
				"urn:mace:dir:attribute-def:eduPersonPrincipalName": [{
					"source": "idp",
					"value": "*"
				}],
				"urn:mace:dir:attribute-def:mail": [{
					"source": "idp",
					"value": "*"
				}],
				"urn:mace:terena.org:attribute-def:schacHomeOrganization": [{
					"source": "idp",
					"value": "*"
				}]
			},
			"enabled": true
		},
		"metaDataFields": {
			"name:en": "Example SP",
			"name:nl": "Voorbeeld-SP"
		}
	}
}]

You can also use a list as the search value and this gets converted to a query where the key is in the provided list of values.

Example which selects entities that have any of a number of urls in their entity_categories fields:

{
  "ALL_ATTRIBUTES": true,
  "metaDataFields.coin:entity_categories:1": [
    "http://refeds.org/category/research-and-scholarship",
    "http://www.geant.net/uri/dataprotection-code-of-conduct/v1",
    "https://refeds.org/category/code-of-conduct/v2"
  ],
  "metaDataFields.coin:entity_categories:2": [
    "http://refeds.org/category/research-and-scholarship",
    "http://www.geant.net/uri/dataprotection-code-of-conduct/v1",
    "https://refeds.org/category/code-of-conduct/v2"
  ],
  "metaDataFields.coin:entity_categories:3": [
    "http://refeds.org/category/research-and-scholarship",
    "http://www.geant.net/uri/dataprotection-code-of-conduct/v1",
    "https://refeds.org/category/code-of-conduct/v2"
  ],
  "metaDataFields.coin:entity_categories:4": [
    "http://refeds.org/category/research-and-scholarship",
    "http://www.geant.net/uri/dataprotection-code-of-conduct/v1",
    "https://refeds.org/category/code-of-conduct/v2"
  ],
  "LOGICAL_OPERATOR_IS_AND": false
}

If you omit the REQUESTED_ATTRIBUTES and add the parameter ALL_ATTRIBUTES with the value true all the fields will be returned. Example:

curl -H 'Content-Type: application/json' -u pdp:secret  -X POST -d '{"ALL_ATTRIBUTES":true}' 'http://localhost:8080/manage/api/internal/search/saml20_sp' | jq .

Will return (result shortened for readability):

[    
  {
        "_id": "0eda9504-0be6-4966-ad01-37b2daae2439",
        "data": {
            "active": true,
            "allowedEntities": [],
            "allowedall": true,
            "arp": {
                "attributes": {
                    "urn:mace:dir:attribute-def:displayName": [
                        {
                            "source": "idp",
                            "value": "*"
                        }
                    ],
                    "urn:mace:dir:attribute-def:eduPersonPrincipalName": [
                        {
                            "source": "idp",
                            "value": "*"
                        }
                    ]
                },
                "enabled": true
            },
            "eid": 54,
            "entityid": "http://example.openconext.org/authentication/metadata",
            "id": 348,
            "ip": "145.100.191.122",
            "manipulation": null,
            "metaDataFields": {
                "AssertionConsumerService:0:Binding": "urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST",
                "AssertionConsumerService:0:Location": "http://example.openconext.org/authentication/consume-assertion",
                "AssertionConsumerService:0:index": "0",
                "NameIDFormat": "urn:oasis:names:tc:SAML:2.0:nameid-format:transient",
                "contacts:0:contactType": "technical",
                "contacts:1:contactType": "technical",
                "contacts:2:contactType": "technical",
                "description:en": "Connection server - now",
                "description:nl": "Connection server - now",
                "logo:0:height": "60",
                "logo:0:url": "https://.png",
                "logo:0:width": "120",
                "name:en": "OpenConext connection server",
                "name:nl": "OpenConext connection server"
            },
            "metadataurl": "http://example.openconext.org/authentication/metadata",
            "notes": null,
            "revisionid": 4,
            "revisionnote": "No revision note",
            "state": "prodaccepted",
            "type": "saml20-sp",
            "user": "urn:collab:person:example.com:admin"
        },
        "revision": {
            "created": 1453990922000,
            "number": 4,
            "updatedBy": "urn:collab:person:example.com:admin"
        },
        "type": "saml20_sp",
        "version": 0
    }
]

Another example to retrieve all single tenant templates:

curl -H 'Content-Type: application/json' -u pdp:secret  -X POST -d '{"ALL_ATTRIBUTES":true}' 'https://manage.example.org/manage/api/internal/search/single_tenant_template' | jq .

Or to retrieve a single / complete IDP by entityId:

 curl -H 'Content-Type: application/json' -u pdp:secret  -X POST -d '{"ALL_ATTRIBUTES":true, "entityid":"http://mock-idp"}' 'https://manage.example.org/manage/api/internal/search/saml20_idp' | jq .

Add the URL parameter nested=1 to the search api request to instead of (e.g.) "displayName:nl": "aap" receive "displayName": {"nl": "aap"}.

Revisions

Given an entityID you can fetch the revisions for that entity:

curl -H 'Content-Type: application/json' -u yyy:xxx  -X POST -d '{"entityid":"https://www.example.edu/wayf/module.php/saml/sp/metadata.php/default-sp"}' 'http://localhost:8080/manage/api/internal/search/saml20_sp_revision'

(The call implicitly only searches the data dict, so you can only specify fields in that dict.)

Document retrieval API

In order to use this API call you need to know the primary key of the MetaData which is returned in the Search API. You can not store the primary key in your own database as it might change (as we re-migrate the data).

curl -u sp-portal:secret -H 'Content-Type: application/json' 'http://localhost:8080/manage/api/internal/metadata/saml20_sp/176584e4-6f0c-4ec2-a0b9-e1bac1ad1aab'

WRITE API

There are basically two Write API's. One is based on a merge principle and one is based on a you-provide-everything principle.

MERGE WRITE

You specify the exact path of the values and the actually new values of the fields you want to update. If you want to delete a metaDataField then provide a null value. The following example adds an IdP to the array of allowedEntities of the specified SP, sets the top-level attribute allowedall to false and finally updates or adds the metadata field description:en. Note that the id of the Metadata you want to update must already be retrieved by a search query:

curl -H 'Content-Type: application/json' -u sp-portal:secret \
-X PUT -d '{"id": "bac56ecd-29aa-449f-81ff-109cff1f90c4", "type": "saml20_sp", "pathUpdates": { "allowedall": false, "allowedEntities": [{ "name": "https://allow-me" }], "metaDataFields.description:en": "New description", "revisionnote":"setting allowedEntities to allow-me, updating description" }}' \
'http://localhost:8080/manage/api/internal/merge'

The entire updated document is sent back in response to the update:

{
	"id": "fa4fd71d-b9d1-4da7-95c2-3be6bbeaf1f0",
	"version": 9,
	"type": "saml20_sp",
	"revision": {
		"number": 22,
		"created": 1504195745.015000000,
		"parentId": null,
		"updatedBy": "sp-portal"
	},
	"data": {
		"id": 281,
		"eid": 1,
		"entityid": "https://engine.test.openconext.org/authentication/sp/metadata",
		"revisionid": 13,
		"state": "prodaccepted",
		"type": "saml20-sp",
		"metadataurl": "https://metatdataurl",
		"allowedall": false,
		"manipulation": null,
		"user": "urn:collab:person:example.com:admin",
		"created": "2015-12-16T17:11:17+01:00",
		"ip": "145.100.191.122",
		"revisionnote": "setting allowedEntities to allow-me, updating description",
		"active": true,
		"arp": {
			"attributes": {},
			"enabled": false
		},
		"notes": null,
		"metaDataFields": {
			"displayName:nl": "OpenConext Engine",
			"NameIDFormats:0": "urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified",
			"url:en": "https://engine.openconext.org",
			"description:en": "OpenConext SSO Proxy",
			"contacts:2:givenName": "Support",
			"contacts:1:surName": "OpenConext",
			"contacts:2:surName": "OpenConext",
			"contacts:2:emailAddress": "[email protected]",
			"description:nl": "OpenConext SSO Proxy",
			"logo:0:width": "96",
			"logo:0:url": "https://static.test.openconext.org.nl/media/conext_logo.png",
			"name:en": "OpenConext Engine",
			"contacts:0:givenName": "Support",
			"contacts:0:surName": "OpenConext Engine",
			"contacts:1:emailAddress": "[email protected]",
			"contacts:0:emailAddress": "[email protected]",
			"AssertionConsumerService:0:Location": "https://engine.test.openconext.org/authentication/sp/consume-assertion",
			"contacts:0:contactType": "technical",
			"contacts:2:contactType": "administrative",
			"logo:0:height": "96",
			"contacts:1:contactType": "technical",
			"contacts:1:givenName": "Support",
			"name:nl": "OpenConext Engine",
			"NameIDFormat": "urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified",
			"AssertionConsumerService:0:Binding": "urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST",
			"url:nl": "https://engine.test.openconext.org",
			"displayName:en": "OpenConext Engine"
		},
		"allowedEntities": []
	}
}

If an invalid request results in errors because of schema validations then a BAD_REQUEST is returned with the validation errors. The invalid request - invalid enum for NameIDFormat -:

curl -H 'Content-Type: application/json' -u sp-portal:secret  -X PUT \
-d '{"id": "fa4fd71d-b9d1-4da7-95c2-3be6bbeaf1f0","type": "saml20_sp","pathUpdates": {"metaDataFields.NameIDFormats:0": "bogus"}}' \
'http://localhost:8080/manage/api/internal/metadata'

And the result:

{
	"timestamp": 1506060789246,
	"status": 400,
	"error": "org.everit.json.schema.ValidationException",
	"exception": "org.everit.json.schema.ValidationException",
	"message": "#/metaDataFields/NameIDFormats:0: bogus is not a valid enum value",
	"path": "/manage/api/internal/metadata",
	"validations": "#/metaDataFields/NameIDFormats:0: bogus is not a valid enum value"
}

Warning: In order to update arp attributes or make changes to the allowedEntities you will need to provide all the data +for those metadata parts.

PROVIDE-EVERYTHING WRITE

To create a new MetaData you can POST the entire document to the API:

curl -H 'Content-Type: application/json' -u sp-portal:secret \
-X POST -d '${THE_ENTIRE_DOCUMENT}' \
'http://localhost:8080/manage/api/internal/metadata'

With the provide-everything WRITE API you can also change the document by a PUT of the entire valid MetaData to the server. It is required that you have all the data and thus it is required you fetched the MetaData by its primary key and not using the Search API:

After changing MetaData you can use the following PUT to make updates:

curl -H 'Content-Type: application/json' -u sp-portal:secret \
-X PUT -d '${THE_ENTIRE_DOCUMENT}' \
'http://localhost:8080/manage/api/internal/metadata'

Validation

There is also a Validate endpoint which can be used to validate metadata before actually saving the metadata:

curl -H 'Content-Type: application/json' -u sp-portal:secret  -X POST \
 -d '{"type":"saml20_sp","data":{"entityid":"Duis ad do 2","state":"testaccepted","allowedall":true,"metaDataFields":{"name:en":"Test","AssertionConsumerService:0:Binding":"bogus","AssertionConsumerService:0:Location":"https://acs"}}}' \ 'http://localhost:8080/manage/api/internal/validate/metadata'

Note that you need to handle bad requests, because that is the result if there are validation exceptions

HTTP/1.1 400
{
	"timestamp": 1506676418561,
	"status": 400,
	"error": "org.everit.json.schema.ValidationException",
	"exception": "org.everit.json.schema.ValidationException",
	"message": "#/metaDataFields/AssertionConsumerService:0:Binding: bogus is not a valid enum value",
	"path": "/manage/api/internal/validate/metadata",
	"validations": "#/metaDataFields/AssertionConsumerService:0:Binding: bogus is not a valid enum value"
}

Delete an entity

The delete entity endpoint can be called at /manage/api/internal/metadata/<type>/<id> with the DELETE method:

curl -u user:example -X DELETE http://localhost:8080/manage/api/internal/metadata/saml20_sp/fa4fd71d-b9d1-4da7-95c2-3be6bbeaf1f0

Push new metadata to EB

After updating the MetaData the changes need to be pushed by Manage to EB. There is a endpoint to initiate the push. The configured API client will need the PUSH scope in order to perform the PUSH.

curl -u sp-portal:secret -H 'Content-Type: application/json' 'http://localhost:8080/manage/api/internal/push'

Delete metaData key API

The internal API has an endpoint to delete a metaData key from all entities with that key in the metaDataFields attribute.

curl -u sp-portal:secret -X PUT -H "Content-Type: application/json" -d '{"type":"saml20_sp","metaDataKey":"description:nl"}' "http://localhost:8080/manage/api/internal/delete-metadata-key" | jq .

The return value is an array with all entity Ids that are updated. For each updated metaData a revision is created to preserve the history.

Recent activity

The internal API has an endpoint to retrieve the recently updated MetaData.

curl -H 'Content-Type: application/json' -u sysadmin:secret  -X POST -d \
'{"types":["saml20_idp","saml20_sp","oidc10_rp","single_tenant_template"], "limit": 25}' \
'http://localhost:8080/manage/api/internal/recent-activity'

Will return the following information sorted by revision.created:

[
  {
    "id": "d8865016-8719-44cb-989d-d301def43084",
    "version": null,
    "type": "saml20_sp",
    "revision": {
      "number": 0,
      "created": "2020-09-14T14:09:26.307Z",
      "parentId": null,
      "updatedBy": "edugain-import",
      "terminated": null
    },
    "data": {
      "entityid": "https://www.edu-zone.eu/wayf/module.php/saml/sp/metadata.php/default-sp",
      "state": "prodaccepted",
      "metaDataFields": {
        "name:en": "EduZone - All your clouds under one umbrella"
      },
      "revisionnote": "edugain-import"
    }
  },
  {
    "id": "a6d4d901-8606-4a4d-8f6f-6ef131be9df6",
    "version": null,
    "type": "saml20_sp",
    "revision": {
      "number": 0,
      "created": "2020-09-14T14:09:26.300Z",
      "parentId": null,
      "updatedBy": "edugain-import",
      "terminated": null
    },
    "data": {
      "entityid": "https://monitor.eduroam.org/sp/module.php/saml/sp/metadata.php/default-sp",
      "state": "prodaccepted",
      "metaDataFields": {
        "name:en": "eduroam supporting services"
      },
      "revisionnote": "edugain-import"
    }
  }
]

The default for types is ["saml20_idp","saml20_sp","oidc10_rp"] and the default for limit is 25 and the maximum value for limit is 100. To use the defaults you can omit the request body:

curl -H 'Content-Type: application/json' -u sysadmin:secret  -X POST \
'http://localhost:8080/manage/api/internal/recent-activity'

Authentication & authorization configuration

In the Manage application configuration you specify - or override using ansible - the location of the Manage api user configuration. This YML file contains all the internal API users, passwords and scopes. Example:

apiUsers:
  - {
      name: "attribute-aggregator",
      password: "secret",
      scopes: [READ]
    }
  - {
      name: "pdp",
      password: "secret",
      scopes: [READ]
    }
  - {
      name: "sp-portal",
      password: "secret",
      scopes: [READ, WRITE_SP, PUSH]
    }

The following scopes can be configured:

Scope Description
ADMIN Standard scope for all GUI related endpoint (e.g. /manage/api/client/** endpoints)
CHANGE_REQUEST_IDP Allowed to create change requests for IdP
CHANGE_REQUEST_SP Allowed to create change requests for SP
POLICIES Allowed to create (excluding Identity Providers) and update all entities
PUSH Allowed to push changes to EB & OIDC-NG
READ Allowed to read entities
SYSTEM Allowed everything including Attribute Manipulation
TEST Only used internally
WRITE_SP Allowed to CRU SP / RP / RS / SRAM
DELETE_SP Allowed to Delete SP / RP / RS
WRITE_IDP Allowed to CRUD IdP
Clone this wiki locally