//
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements.  See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership.  The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License.  You may obtain a copy of the License at
//
//   http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied.  See the License for the
// specific language governing permissions and limitations
// under the License.
//
[[core-usage]]
=== Core

All the features provided by the <<core>> are available as RESTful services.

The base URL for invoking such services is normally set as

....
protocol://host:port/syncope/rest/
....

where `protocol`, `host` and `port` reflect your deployment.

[NOTE]
.REST Reference
====
A complete REST reference generated from https://swagger.io/specification/[OpenAPI specification 3.0^] is
https://syncope.apache.org/rest/4.1/openapi.json[published^] as well as made available with each deployment at

....
protocol://host:port/syncope/rest/openapi.json
....

where `protocol`, `host` and `port` reflect your deployment.

REST APIs are available to visualize and interact via https://swagger.io/tools/swagger-ui/[Swagger UI^] at

....
protocol://host:port/syncope/
....
====

==== REST Authentication and Authorization

The <<core>> authentication and authorization is based on https://spring.io/projects/spring-security[Spring Security^].

As an initial step, https://en.wikipedia.org/wiki/Basic_access_authentication[HTTP Basic Authentication] is required to
obtain, in the `X-Syncope-Token` HTTP header, the unique signed
https://en.wikipedia.org/wiki/JSON_Web_Token[JSON Web Token^].

By providing the token received in the initial exchange for
https://en.wikipedia.org/wiki/Access_token[HTTP Bearer Token Authentication], the requester can be identified and
checked for authorization, based on owned <<entitlements,entitlements>>.

===== JWTSSOProvider

Besides validating and accepting the JSON Web Tokens generated during the authentication process as sketched above,
Apache Syncope can be enabled to cope with tokens generated by third parties, by providing implementations of the
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/core/spring/src/main/java/org/apache/syncope/core/spring/security/JWTSSOProvider.java[JWTSSOProvider^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/tree/4_1_X/core/spring/src/main/java/org/apache/syncope/core/spring/security/JWTSSOProvider.java[JWTSSOProvider^]
endif::[]
interface. +
An example implementation is provided by
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/core/spring/src/main/java/org/apache/syncope/core/spring/security/MSEntraJWTSSOProvider.java[MSEntraJWTSSOProvider^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/tree/4_1_X/core/spring/src/main/java/org/apache/syncope/core/spring/security/MSEntraJWTSSOProvider.java[MSEntraJWTSSOProvider^]
endif::[]
, to support authentication via Microsoft Entra.

===== Authentiation Credentials

By default, Basic Authentication requires username and password.

When https://en.wikipedia.org/wiki/multi-factor_authentication[multi-factor authentication^] (MFA) is enabled via the
`mfa.enabled` <<configuration-parameters,parameter>>, however, an additional OTP value shall be provided, following the
https://en.wikipedia.org/wiki/Time-based_one-time_password[TOTP] algorithm.

One of apps supporting TOTP, such as https://freeotp.github.io/[FreeOTP^] (or any proprietary or open source
alternative), shall be enrolled first by each User, in order to be able to provide all the credentials requested.

[[rest-authorization-summary]]
.REST Authorization Summary
****
The set of RESTful services provided by Apache Syncope can be divided as:

. endpoints accessible without any sort of authentication (e.g. truly anonymous), for self-registration and
<<password-reset,password reset>>;
. endpoints disclosing information about the given Syncope deployment (available <<schema,schema>>, configured
<<extensions,extensions>>, Groups, ...), requiring some sort of shared authentication defined by the
`security.anonymousKey` Spring Boot property value - for more information, read about Spring Security's
https://docs.spring.io/spring-security/reference/6.5/servlet/authentication/anonymous.html#page-title[Anonymous Authentication^];
. endpoints for self-service (self-update, password change, ...), requiring user authentication and no entitlements;
. endpoints for administrative operations, requiring user authentication with authorization granted by the related
<<entitlements,entitlements>>, handed over to users via <<roles,roles>>.
****

==== REST Headers

Apache Syncope supports a number of HTTP headers as detailed below, in addition to the common HTTP headers such as
`Accept`, `Content-Type`, etc.

[TIP]
It is possible to deal with the headers below when using the <<client-library>> via the `SyncopeClient` class methods.

===== X-Syncope-Token

`X-Syncope-Token` is returned on response to <<rest-authentication-and-authorization,successful authentication>>, and
contains the unique signed https://en.wikipedia.org/wiki/JSON_Web_Token[JSON Web Token^] identifying the authenticated
user.

The value returned for the `X-Syncope-Token` header must be included in all subsequent requests, in order for the
requester to be checked for authorization, as part of the standard https://tools.ietf.org/html/rfc6750[Bearer^]
`Authorization` header.

.Obtaining JWT with http://curl.haxx.se/[curl^] 
====
....
curl -I -u admin:password -X POST http://localhost:9080/syncope/rest/accessTokens/login
....
(when MFA is not enabled)

or

....
curl -I -u admin:password:123456 -X POST http://localhost:9080/syncope/rest/accessTokens/login
....
(with MFA enabled)

returns

....
HTTP/1.1 204 
X-Syncope-Token: eyJ0e..
....

which can then be used to make a call to the REST API

.....
curl -I -H "Authorization: Bearer eyJ0e.." http://localhost:9080/syncope/rest/users/self
.....
====

The token duration can be configured via the `jwt.lifetime.minutes` property - see
<<configuration-parameters, below>> for details.

===== X-Syncope-Domain

`X-Syncope-Domain` can be optionally set for requests (when not set, `Master` is assumed) to select the target
<<domains,domain>>. +
The value for this header is provided in all responses.

===== X-Syncope-Key and Location

When creating an entity (User, Group, Schema, External Resource, ...) these two headers are populated respectively with
the entity key (which may be auto-generated) and the absolute URI identifying the new REST resource.

===== X-Application-Error-Code and X-Application-Error-Info

If the requested operation is in error, `X-Application-Error-Code` will contain the error code (mostly from
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/common/idrepo/lib/src/main/java/org/apache/syncope/common/lib/types/ClientExceptionType.java[ClientExceptionType^])
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/4_1_X/common/idrepo/lib/src/main/java/org/apache/syncope/common/lib/types/ClientExceptionType.java[ClientExceptionType^])
endif::[]
and `X-Application-Error-Info` might be optionally populated with more details, if available.

===== X-Syncope-Delegated-By

When requesting an operation under <<delegation,Delegation>>, this header must be provided to indicate the delegating
User, either by their username or key.

===== X-Syncope-Null-Priority-Async

When set to `true`, this request header instructs the <<propagation,propagation process>> not to wait for completion
when communicating with <<external-resource-details,External Resources>> with no priority set.

===== X-Syncope-Verfied

This response header bears the boolean result of a verification operation.

===== Prefer and Preference-Applied

Some REST endpoints allow the clients to request certain behavior; this is done via the `Prefer` header.

When `Prefer` is specified in the request, the response will feature the `Preference-Applied` header, with value set
to the effective preference applied.

====== return-content / return-no-content
 
REST endpoints for creating, updating or deleting Users, Groups or Any Objects return the entity in the response payload
by default. +
If this is not required, the `Prefer` request header can be set to `return-no-content` (`return-content` will instead
keep the default behavior).

[TIP]
Use `Prefer: return-no-content` in scenarios where it is important to avoid unnecessary data in the response payload.

====== respond-async

The <<batch>> endpoint can be requested for <<asynchronous-batch-processing,asynchronous processing>>.

===== ETag, If-Match and If-None-Match

For each response containing Users, Groups or Any Objects, the https://en.wikipedia.org/wiki/HTTP_ETag[ETag^] header is
generated, which contains the latest modification date.

This value can be passed, during subsequent requests to modify the same entity, via the `If-Match` or
`If-None-Match` headers. +
When the provided `If-Match` value does not match the latest modification date of the entity, an error is reported and
the requested operation is not performed.

[TIP]
The combined usage of `ETag` and `If-Match` can be enforced to implement optimistic concurrency control over Users,
Groups and Any Objects operations.

===== X-Syncope-Entitlements

When invoking the REST endpoint `/users/self` in `GET`, the `X-Syncope-Entitlements` response header will list all
the <<entitlements,entitlements>> owned by the requesting user.

===== X-Syncope-Delegations

When invoking the REST endpoint `/users/self` in `GET`, the `X-Syncope-Delegations` response header will list all
delegating users for each <<delegation,Delegation>> for which the requesting user is delegated.

==== Batch

Batch requests allow grouping multiple operations into a single HTTP request payload. +
A batch request is represented as a https://tools.ietf.org/html/rfc2046[Multipart MIME v1.0 message^], a standard format
allowing the representation of multiple parts within a single request.

Batch requests are handled by the `/batch` REST endpoint: via HTTP `POST` method to submit requests, via HTTP `GET`
method to fetch responses <<asynchronous-batch-processing,asynchronously>>.

[NOTE]
The specification and implementation of batch processing in Apache Syncope is inspired by the standards defined
by http://docs.oasis-open.org/odata/odata/v4.0/os/part1-protocol/odata-v4.0-os-part1-protocol.html#_Toc372793748[OData 4.0^]

===== Batch requests

The batch request must contain a `Content-Type` header specifying a content type of `multipart/mixed` and a boundary
specification as defined in https://tools.ietf.org/html/rfc2046[RFC2046^].

The body of a batch request is made up of a series of individual requests, each represented as a distinct MIME part
(i.e. separated by the boundary defined in the `Content-Type` header).

Core will process the requests within a batch request sequentially.

An individual request must include a `Content-Type` header with value `application/http` and a
`Content-Transfer-Encoding` header with value `binary`.

.Sample batch request
====
----
--batch_61bfef8d-0a00-41aa-b775-7b6efff37652 // <1>
Content-Type: application/http
Content-Transfer-Encoding: binary
^M // <2>
POST /users HTTP/1.1 // <3>
Accept: application/json
Content-Length: 1157
Content-Type: application/json
^M
{"@class":"org.apache.syncope.common.lib.to.UserTO","key":null,"type":"USER","realm":"/"}
--batch_61bfef8d-0a00-41aa-b775-7b6efff37652
Content-Type: application/http
Content-Transfer-Encoding: binary
^M
POST /groups HTTP/1.1 // <4>
Accept: application/xml
Content-Length: 628
Content-Type: application/xml
^M
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><syncope30:group xmlns:syncope30="https://syncope.apache.org/3.0">
</syncope30:group>
--batch_61bfef8d-0a00-41aa-b775-7b6efff37652
Content-Type: application/http
Content-Transfer-Encoding: binary
^M
PATCH /users/24eb15aebatch@syncope.apache.org HTTP/1.1 // <5>
Accept: application/json
Content-Length: 362
Content-Type: application/json
Prefer: return-no-content
^M
{"@class":"org.apache.syncope.common.lib.request.UserUR","key":"24eb15aebatch@syncope.apache.org"}
--batch_61bfef8d-0a00-41aa-b775-7b6efff37652
Content-Type: application/http
Content-Transfer-Encoding: binary
^M
DELETE /groups/287ede7c-98eb-44e8-979d-8777fa077e12 HTTP/1.1 // <6>
--batch_61bfef8d-0a00-41aa-b775-7b6efff37652--
----
<1> message boundary
<2> represents CR LF
<3> user create, with JSON payload (shortened)
<4> group create, with XML payload (shortened)
<5> user update, with JSON payload (shortened)
<6> group delete
====

===== Batch responses

Requests within a batch are evaluated according to the same semantics used when the request appears outside the context
of a batch.

The order of individual requests in a batch request is significant.

If the set of request headers of a batch request are valid (the `Content-Type` is set to `multipart/mixed`, etc.)
Core will return a `200 OK` HTTP response code to indicate that the request was accepted for processing, and the
related execution results.

If Core receives a batch request with an invalid set of headers it will return a `400 Bad Request` code and perform no
further processing of the request.

A response to a batch request must contain a `Content-Type` header with value `multipart/mixed`.

Structurally, a batch response body must match one-to-one with the corresponding batch request body, such that the same 
multipart MIME message structure defined for requests is used for responses

.Sample batch response
====
----
--batch_61bfef8d-0a00-41aa-b775-7b6efff37652 // <1>
Content-Type: application/http
Content-Transfer-Encoding: binary
^M // <2>
HTTP/1.1 201 Created // <3>
Content-Type: application/json
Date: Thu, 09 Aug 2018 09:55:46 GMT
ETag: "1533808545975"
Location: http://localhost:9080/syncope/rest/users/d399ba84-12e3-43d0-99ba-8412e303d083
X-Syncope-Domain: Master
X-Syncope-Key: d399ba84-12e3-43d0-99ba-8412e303d083
^M
{"entity":{"@class":"org.apache.syncope.common.lib.to.UserTO"}
--batch_61bfef8d-0a00-41aa-b775-7b6efff37652
Content-Type: application/http
Content-Transfer-Encoding: binary
^M
HTTP/1.1 201 Created // <4>
Content-Type: application/xml
Date: Thu, 09 Aug 2018 09:55:46 GMT
ETag: "1533808546342"
Location: http://localhost:9080/syncope/rest/groups/843b2fc3-b8a8-4a8b-bb2f-c3b8a87a8b2e
X-Syncope-Domain: Master
X-Syncope-Key: 843b2fc3-b8a8-4a8b-bb2f-c3b8a87a8b2e
^M
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<syncope30:provisioningResult xmlns:syncope30="https://syncope.apache.org/3.0"></syncope30:provisioningResult>
--batch_61bfef8d-0a00-41aa-b775-7b6efff37652
Content-Type: application/http
Content-Transfer-Encoding: binary
^M
HTTP/1.1 204 No Content // <5>
Content-Length: 0
Date: Thu, 09 Aug 2018 09:55:47 GMT
Preference-Applied: return-no-content
X-Syncope-Domain: Master
^M
--batch_61bfef8d-0a00-41aa-b775-7b6efff37652
Content-Type: application/http
Content-Transfer-Encoding: binary
^M
HTTP/1.1 200 OK // <6>
Content-Type: application/json
Date: Thu, 09 Aug 2018 09:55:47 GMT
X-Syncope-Domain: Master
^M
{"entity":{"@class":"org.apache.syncope.common.lib.to.GroupTO"}
--batch_61bfef8d-0a00-41aa-b775-7b6efff37652--
----
<1> message boundary (same as request)
<2> represents CR LF
<3> user create response, with JSON payload (shortened)
<4> group create respose, with XML payload (shortened)
<5> user update, no content as `Prefer: return-no-content` <<return-content-return-no-content,was specified>>
<6> group delete response, with JSON payload (shortened)
====

===== Asynchronous Batch Processing

Batch requests may be executed asynchronously by <<respond-async,including>> the `respond-async` preference in the
`Prefer` header.

Core will return an empty response, with status `202 Accepted`.

Clients can poll the `/batch` endpoint in `GET` by passing the same boundary used for request: if `202 Accepted` is
returned, then the request is still under processing; otherwise, `200 OK` will be returned, along with the full batch
response. +
Once retrieved, the batch response is not available any more from the `/batch` endpoint.

==== Search

It is possible to search for Users, Groups and Any Objects matching a set of given conditions expressed through
https://cxf.apache.org/docs/jax-rs-search.html#JAX-RSSearch-FeedItemQueryLanguage[FIQL^].

The https://tools.ietf.org/html/draft-nottingham-atompub-fiql-00[Feed Item Query Language^] (FIQL, pronounced “fickle”)
is a simple but flexible, URI-friendly syntax for expressing filters across the entries in a syndicated feed.

The FIQL queries can be passed (among other parameters) to the search endpoints available, e.g.

* `GET /users?fiql=query`
* `GET /groups?fiql=query`
* `GET /anyObjects?fiql=query`
* `GET /resources/{resource}/{anytype}?fiql=query`

where:

* `query` is an URL-encoded string representation of the given FIQL query, as in the following examples;
* `resource` is one of defined <<external-resources,external resources>>;
* `anytype` is one of defined <<anytype,any types>>.

.Simple attribute match
====
----
username==rossini
----
====

.Wildcard attribute match
====
----
username==*ini
----
====

.Case-insensitive attribute match
====
----
username=~rOsSiNi
----
====

.Case-insensitive wildcard attribute match
====
----
username=~*iNi
----
====

.Null attribute match
====
----
loginDate==$null
----
====

.Date attribute comparison
====
----
lastLoginDate=ge=2016-03-02 15:21:22
----
====

.Auxiliary Any Type class assignment
====
----
$auxClasses==csv
----
====

.Resource assignment match
====
----
$resources==resource-ldap
----
====

.Group membership match (only for Users and Any Objects)
====
----
$groups==root
----
====
.Wildcard group membership match (only for Users and Any Objects)
====
----
$groups==*child
----
====

.Role membership match (only for Users)
====
----
$roles==Other
----
====

.Relationship type match (only for Users and Any Objects)
====
----
$relationshipTypes==neighborhood
----
====

.Relationship match (only for Users and Any Objects)
====
----
$relationships==Canon MF 8030c
----
====

.Type match (only for Any Objects)
====
----
$type==PRINTER
----
====

.Complex match (featuring logical AND and OR)
====
----
username=~*iNi;(loginDate==$null,$roles==Other)
----
====

===== Sorting Search Results

Search results can be requested for sorting by passing the optional `orderBy` query parameter to the search endpoints
available, e.g.

* `GET /users?fiql=query&orderBy=sort`
* `GET /groups?fiql=query&orderBy=sort`
* `GET /anyObjects?fiql=query&orderBy=sort`
* `GET /resources/{resource}/{anytype}?orderBy=sort`

where `sort` is an URL-encoded string representation of the sort request, as in the following examples.

.Single attribute sort, default direction (`ASC`)
====
----
username
----
====

.Single attribute sort, with direction
====
----
username DESC
----
====

.Multiple attribute sort, with directions
====
----
email DESC, username ASC
----
====

[[jexl]]
==== JEXL support

https://commons.apache.org/proper/commons-jexl/[Apache Commons JEXL^] is supported as a mean to implement templating
and dynamic value calculation.

Besides https://commons.apache.org/proper/commons-jexl/reference/syntax.html[standard syntax^], the following additional
functions are defined:

* `syncope:fullPath2Dn(fullPath, attr)` - converts full path into the equivalent DN; for example, `/a/b/c` becomes
`ou=c,ou=b,ou=a`
* `syncope:fullPath2Dn(fullPath, attr, prefix)` - converts full path into the equivalent DN, with prefix;
for example, `/a/b/c` with prefix `o=isp,` becomes `o=isp,ou=c,ou=b,ou=a`
* `syncope:connObjAttrValues(connObj, name)` - extracts the values of the attribute with given name from the given
connector object, or empty list if not found
* `syncope:base64Encode(value)` - encodes the given byte array as Base64-encoded string
* `syncope:base64Decode(value)` - decodes the given string as byte array using Base64 encoding

[TIP]
====
More custom functions can be added by providing implementations of the
ifeval::["{snapshotOrRelease}" == "release"]
https://github.com/apache/syncope/blob/syncope-{docVersion}/core/provisioning-api/src/main/java/org/apache/syncope/core/provisioning/api/jexl/JexlFunctions.java[JexlFunctions^]
endif::[]
ifeval::["{snapshotOrRelease}" == "snapshot"]
https://github.com/apache/syncope/blob/4_1_X/core/provisioning-api/src/main/java/org/apache/syncope/core/provisioning/api/jexl/JexlFunctions.java[JexlFunctions^]
endif::[]
interface and registering them as beans, taking care of chosing unique namespace identifiers other than `syncope`.
====
