Async API

Overview

Elide APIs are designed for synchronous request and response behavior. The time allowed to service a synchronous response can be limited by proxy servers and gateways. Analytic queries can often take longer than these limits and result in a server timeout. Elide’s asynchronous API decouples the submission of a request and the delivery of the response into separate client calls. Some of the features available are:

  • Queries are run in a background thread that posts the results into a persistence store.
  • Results can be retrieved as an embedded response or URL for downloading later.
  • Supported downloading formats - CSV and JSON.
  • Queries can be configured to execute synchronously before switching to asynchronous mode if not finished by a client provided threshold.
  • Queries that are interrupted due to an application crash/reboot are automatically flagged as TIMEDOUT.
  • Persisted queries and results are deleted after a configurable threshold.

Design

API

The Async API supports two different query abstractions built using standard Elide models (AsyncQuery and TableExport):

  • Any read-only API request in JSON-API or GraphQL can be submitted by creating an AsyncQuery object. After creation, the client can poll the AsyncQuery object asynchronously for status updates. Once complete, the AsyncQuery object stores the query result as an embedded attribute.
  • For large response bodies, Elide supports a streaming abstraction called TableExport. It works similarly to the AsyncQuery model but with three important differences:
    • TableExport can only be leveraged when:
      • The query is a simple read.
      • The model being queried has no relationships.
      • Only one model is being queried in the request.
    • Upon successful completion, the TableExport model includes a separate URL attribute that references the query results for streaming downloads.
    • The results of the GraphQL or JSON-API query are converted into a simple, flat JSON or CSV format.

Example API requests and responses can be found here.

Threading

Each Elide instance runs a scheduler that is responsible for executing these requests in background threads. New async requests are initially marked in the QUEUED state. The requests are picked for execution as the threads become available. Upon completion, the background thread posts the query status and results to a persistent store. The size of the thread pool can be configured as mentioned here.

Async After

The Async requests can be configured to execute synchronously before switching to asynchronous mode. The requests not finished synchronously by the client provided threshold are handed off to a separate thread for posting the results once complete. The default value for asyncAfterSeconds is 10 seconds. Setting asyncAfterSeconds to 0 will execute the request in asynchronous mode upon submission.

Background Cleaner

Each Elide instance will also run a scheduler for maintenance and cleanup.

  • It cleans up requests and results stored in the persistent store.
  • It scans and flags the queries that failed due to an application crash/reboot automatically as TIMEDOUT.
  • It is responsible for the graceful cancellation of async requests.
    • The user can update the status of the async requests to CANCELLED. The cleaner polls the AsyncQuery and TableExport models to find any new requests that were CANCELLED and tries to terminate the transaction associated with that execution and change the status to CANCEL_COMPLETE.
    • Long-running transactions that exceed the run time threshold are terminated. Any background thread executing the original request is interrupted.
  • The retention, polling interval, and max run time thresholds can be configured during application startup.
  • This scheduler can be disabled by setting cleanupEnabled to false as mentioned here.

TableExport Results Download

Elide has built-in support for streaming the results of a TableExport request through the export endpoint. Upon successful completion, the TableExport model includes a separate URL attribute where results can be downloaded from.

Enabling the end-point, timeouts, path, download attachment extensions, etc. can be configured during application startup as mentioned here.

Supported Query Types

Below are the supported values for query type in asynchronous calls:

  • GRAPHQL_V1_0
  • JSONAPI_V1_0

Supported Result Types

Elide can transform the results into a pre-selected format while persisting them via the ResultStorageEngine. Below are the supported formats for Table Export results:

  • JSON
  • CSV

Query Status

Below are the different states of an asynchronous request:

Status Description
QUEUED Request is submitted and waiting to be picked up for execution.
PROCESSING Request has been picked up for execution.
COMPLETE Request has completed.
CANCELLED The client has requested to cancel a running request.
TIMEDOUT Request did not finish within the configured maximum run time.
FAILURE Request not completed due to one or more failures encountered by the scheduler.
CANCEL_COMPLETE Request has been canceled by the background cleaner.

Malformed or invalid queries provided in the Async request will finish with COMPLETE status and the actual error message will be available in the result property of AsyncQuery and TableExport models.

Security

The Async API models (AsyncQuery and TableExport) have a simple permission model: Only the principal who submitted a query and principals which belong to an administrative role are allowed to retrieve its status or results. Principals can be assigned roles when constructing the Elide user object.

Enable the Async API

By default the async API is disabled. The elide models (AsyncQuery and TableExport) needed to support the Async API are JPA models that are mapped to a specific database schema. This schema must be created in your target database. Feel free to modify the query/result column sizes per your needs.

Configuration Description Default
enabled Enable the Async API feature. false
cleanupEnabled Enable cleaning up of Async API requests history, update the status of interrupted/timedout requests, and cancel requests. false


Update your application.yaml with the additional code below. (Only the relevant portions are included.).

If you rely on Spring to autodiscover the entities which are placed in the same package/sub-package as the application class with @SpringBootApplication annotation, you will have to add the @EntityScan annotation to that application class for those entities to be discovered after async is enabled.

elide:
  async:
    enabled: true
    cleanupEnabled: true


Update your implementation of ElideStandaloneSettings interface with the below mentioned additional code. (Only the relevant portions are included.)

@Override
public ElideStandaloneAsyncSettings getAsyncProperties() {
    ElideStandaloneAsyncSettings asyncProperties = new ElideStandaloneAsyncSettings() {
        @Override
        public boolean enabled() {
            return true;
        }

        @Override
        public boolean enableCleanup() {
            return true;
        }
    }
    return asyncProperties;
}

Additional Configuration

These additional configuration settings control timeouts, cleanup, export end-point, resultStorageEngine and the sizes of thread pools.

Configuration Description Default
threadPoolSize Number of requests to run in parallel. 5
maxRunTimeSeconds Maximum query run time for requests before TIMEDOUT. 3600
maxAsyncAfterSeconds Maximum permissible value for asyncAfterSeconds. 10
queryCleanupDays Number of days to retain request executions and result history. 7
queryCancellationIntervalSeconds A background cleaner is responsible for canceling the transactions for requests marked as CANCELLED and changing the status to CANCEL_COMPLETE. Polling interval to identify CANCELLED requests and update status. 300
enableExport Enable the TableExport feature i.e. request submission, download. false
exportApiPathSpec API root path specification for the export end-point. /export
skipCSVHeader Skip Header Record when exporting as CSV. false
storageDestination Location to persist export results through the default ResultStorageEngine. /tmp
extensionEnabled Attachment to have a file name extension or not. Supported extensions CSV (.csv), JSON (.json). false

These additional configuration settings are only applicable for Elide’s Standalone module. When using Spring, please configure the TaskExecutor used by Spring MVC for executing and managing the asynchronous requests.

Configuration Description Default
exportAsyncResponseTimeoutSeconds Default timeout for TableExport’s result download end-point. 30
exportAsyncResponseExecutor Executor for executing TableExport’s result download request asynchronously. A java.util.concurrent.ExecutorService instance 
elide:
  async:
    threadPoolSize: 10
    maxRunTimeMinutes: 120
    queryCleanupDays: 10
    queryCancellationIntervalSeconds: 600
    maxAsyncAfterSeconds: 30
    export:
      enabled: true
      path: /export
      skipCSVHeader: false
      storageDestination: /tmp

@Override
public ElideStandaloneAsyncSettings getAsyncProperties() {
    ElideStandaloneAsyncSettings asyncProperties = new ElideStandaloneAsyncSettings() {
        @Override
        default Integer getThreadSize() {
            return 10;
        }

        @Override
        default Integer getMaxRunTimeSeconds() {
            return 120;
        }

        @Override
        default Integer getQueryCleanupDays() {
            return 10;
        }

        @Override
        default Integer getQueryCancelCheckIntervalSeconds() {
            return 600;
        }

        @Override
        default Integer getMaxAsyncAfterSeconds() {
            return 30;
        }

        @Override
        default String getExportApiPathSpec() {
            return "/export/*";
        }

        @Override
        default boolean enableExport() {
            return false;
        }

        @Override
        default boolean skipCSVHeader() {
            return false;
        }

        @Override
        default String getStorageDestination() {
            return "/tmp";
        }

        @Override
        default Integer getExportAsyncResponseTimeoutSeconds() {
            return 30;
        }

        @Override
        default ExecutorService getExportAsyncResponseExecutor() {
            return enableExport() ? Executors.newFixedThreadPool(getThreadSize() == null ? 6 : getThreadSize()) : null;
        }
    }
    return asyncProperties;
}

Running

After configuring and starting your service, the following commands illustrate how to make asynchronous requests. Don’t forget to replace localhost:8080 with your URL. The example below makes use of the models and sample data that the liquibase migrations added through our example is available here.

Submitting query

curl -X POST http://localhost:8080/api/v1/asyncQuery/ \
    -H"Content-Type: application/vnd.api+json" -H"Accept: application/vnd.api+json" \
    -d'{
            "data": {
                       "type": "asyncQuery",
                       "id": "ba31ca4e-ed8f-4be0-a0f3-12088fa9263d",
                       "attributes": {
                                       "query": "/group?sort=commonName&fields%5Bgroup%5D=commonName,description",
                                       "queryType": "JSONAPI_V1_0",
                                       "status": "QUEUED"
                                     }
                    }
        }'

curl -g -X POST -H"Content-Type: application/json" \
    -H"Accept: application/json" "http://localhost:8080/graphql/api/v1" \
    -d'{
            "query" : "mutation { asyncQuery(op: UPSERT, data: {id: \"bb31ca4e-ed8f-4be0-a0f3-12088fb9263e\", query: \"{\\\"query\\\":\\\"{ group { edges { node { name } } } }\\\",\\\"variables\\\":null}\", queryType: GRAPHQL_V1_0, status: QUEUED}) { edges { node { id query queryType status result {completedOn responseBody contentLength httpStatus recordCount} } } } }"
       }'

curl -X POST http://localhost:8080/api/v1/tableExport/ \
    -H"Content-Type: application/vnd.api+json" -H"Accept: application/vnd.api+json" \
    -d'{
            "data": {
                       "type": "tableExport",
                       "id": "ba31ca4e-ed8f-4be0-a0f3-12088fa9263f",
                       "attributes": {
                                       "query": "/group?sort=commonName&fields%5Bgroup%5D=commonName,description",
                                       "queryType": "JSONAPI_V1_0",
                                       "status": "QUEUED",
                                       "resultType": "CSV"
                                     }
                    }
        }'

curl -g -X POST -H"Content-Type: application/json" \
    -H"Accept: application/json" "http://localhost:8080/graphql/api/v1" \
    -d'{
            "query" : "mutation { tableExport(op: UPSERT, data: {id: \"bb31ca4e-ed8f-4be0-a0f3-12088fb9263g\", query: \"{\\\"query\\\":\\\"{ group { edges { node { name } } } }\\\",\\\"variables\\\":null}\", queryType: GRAPHQL_V1_0, resultType: CSV, status: QUEUED}) { edges { node { id query queryType resultType status result {completedOn url message httpStatus recordCount} } } } }"
       }'

Here are the respective responses:

{
  "data": {
      "type": "asyncQuery",
      "id": "ba31ca4e-ed8f-4be0-a0f3-12088fa9263d",
      "attributes": {
          "asyncAfterSeconds": 10,
          "principalName": null,
          "createdOn": "2020-04-08T23:29Z",
          "query": "/group?sort=commonName&fields%5Bgroup%5D=commonName,description",
          "queryType": "JSONAPI_V1_0",
          "status": "COMPLETE",
          "updatedOn": "2020-04-08T23:29Z",
          "result": {
              "recordCount": 2,
              "httpStatus": 200,
              "completedOn": "2020-04-08T23:29Z",
              "contentLength": 282,
              "responseBody": "{\"data\":[{\"type\":\"group\",\"id\":\"com.yahoo.elide\",\"attributes\":{\"commonName\":\"Elide\",\"description\":\"The magical library powering this project\"}},{\"type\":\"group\",\"id\":\"com.example.repository\",\"attributes\":{\"commonName\":\"Example Repository\",\"description\":\"The code for this project\"}}]}"
          }
      }
  }
}

{
  "data": {
      "asyncQuery": {
          "edges": [{
              "node": {
                  "id": "bb31ca4e-ed8f-4be0-a0f3-12088fb9263e",
                  "query": "{\"query\":\"{ group { edges { node { name } } } }\",\"variables\":null}",
                  "queryType": "GRAPHQL_V1_0",
                  "status": "COMPLETE",
                  "result" : {
                      "completedOn": "2020-04-08T21:25Z",
                      "responseBody": "{\"data\":{\"group\":{\"edges\":[{\"node\":{\"name\":\"com.example.repository\"}},{\"node\":{\"name\":\"com.yahoo.elide\"}},{\"node\":{\"name\":\"elide-demo\"}}]}}}",
                      "contentLength": 109,
                      "httpStatus": 200,
                      "recordCount": 2
                  }
              }
          }]
      }
  }
}

{
  "data": {
      "type": "tableExport",
      "id": "ba31ca4e-ed8f-4be0-a0f3-12088fa9263f",
      "attributes": {
          "asyncAfterSeconds": 10,
          "principalName": null,
          "createdOn": "2020-04-08T23:29Z",
          "query": "/group?sort=commonName&fields%5Bgroup%5D=commonName,description",
          "queryType": "JSONAPI_V1_0",
          "resultType": "CSV",
          "status": "COMPLETE",
          "updatedOn": "2020-04-08T23:29Z",
          "result": {
              "recordCount": 2,
              "httpStatus": 200,
              "completedOn": "2020-04-08T23:29Z",
              "url": "http://localhost:8080/export/ba31ca4e-ed8f-4be0-a0f3-12088fa9263f",
              "message": null
          }
      }
  }
}

{
  "data": {
      "asyncQuery": {
          "edges": [{
              "node": {
                  "id": "bb31ca4e-ed8f-4be0-a0f3-12088fb9263g",
                  "query": "{\"query\":\"{ group { edges { node { name } } } }\",\"variables\":null}",
                  "queryType": "GRAPHQL_V1_0",
                  "resultType": "CSV",
                  "status": "COMPLETE",
                  "result": {
                      "completedOn": "2020-04-08T21:25Z",
                      "url": "http://localhost:8080/export/bb31ca4e-ed8f-4be0-a0f3-12088fb9263g",
                      "message": null,
                      "httpStatus": 200,
                      "recordCount": 2
                  }
              }
          }]
      }
  }
}

Retrieving status and result

Long running queries in the QUEUED or PROCESSING state may not return with the result property populated in the responses above. The client can poll the AsyncQuery and TableExport objects asynchronously for status updates.

curl -X GET http://localhost:8080/api/v1/asyncQuery/ba31ca4e-ed8f-4be0-a0f3-12088fa9263d \
    -H"Content-Type: application/vnd.api+json" -H"Accept: application/vnd.api+json"

curl -g -X POST -H"Content-Type: application/json" -H"Accept: application/json" \
    "http://localhost:8080/graphql/api/v1" \
    -d'{
           "query" : "{ asyncQuery (ids: \"bb31ca4e-ed8f-4be0-a0f3-12088fb9263e\") { edges { node { id query queryType status result {completedOn responseBody contentLength httpStatus recordCount}} } } }"
      }'

curl -X GET http://localhost:8080/api/v1/tableExport/ba31ca4e-ed8f-4be0-a0f3-12088fa9263f \
    -H"Content-Type: application/vnd.api+json" -H"Accept: application/vnd.api+json"

curl -g -X POST -H"Content-Type: application/json" -H"Accept: application/json" \
    "http://localhost:8080/graphql/api/v1" \
    -d'{
           "query" : "{ tableExport (ids: \"bb31ca4e-ed8f-4be0-a0f3-12088fb9263g\") { edges { node { id query queryType resultType status result {completedOn url message httpStatus recordCount}} } } }"
      }'

Here are the respective responses:

{
  "data": {
      "type": "asyncQuery",
      "id": "ba31ca4e-ed8f-4be0-a0f3-12088fa9263d",
      "attributes": {
          "asyncAfterSeconds": 10,
          "principalName": null,
          "createdOn": "2020-04-08T21:25Z",
          "query": "/group?sort=commonName&fields%5Bgroup%5D=commonName,description",
          "queryType": "JSONAPI_V1_0",
          "status": "COMPLETE",
          "updatedOn": "2020-04-08T21:25Z",
          "result": {
              "recordCount": 2,
              "httpStatus": 200,
              "completedOn": "2020-04-08T23:29Z",
              "contentLength": 282,
              "responseBody": "{\"data\":[{\"type\":\"group\",\"id\":\"com.yahoo.elide\",\"attributes\":{\"commonName\":\"Elide\",\"description\":\"The magical library powering this project\"}},{\"type\":\"group\",\"id\":\"com.example.repository\",\"attributes\":{\"commonName\":\"Example Repository\",\"description\":\"The code for this project\"}}]}"
          }
      }
  }
}

{
  "data": {
      "asyncQuery": {
          "edges": [{
              "node": {
                  "id": "bb31ca4e-ed8f-4be0-a0f3-12088fb9263e",
                  "query": "{\"query\":\"{ group { edges { node { name } } } }\",\"variables\":null}",
                  "queryType": "GRAPHQL_V1_0",
                  "status": "COMPLETE",
                  "result" : {
                      "completedOn": "2020-04-08T21:25Z",
                      "responseBody": "{\"data\":{\"group\":{\"edges\":[{\"node\":{\"name\":\"com.example.repository\"}},{\"node\":{\"name\":\"com.yahoo.elide\"}},{\"node\":{\"name\":\"elide-demo\"}}]}}}",
                      "contentLength": 109,
                      "httpStatus": 200,
                      "recordCount": 2
                  }
              }
          }]
      }
  }
}

{
  "data": {
      "type": "tableExport",
      "id": "ba31ca4e-ed8f-4be0-a0f3-12088fa9263f",
      "attributes": {
          "asyncAfterSeconds": 10,
          "principalName": null,
          "createdOn": "2020-04-08T21:25Z",
          "query": "/group?sort=commonName&fields%5Bgroup%5D=commonName,description",
          "queryType": "JSONAPI_V1_0",
          "resultType": "CSV",
          "status": "COMPLETE",
          "updatedOn": "2020-04-08T21:25Z",
          "result": {
              "recordCount": 2,
              "httpStatus": 200,
              "completedOn": "2020-04-08T23:29Z",
              "url": "http://localhost:8080/export/ba31ca4e-ed8f-4be0-a0f3-12088fa9263f",
              "message": null
          }
      }
  }
}

{
  "data": {
      "asyncQuery": {
          "edges": [{
              "node": {
                  "id": "bb31ca4e-ed8f-4be0-a0f3-12088fb9263e",
                  "query": "{\"query\":\"{ group { edges { node { name } } } }\",\"variables\":null}",
                  "queryType": "GRAPHQL_V1_0",
                  "resultType":"CSV",
                  "status": "COMPLETE",
                  "result" : {
                      "completedOn": "2020-04-08T21:25Z",
                      "url": "http://localhost:8080/export/bb31ca4e-ed8f-4be0-a0f3-12088fb9263g",
                      "message": null,
                      "httpStatus": 200,
                      "recordCount": 2
                  }
              }
          }]
      }
  }
}

Downloading the TableExport results

The TableExport request will return a URL to download the results as shown in the example response below.

{
  "result" : {
      "completedOn": "2020-04-08T21:25Z",
      "url": "http://localhost:8080/export/bb31ca4e-ed8f-4be0-a0f3-12088fb9263g",
       "message": null,
       "httpStatus": 200,
       "recordCount": 2
  }
}

[
  {"commonName":"Elide","description":"The magical library powering this project"}
  ,{"commonName":"Example Repository","description":"The code for this project"}
]

"commonName", "description"
"Elide", "The magical library powering this project"
"Example Repository", "The code for this project"

Overriding the AsyncAPIDAO

The Async API interacts with the persistence layer through an abstraction - the AsyncAPIDAO, for status updates, query cleanup, etc. This can be customized by providing your own implementation. Elide provides a default implementation of AsyncAPIDAO.


Update your application.yaml with the additional code below. (Only the relevant portions are included.). You will have to initialize your implementation as a Bean.

elide:
  async:
    defaultAsyncAPIDAO: false

  /**
   * Configure the AsyncAPIDAO used by async requests.
   * @return an AsyncAPIDAO object.
   */
  @Bean
  public AsyncAPIDAO buildAsyncAPIDAO() {
      return YourDAOImplementationObject;
  }

@Override
public ElideStandaloneAsyncSettings getAsyncProperties() {
    ElideStandaloneAsyncSettings asyncProperties = new ElideStandaloneAsyncSettings() {
        /**
         * Implementation of AsyncAPIDAO to use.
         * @return AsyncAPIDAO type object.
         */
        @Override
        default AsyncAPIDAO getAPIDAO() {
            return yourDAOImplementationObject;
        }
    }
    return asyncProperties;
}

Overriding the ResultStorageEngine

Table exports leverage a reactive abstraction (ResultStorageEngine) for streaming results to and from a persistence backend. This can be customized by providing your own implementation. Elide provides default implementation of ResultStorageEngine.


Update your application.yaml with the additional code below. (Only the relevant portions are included.). You will have to initialize your implementation as a Bean.

elide:
  async:
    defaultResultStorageEngine: false

  /**
   * Configure the ResultStorageEngine used by TableExport requests.
   * @return a ResultStorageEngine object.
   */
  @Bean
  public ResultStorageEngine buildResultStorageEngine() {
      return yourResultStorageEngineImplementationObject;
  }

@Override
public ElideStandaloneAsyncSettings getAsyncProperties() {
    ElideStandaloneAsyncSettings asyncProperties = new ElideStandaloneAsyncSettings() {
        /**
         * Implementation of ResultStorageEngine to use.
         * @return ResultStorageEngine type object.
         */
        @Override
        default ResultStorageEngine getResultStorageEngine() {
            return yourResultStorageEngineImplementationObject;
        }
    }
    return asyncProperties;
}