Specifying Hard Throttling Limits for APIs

WSO2 API Manager 1.10 allows you to specify a limit on the number of requests a particular API can make on a backend.

With WSO2 API Manager requests can be throttled at three different levels.

  1. Throttle Requests by API Context.
  2. Throttle Requests by Application.
  3. Throttle by API Resource.

At the time of creating an API, Throttling policies available for the API and policies for each resource can be specified. With each new token generated for the API, the requester gets a quota specified by the Tier. For example, if the policy for the context allows 50 Requests per minute, then each new user would be able to make 50 calls on that API within a minute. Policy defined for the resource level behaves in a similar way. Throttling at the Application level ensures, that the total number of requests made by all the APIs coming under the Application doesn’t exceed the limit specified in the Application level policy.

All the existing throttling limits specify the Quota the invoker would get, not the limit Backend cans handle. Even after applying all these throttling mechanisms, it’s still possible to overuse the backend, since there isn’t a single place to specify, the total number of calls the API Manager would be allowed to make on the Backend. The latest API Manager release, addresses this issue by providing the capability to specify a Hard Throttling limit.

Hard limit is the total number of requests that are allowed to be made on that API. This feature can be used to specify the number of requests API Manager can make on a  particular Backend.

How to enable

Once you reach Manage Page while creating an API, you’ll see an option to enable Hard Throttling limits.

EnableHardLimitThrottle

Enabling Hard Throttling Limits

Specify limits of the Production & Sandbox endpoints. Since the two endpoints can come from two servers with different capacities, option is provided to specify different throttling limits.

Save and Publish the API. If you take a look at the synapse config, you might notice that now under APIThrottleHandler, several additional properties are defined.

 <handlers>
      <handler class="org.wso2.carbon.apimgt.gateway.handlers.security.CORSRequestHandler">
         <property name="apiImplementationType" value="ENDPOINT"/>
      </handler>
      <handler class="org.wso2.carbon.apimgt.gateway.handlers.security.APIAuthenticationHandler"/>
      <handler class="org.wso2.carbon.apimgt.gateway.handlers.throttling.APIThrottleHandler">
         <property name="id" value="A"/>
         <property name="productionMaxCount" value="800"/>
         <property name="sandboxMaxCount" value="500"/>
         <property name="policyKey" value="gov:/apimgt/applicationdata/tiers.xml"/>
      </handler>
      <handler class="org.wso2.carbon.apimgt.usage.publisher.APIMgtUsageHandler"/>

There are two properties to keep Production TPS and Sandbox TPS.

Changing Unit time

Normally Hard limits are counted over a duration of 1 second (Since it’s a TPS we are specifying). But needs may arise to apply Throttling over a larger time window – like over a 1 minute. Time window can be defined using the properties productionUnitTime and sandboxUnitTime.

 <handlers>
      <handler class="org.wso2.carbon.apimgt.gateway.handlers.security.CORSRequestHandler">
         <property name="apiImplementationType" value="ENDPOINT"/>
      </handler>
      <handler class="org.wso2.carbon.apimgt.gateway.handlers.security.APIAuthenticationHandler"/>
      <handler class="org.wso2.carbon.apimgt.gateway.handlers.throttling.APIThrottleHandler">
         <property name="id" value="A"/>
         <property name="productionMaxCount" value="600"/>
         <property name="productionUnitTime" value="60000"/>
         <property name="policyKey" value="gov:/apimgt/applicationdata/tiers.xml"/>
      </handler>
      <handler class="org.wso2.carbon.apimgt.usage.publisher.APIMgtUsageHandler"/>

This configuration will allow 600 requests, within a duration of 1 minute. Unit time can only be changed by directly editing the synapse config.

Testing

1. Create an API giving the Production Limit as 5. While selecting Throttling tiers, make sure you have selected Bronze and Silver plans as well.

2. For the purpose of testing, let’s edit the synapse config and change the unit Time as 1 minute.

 <handlers>
      <handler class="org.wso2.carbon.apimgt.gateway.handlers.security.CORSRequestHandler">
         <property name="apiImplementationType" value="ENDPOINT"/>
      </handler>
      <handler class="org.wso2.carbon.apimgt.gateway.handlers.security.APIAuthenticationHandler"/>
      <handler class="org.wso2.carbon.apimgt.gateway.handlers.throttling.APIThrottleHandler">
         <property name="id" value="A"/>
         <property name="productionMaxCount" value="5"/>
         <property name="productionUnitTime" value="60000"/>
         <property name="policyKey" value="gov:/apimgt/applicationdata/tiers.xml"/>
      </handler>
      <handler class="org.wso2.carbon.apimgt.usage.publisher.APIMgtUsageHandler"/>

3. Log into Store, create an Application and then Subscribe to the API. While subscribing make sure that you select a Tier that allows more than 5 reqeusts/min (Silver allows 10).

4. Generate the Keys, get the access token and invoke the API for more than 5 times. If you are using a curl command, make sure you have put -v option too, since it shows the request and response headers.

Once you exceed the limit, you’ll get an error like this;

< HTTP/1.1 503 Service Unavailable
< Access-Control-Allow-Headers: authorization,Access-Control-Allow-Origin,Content-Type
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Methods: GET
< Content-Type: application/xml; charset=UTF-8
< Accept: */*
< Date: Sun, 01 Nov 2015 00:00:48 GMT
< Transfer-Encoding: chunked
< Connection: Close
<
* Closing connection 0
<amt:fault xmlns:amt="http://wso2.org/apimanager/throttling"><amt:code>900801</amt:code><amt:message>API Limit Reached</amt:message><amt:description>API not accepting requests</amt:description></amt:fault>

Note that the response gives 503 Service Unavailable and a response message saying API is not accepting requests anymore.

5. To see how this differs from Soft Throttling, delete the subscription and re-subscribe the API with Bronze tier. This time when throttling limit exceeds following response will be given instead;

< HTTP/1.1 429
< Access-Control-Allow-Headers: authorization,Access-Control-Allow-Origin,Content-Type
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Methods: GET
< Content-Type: application/xml; charset=UTF-8
< Accept: */*
< Date: Sun, 01 Nov 2015 00:02:23 GMT
< Transfer-Encoding: chunked
<
* Connection #0 to host 127.0.0.1 left intact
<amt:fault xmlns:amt="http://wso2.org/apimanager/throttling"><amt:code>900800</amt:code><amt:message>Message throttled out</amt:message><amt:description>You have exceeded your quota</amt:description></amt:fault>
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s