While developing and deploying Function as a service(Faas) applications leveraging AWS Lambda ,API Gateways etc. Serverless frameworks promised to keep development and deployments simple and enabling public cloud of choice.
Serverless Enterprise/Pro Edition which offers monitoring,alerts etc to serverless applications resulted in huge logging for each function invocations e.g.
SERVERLESS_ENTERPRISE
{
"type": "transaction",
"origin": "sls-agent",
"payload": {
"duration": 576.2143135070801,
"endTime": "2020-02-15T05:18:46.646536Z",
"logs": {},
"operationName": "s-transaction-function",
"schemaType": "s-span",
"schemaVersion": "0.0",
"spanContext": {
"spanId": "1c28085b-efd5-4591-865a-e38ef0bda0e7",
"traceId": "0874e9f5-ed95-4e3c-aa37-d16a54f3602c",
"xTraceId": "Root=1-5e477f35-53f776209c7518c6e40ea991;Parent=5412c6a632e94e9d;Sampled=0"
},
.... tens of such lines for each function invocation
There are no documented ways to disable serverless enterprise logging and reporting, A simple Programming mistake which triggered hundreds of function calls to Lambda resulted in >80 GB Cloud watch logs, and more than 2GB data transfer out to serverless.com, which exceeded AWS budgets by many folds.
Steps to Control serverless logging and bill shocks in AWS :
- Simple step to disable serverless enterprise logging is - logout from serverless using cli
sls logout
and commentapp
andorg
labels inservererless.yml
- Disable unwanted logging/warnings generated by functions, Serverless Enterprise framework with Python3.8 caused multiple warnings, migrating it back to python3.6 resolved it and saved logging.
- Implement Guard Rails for number of AWS lambda invocations using ReservedConcurrency to reasonable limits (default account concurrency limit which allows upto 1000 lamnba functions to run in parallel)
- Implement Rate limiter for AWS API gateway end points to be within reasonable/expected limits.
Comments