AWS – Using IOPipe to instrument your lambda functions

IOPipe is a multi-tenant SaaS application that hooks into your Lambda functions via a function wrapper that you apply to your event handlers. It records detailed trace data from your function invocations so that you can profile your code as it runs. It helps developers while they are debugging their code, and it also helps operations spot problems quickly. Each time you make an AWS API call from within your code, the performance of that call is recorded so that you can see exactly where excess time is being spent. For example, you can see the average time it takes to execute a query of your DynamoDB table, or how long it takes to put an object into an S3 bucket.

How to do it…

In this recipe, you will create an IOPipe trial account, author a new lambda function, and observe performance information for that function as it runs. Let’s get started:

  1. Go to the IOPipe site at https://www.iopipe.com/ and click Get Started.
  2. Fill out the user profile and create your trial account.
  1. On the following screen, enter a default team name and then click New Project. Enter a name for your project:

IOPipe setup
  1. Under Plans, select Lite, which is free of charge. You don’t have to enter credit card details for the lite plan. Click Create Team.
  2. On the following screen, you will see your security token and links to tutorials for various programming languages.
  1. Log in to your AWS account and go to the CodeStar dashboard. CodeStar is an AWS service that allows you to create a complete serverless application, including a code repository in CodeCommit, a lambda function, a CodeBuild project, and a CodePipeline configuration so that you can automatically deploy your code after each change. We are using CodeStar for this recipe because we need to be able to install a Node.js package, which is not currently possible via the lambda console. Also, it’s good practice to avoid directly editing lambda functions in the console, and CodeStar gives us a Cloud9 web-based Integrated Development Environment (IDE) for this:

Starting a new project with AWS CodeStar
  1. Click Start a project. You will be presented with a dialog asking you if you wish to create the CodeStar service role. Click Yes, create role:

Creating the CodeStar service role
  1. There are a variety of project templates to choose from with CodeStar. For this recipe, select the Node.js lambda web application:

CodeStar project templates
  1. Give the project a unique name and click Next. Then, click Create project.
  1. When you are given a choice of code editors, select AWS Cloud9:

CodeStar editor selection
  1. Go with the default configuration for Cloud9 and click Next.
  2. It will take a few minutes for CodeStar to complete the setup process, but, once it’s done, you’ll see a screen similar to the following:

AWS CodeStar dashboard
  1. Once the setup is complete, select the IDE from the left-hand side menu to open your Cloud9 environment, as follows:

AWS Cloud9
  1. Expand the project folder and open both index.js and template.yml. The YAML file is your Serverless Application Model ( SAM) template that defines your resources, and the JavaScript file is your lambda function.
  2. In the Terminal window below the editor pane, install the @iopipe/iopipe NPM package using the npm install --save command, as follows:
ezb.packt.admin1:~/environment $ cd packt-iopipe/
ezb.packt.admin1:~/environment/packt-iopipe (master) $ npm install --save @iopipe/iopipe
npm notice created a lockfile as package-lock.json. You should commit this file.
+ @iopipe/iopipe@1.14.0
added 85 packages from 59 contributors and audited 245 packages in 7.321s
found 0 vulnerabilities
  1. In the template.yml file, define an environment variable to hold your IOPipe token, as follows:
    Type: AWS::Serverless::Function
      Handler: index.get
      Runtime: nodejs8.10
        - LambdaExecutionRole
        - Arn
      Events: [...]
  1. Also in the template.yml file, change the deployment type to be AllAtOnce, which will speed up the deployment for testing purposes:
    AutoPublishAlias: live
      Enabled: true
      Type: AllAtOnce
      Role: !Ref CodeDeployRole
  1. To make a change to the deployment preference, you will need to modify the policy attached to the CodeStar CloudFormation role. Go to the IAM dashboard and search for a role named CodeStarWorker-[PROJECT NAME]-CloudFormation.
  1. Edit the inline policy in that role to include codedeploy:UpdateDeploymentGroup, as follows:

Editing the CodeStar CloudFormation policy to allow changes to deployment preferences
  1. Modify the lambda function in index.js so that it uses the IOPipe wrapper:
'use strict';

var fs = require('fs');
var path = require('path');
var iopipe = require('@iopipe/iopipe')();

exports.get = iopipe(function(event, context, callback) {
  var contents = fs.readFileSync(`public${path.sep}index.html`);
  var result = {
    statusCode: 200,
    body: contents.toString(),
    headers: {'content-type': 'text/html'}
  callback(null, result);
  1. Save index.js and template.yml.
  2. Create a .gitignore file and add node_modules to it, since you don’t want to commit all of the NPM packages that have been installed. Then, commit your changes to the CodeCommit repository, as follows:
ezb.packt.admin1:~/environment/packt-iopipe (master) $ git add .
ezb.packt.admin1:~/environment/packt-iopipe (master) $ git commit -m "IOPipe"
ezb.packt.admin1:~/environment/packt-iopipe (master) $ git push
  1. CodeStar will detect the change to the repository and kick off the CodePipeline project that was automatically set up for you. Go back to the CodeStar dashboard to watch the progress of your build:

CodeStar deployment progress
  1. Once the deployment progress has completed, scroll up to application endpoints and click the URL for your application to test it:

CodeStar automatically deploys your application to a production endpoint after each push
  1. Refresh that page a few times so that we have a few lambda invocations to track using IOPipe.
  2. Go back to the IOPipe dashboard and look at the invocations:

IOPipe dashboard: The text and numbers in this image are intentionally illegible
  1. Click the name of the function at the bottom of the screen to see details about the function:

IOPipe function details: The text and numbers in this image are intentionally illegible

You now have experience with rapidly creating a new web application with AWS CodeStar and wiring up detailed tracing to your Lambda function with IOPipe. Take some time to explore the functionality offered by IOPipe, and then, if you no longer need this CodeStar project, be sure to delete it so that you do not incur any future charges on your AWS account.

How it works…

The wrapper you put around your Lambda event handler instruments your code so that asynchronous messages are sent to IOPipe each time your code makes an action. This data is recorded and cataloged to enable a variety of reports and alerts. With Node.js, you can review the IOPipe package located at https://www.npmjs.com/package/@iopipe/iopipe. The source code can be found in this book’s GitHub repository at https://github.com/iopipe/iopipe-js.

As an Advanced AWS Technology Partner, and as a DevOps competency holder, IOPipe’s architecture, which naturally runs on AWS, has been audited for best practices around security and reliability, so you can rest assured that the data being emitted by your lambda functions is safe and secure.

We will study some more functions in the following subsections:

  • Metrics dashboards
  • Alerting
  • Profiling
  • Labels and search

Metrics dashboards

A real-time view into the number of times a certain function has been invoked, along with the average duration of those function calls, can be a huge help when troubleshooting code that is misbehaving, and it can also be used by operations to quickly spot outlier behavior. 


IOPipe has a flexible alert system that allows you to configure rules and integrations with systems such as Slack and PagerDuty. You can be alerted any time a lambda function causes an error, or when the average invocation time of a function exceeds a certain threshold.


If you are using Node.js, which is one of the most popular languages available for Lambda, you can do CPU profiling of a running function. This is a detailed profile of the performance characteristics of each individual statement in your code, which means you will spend a lot less time adding log statements to the code to see what’s going on. You can compare profiles after revisions to see the effect of your latest changes.

Labels and search

If you have a large number of complex functions, which is inevitable if you have converted to a completely server-less application architecture, then you know that it can be very difficult to sort through your logs when something goes wrong. With IOPipe, it’s possible to add labels to your code so that you can easily categorize various segments across different lambda functions, and a flexible search engine lets you dig through extensive log files to find exactly what you are looking for.

There’s more…

IOPipe will also send you a weekly email report that details the performance of your functions. This email points out changes in behavior for your functions, common errors that are thrown by your code, and it also summarizes the alerts that were sent based on rules that you have configured.

Comments are closed.