Google Cloud Platform – Stackdriver monitoring and logging

How to Install Windows Server 2019

Stackdriver monitoring provides you the ability to monitor your applications deployed on GCP. Stackdriver also allows you to monitor your applications deployed in AWS as well. You can monitor your applications for performance and uptime, and all these metrics and events are collected and stored by Stackdriver. Stackdriver monitoring also generates dashboards for easy visual consumption of the data. The feature also integrates with alerting so timely alerts can be sent out for any event. Remember that monitoring and logging are closely integrated.

A better way to explore Stackdriver monitoring is to set it up for a compute engine instance. We will discuss its concepts as we go through the setup.

I will be following Google’s official documentation example here, so it will be easier for you to relate what we do here with the official guide.

Before we begin, let’s review all the steps in this exercise:

  1. Create a compute engine instance
  2. Install an Apache HTTP server
  3. Create a Stackdriver account and install agents
  4. Create checks and policies
  5. Create dashboards and alerts
  6. Review logs

As with every other exercise we have done before, let’s begin by creating a project first. If you don’t already have one, go ahead and create a project called stackdriver-test-project:

$ gcloud projects create stackdriver-test-123 --name stackdriver-test-project 
Create in progress for [https://cloudresourcemanager.googleapis.com/v1/projects/stackdriver-test-123]. 
Waiting for [operations/cp.7718643467432378132] to finish...done.  
$ gcloud projects list 
PROJECT_ID              NAME                      PROJECT_NUMBER 
kubecluster-206421      KubeCluster               944435304139 
premium-bearing-206421  My First Project          766565060122 
stackdriver-test-123    stackdriver-test-project  968775468325 
Project IDs and project numbers are unique. Your IDs and numbers will differ from what you see here.

Now that we have the project created, let’s enable billing for it. Before we do that, let’s make sure that we set our project to stackdriver-test-project:

$ gcloud config list 
account = hidden@gmail.com 
disable_usage_reporting = True 
project = kubecluster-206421 
$ gcloud config unset project 
Unset property [core/project]. 
$ gcloud config list 
account = hidden@gmail.com 
disable_usage_reporting = True 
Your active configuration is: [abcd] 
$ gcloud config set project stackdriver-test-123 
Updated property [core/project]. 
$ gcloud config list 
account = hidden@gmail.com 
disable_usage_reporting = True 
project = stackdriver-test-123 
Your active configuration is: [abcd] 
$ gcloud config list 
account = hidden@gmail.com 
disable_usage_reporting = True 
project = stackdriver-test-123 
Your active configuration is: [abcd]  

If you are not in the gcloud command line yet, this can be done easily using the GCP console:

Let’s enable billing for this project:

$ gcloud alpha billing projects link stackdriver-test-123 --billing-account 01617F-ABCSDE-AA20F0 
billingAccountName: billingAccounts/01617F-B8843D-AA20F0 
billingEnabled: true 
name: projects/stackdriver-test-123/billingInfo 
projectId: stackdriver-test-123 

We can now create our compute engine instance in our new project. We will use the console going forward to keep things simple. If gcloud seems exciting, hang in there as we visit gcloud in Chapter 9, Google Cloud Platform Tools:

  1. Go to your main side tab | Compute Engine | VM Instances.
  2. Click Create after initialization is complete.
  3. We will call our machine small-http-stack and will create it with the minimum requirements. Choose to Allow HTTP traffic and Allow HTTPS traffic:
  1. Click Create when done. Once deployed, click on the Connect option and open SSH in your preferred method:

Once in, run the following commands:

~$ sudo apt-get install apache2 php7.0

This installs an apache2 http server and a PHP on the machine. Let’s make sure the service is up and running:

~$ sudo /etc/init.d/apache2 status
● apache2.service - The Apache HTTP Server
Loaded: loaded (/lib/systemd/system/apache2.service; enabled; vendor preset: enabled)
Active: active (running) since Sun 2018-06-17 19:33:58 UTC; 7min ago
Main PID: 8265 (apache2)
Tasks: 6 (limit: 4915)
CGroup: /system.slice/apache2.service
├─8265 /usr/sbin/apache2 -k start
├─8267 /usr/sbin/apache2 -k start
├─8268 /usr/sbin/apache2 -k start
├─8269 /usr/sbin/apache2 -k start
├─8270 /usr/sbin/apache2 -k start
└─8271 /usr/sbin/apache2 -k start
Jun 17 19:33:57 small-http-stack systemd[1]: Stopped The Apache HTTP Server.
Jun 17 19:33:57 small-http-stack systemd[1]: Starting The Apache HTTP Server...
Jun 17 19:33:58 small-http-stack systemd[1]: Started The Apache HTTP Server.
Some lines were ellipsized; use -l to show in full.
  1. Open up a browser, and go to the IP address of your machine:
  1. Now that we have a functioning HTTP server deployed, let’s head over to Stackdriver to set up monitoring.
  2. Go to the left browsing tab | Stackdriver | and click on Monitoring.

Before we begin, Stackdriver will require us to create an account. Stackdriver accounts hold monitoring configurations for GCP projects. They also hold monitoring information for AWS accounts as well, if configured to do so. Projects are associated with a specific Stackdriver account for it to monitor those resources that are part of the project. While a Stackdriver account is necessary for monitoring purposes, logging does not require a Stackdriver account.

To enable monitoring, you first set up a Stackdriver account and then add your GCP projects and/or your AWS accounts to it. If you have a large number of projects, you can always move GCP projects between Stackdriver accounts or merge two Stackdriver accounts together. Remember that a Stackdriver account can contain up to 100 GCP projects and each project can only be monitored by one Stackdriver account.

When you create a Stackdriver account, it automatically creates additional projects that host all the dashboards and metrics, and also allow it to connect to AWS. The project that you wish to monitor using this Stackdriver account will be in addition to these additional projects that a Stackdriver account creates. To better explain this, let’s look at the following diagram:

You can see that Stackdriver Account A manages two GCP projects, Project A and Project D, and also two AWS accounts that are managed via two GCP projects, Project B and Project C. Each project here is given a role. The dotted lines represent these roles:

  • Hosting project: This project holds the monitoring configurations for the account, which include dashboards, uptime checks, and policies. It is advised not to deploy any of your machines or instances in this hosting project.
  • AWS connector project: When you add an AWS account to a Stackdriver account, it automatically creates the AWS connector project for you. Your AWS instances, such as EC2, must send their logs and metrics to this connector project for Stackdriver to pick them up. If you put any GCP resources in this project, those will not be monitored by Stackdriver.
  • Monitored project: All instances that you deploy for production will need to go into this monitored project. Any new projects that you want to be monitored by this Stackdriver account will automatically be called monitored projects.

Let’s go back to our dashboard and create a Stackdriver account for our project:

  1. Select your stackdriver-test-project and click Create account. Select the projects to add to this account. Because this is a lab, I am going to be using the stackdriver-test-project as a Stackdriver hosting account and also as an account where my compute engine lives:

  1. Click Continue. We will Skip AWS Setup, but it is good to glance at it and note what information is needed to set up your AWS account:
  1. Next, we will need to install the monitoring and logging agents on your instance. You can simply copy these commands and run them on your VM by logging in to it:
  1. Click Continue when done.
  1. Select No reports and click Continue:
  1. Next, you will see Stackdriver gathers all information and metrics through a Launch Monitoring button. Clicking that will give you the landing page for your project:
  1. Let’s create a simple dashboard. Click on CREATE DASHBOARD:
  1. Type in the name for this dashboard; we will call it CPU Perf:
  1. On the right side, click on Add Chart. Name the chart CPU Usage and click in the box to select a Resource type:
  1. Select gce_instance, and in the Metric, scroll down to select CPU usage:
  1. You should already see the right side of the graph populated. Click Save when done. You will see the chart added to our dashboard:

Let’s create some CPU load:

  1. In our GCP compute instance we created, run this command:
~$ sudo apt-get install stress
  1. This will install stress, which allows you to create load on your CPU with a set timeout. Once installed, run the following command and watch the dashboard:
~$ sudo stress --cpu 2 --timeout 120 
  1. Now refresh your dashboard, and you will see a spike in CPU usage:

You can also create uptime checks, alerting policies, and custom metrics, using Stackdriver monitoring. Uptime checks help to monitor and alert, if the application or the check fails. You can configure the uptime checks from the main landing page or on the side-bar tab.

Comments are closed.