Observability - ApostropheCMS, OpenTelemetry, and New Relic

In this article, we will look at how to connect your site to New Relic using the OpenTelemetry observability framework.

What is OpenTelemetry?

OpenTelemetry, an open-source project, offers a set of vendor-neutral APIs that allow us to collect and transfer data to a location. This enables you to understand what your site is doing without having to worry about the tools or programming language being used. For more information, check out Open Telemetry’s website.

What is New Relic?

New Relic is a monitoring tool that can import data emitted by OpenTelemetry to help analyze and resolve application or website issues. It provides real-time data and shows the analytical performance of your site on the New Relic dashboard. It is designed to help discover unexpected outcomes and help you improve performance based on key metrics.

What is Observability?

Observability has become more important for sites in recent years. Most people tend to confuse Observability and Monitoring, but the main difference is that Monitoring tells you when something goes wrong, and Observability tells you why it went wrong. Monitoring has to happen before Observability takes place. Observability is one important concept that needs to be considered in complex systems, because if it doesn’t exist, then you can never know why things go wrong. This, in turn, will affect business performance and customer experience.

Profiling ApostropheCMS with OpenTelemetry already provides instructions for using another tool, Jaeger. The reason we are using New Relic in this article is that it provides more features out-of-the-box besides distributed tracing or monitoring. This article will be a quick guide on configuring this tool, which will extract telemetry data and export them to New Relic for monitoring.

Before moving on with this guide, we recommend you update Apostrophe to version 3.18.0, or preferably the latest 3.x version available.

Setup OpenTelemetry and NewRelic

Before we begin, there are a few things to note; we will be using an SDK for OpenTelemetry, which basically is used for measurement, and those measurements(data) will be exported to New Relic using an OTLP/PROTO Trace Exporter. The reason for using OTLP/PROTO is that New Relic expects its data in Proto format unlike the more familiar JSON format.

Step 1: Install the required dependencies for OpenTelemetry

Install the required dependencies in your project:

$ npm install @opentelemetry/sdk-node@0.27.0 @opentelemetry/auto-instrumentations-node@0.27.4 @opentelemetry/semantic-conventions@1.0.1 @opentelemetry/resources @opentelemetry/exporter-trace-otlp-proto uuid

Step 2: Login to New Relic

Before working with New Relic, you will need to have an account. So, go ahead and create one if you don't have one.

While on the dashboard, head over to the API Keys page and copy the INGEST-License key. This is what will provide permission to get data into New Relic.

With that complete, there are two ways we can export our data to New Relic; exporting directly to New Relic from our app or exporting to New Relic from a collector. But in this guide, we will be exporting directly from our Apostrophe app.

Step 3: Configure OpenTelemetry

At this point, we are about to do the real work where we have to configure OpenTelemetry and export telemetry data to New Relic. Exporting this kind of data relies on a specific protocol; the OpenTelemetry Protocol or OTLP.

In your project root directory, create a file telemetry.js:

const { NodeSDK } = require('@opentelemetry/sdk-node');
    const { getNodeAutoInstrumentations } = require('@opentelemetry/auto-instrumentations-node');
    const { SemanticResourceAttributes } = require('@opentelemetry/semantic-conventions');
    const { Resource } = require('@opentelemetry/resources');
    const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-proto');
    const { v4: uuidv4 } = require('uuid');
    const pkg = require('./package-lock.json');

    // Resources attributes
    const resource = new Resource({
      [SemanticResourceAttributes.SERVICE_NAME]: pkg.name,
      [SemanticResourceAttributes.SERVICE_VERSION]: pkg.version,
      [SemanticResourceAttributes.INSTANCE]: uuidv4(),
    });

    const instrumentations = [getNodeAutoInstrumentations()];

    const traceExporter = new OTLPTraceExporter({
      url: '<https://otlp.nr-data.net:4317/v1/traces>',
      headers: {
        'api-key': '<your-api-key>', // Your license key from NewRelic account
      },
    });

    const sdk = new NodeSDK({
      resource,
      instrumentations,
      traceExporter,
    });

    const shutdown = () => {
      sdk
        .shutdown()
        .then(() => {
          console.log('Shutdown complete');
        })
        .catch((err) => {
          console.error('Shutdown failed', err);
        });
    };

    module.exports = {
      sdk,
      shutdown,
    };

This code imports the necessary packages we had installed earlier and provides the resources attributes, which is more information about the service. It also enables auto-instrumentation (OpenTelemetry project facilitates the instrumentation or non-core components) before configuring the OTLP/PROTO trace exporter.

Finally, this code exports both the configured SDK function we will use and a shutdown function.

With that set, we have to refractor our app.js to make use of the opentelemetry setup:

const apostrophe = require('apostrophe');
    const process = require('process');
    const { sdk, shutdown } = require('./telemetry');

    const config = {
      shortName: 'apos-relic',
      modules: {},
    };

    if (process.env.APOS_OPENTELEMETRY) {
      sdk
        .start()
        .then(() => console.log('Started OpenTelemetry'))
        .then(() => apostrophe(config))
        .catch((err) => console.error('Failed to start OpenTelemetry', err));
    } else {
      apostrophe(config);
    }

    process.on('SIGTERM', () => {
      shutdown()
        .then(() => console.log('Tracing terminated'))
        .catch((error) => console.log('Error terminating tracing', error))
        .finally(() => process.exit(0));
    });

The important things to note in the above code is that we start Apostrophe after OpenTelemetry has started, and we shut down the sdk on process exit.

Step 4: Start your application

After everything is set and configured, we can then start our application.

$ APOS_OPENTELEMETRY=1 node app

Once our app is started, open http://localhost:3000 and head over to the New Relic UI to view your data. If you are logged into your account and navigate to the monitoring home page, you should be on the “Explorer” page. This page allows you to select any services that you have running. You site will be under “Services- OpenTelemetry”. Selecting it will bring up a dashboard showing various data about your site, like response time and error rate.

Three important observations you will need to look at on your New Relic UI are the traces, logs and metrics. Traces, otherwise known as distributed tracing, are basically application requests which are tagged with a unique identifier. Logs, on the other hand, are vital information and trigger alerts as they happen in your application. These may need some setup before any data is logged. Metrics are the core information or better still statistics; viewership statistics, engagement statistics, etc.

You may decide to trace command line tasks as well.

$ APOS_OPENTELEMETRY=1 node app @moduleName:taskName

Observing your application is critical to noticing which part is slow or needs attention. This guide should give you the power to ensure that your site is in perfect health. Good luck!

This post was originally contributed by an Apostrophe Community member, Victor Jonah. If you would like to contribute an article to the community, submit your article here.

We'd love to hear your feedback on this!

• Join our team and other Apostrophe developers from around the world on Discord
• Follow product updates and more on Github Discussions