DEV Community

Lisa Jung
Lisa Jung

Posted on

Part 4: Securely connect Node.js server to Elastic Cloud

Table of Content | Read Next: Part 5: Plan for efficient data storage and search performance in Elasticsearch

Resources

Would you rather watch a video to learn this content? Click on the link below!

The following are the project directories shared in the blog. Click on the link below to access it!

Connect Node.js Server to Elastic Cloud

In part 3, we signed up for a free trial of Elastic Cloud and created a deployment.

This deployment consists of Elasticsearch cluster, Kibana, Fleet, three Elastic Solutions(Enterprise Search, Observability, Security) and more!

Connecting to this deployment will give us access to everything mentioned above, including our Elasticsearch cluster.

We can connect our server to Elastic Cloud deployment via two different authentication mechanisms:

  1. basic authentication
  2. API key

API key is the best practice for production because it gives users flexibility in creating different API keys for different systems or applications.

In order to generate an API key, we need to log into Elastic Cloud as a user first(basic authentication).

Let's get started!

Step 1: Pull up the earthquake_app directory with your code editor

In part 2, we created a project directory called the earthquake_appand we built a Node.js server within it.

Next, we will create a file where we will store Elastic Cloud access credentials.

Step 2: Set up the config directory

In part 2, we have installed a dependency called config.

The config dependency allows you to store node configuration in an external configuration file. It can be customized based on different environments(local/dev/qa/prod/etc).

In our case, we will be using config to store access credentials such as Elastic cloud ID, username, password, and API key.

Within the earthquake_app directory, create a config directory. Within the config directory, create a default.json file as shown below.

Image description

Step 3: Enter the Elastic Cloud access credentials in the default.json file

Copy and paste the following code into the default.json file.



//in config/default.json
{
    "elastic": {
      "cloudID": "DEPLOYMENT_NAME:CLOUD_ID_DETAILS",
      "username": "elastic",
      "password": "LONGPASSWORD"
    }
}


Enter fullscreen mode Exit fullscreen mode

Your default.json file should look like the following screenshot.

Image description

We will replace "DEPLOYMENT_NAME:CLOUD_ID_DETAILS"(line 3) and "LONGPASSWORD"(line 5) with our Elastic Cloud deployment access credentials.

In part 3, we have downloaded and saved the access credentials.

Image description

Find and open the saved credentials.

You should see something like this.

Image description

Copy your password and paste it into line 5 "password" as shown below.

Image description

Next, let's find our "DEPLOYMENT NAME:CLOUD_ID_DETAILS".

In part 3, we learned how to get to the Kibana home page.

From the Kibana home page, click on the menu icon(red box).

Image description

It will display a drop down menu as shown below.

From the drop down menu, click on the Manage this deployment option(pink box).

Image description

You will see the following on your screen.
Image description

Look at the region highlighted with a red box. This is where your deployment name and cloud id information are displayed.

Click on the copy button(blue box) then paste the info into line 3 of the default.json file.

Your default.json file should look like the following screenshot with your credentials pasted into line 3 and line 5.

Image description

Step 4: Connect to Elastic Cloud using basic authentication

Within the server directory, create a elasticsearch directory. Within the elasticsearch directory, create a client.js file as shown below.

Image description

At this point, you might be wondering how we will get our server to communicate with Elastic Cloud so we can establish a secure connection?

giphy

It's actually quite simple!

Elasticsearch Client is the simplest and easiest way to communicate with your Elasticsearch cluster from your app, in a programming language of your choice.

Elasticsearch Client allows you to easily access every endpoint of Elasticsearch API to perform a variety of tasks.

Since we are building a server using Node.js with Express, we will be using the Elasticsearch Javascript Client to connect to our Elasticsearch cluster!

In client.js, we will create an instance of Elasticsearch Client. Elasticsearch Client will contain our Elastic Cloud access credentials that point to our Elasticsearch cluster.

Then, we will set up our server to connect to Elastic Cloud and display a message in the terminal of its connection status.

In the client.js file, copy and paste the following code.



//in server/elasticsearch/client.js
const { Client } = require('@elastic/elasticsearch');
const config = require('config');

const elasticConfig = config.get('elastic');

const client = new Client({
  cloud: {
    id: elasticConfig.cloudID,
  },
  auth: {
    username: elasticConfig.username,
    password: elasticConfig.password
  },
});

client.ping()
  .then(response => console.log("You are connected to Elasticsearch!"))
  .catch(error => console.error("Elasticsearch is not connected."))

module.exports = client;  


Enter fullscreen mode Exit fullscreen mode

Let's go over the code!

Heads up!
For reference purposes only, I have included screenshots of code that I will be explaining.

If you need to copy and paste the code, please refer to the code above or the GitHub repo for connecting to Elastic Cloud with basic authentication.

Line 1
Image description

We require @elastic/elasticsearch library we installed in part 2 and set this equal to a constant Client.

This library is the official Node.js client for Elasticsearch and it comes with a lot of functionalities. You will see this coming into play in line 6 when we create a new instance of Elasticsearch Client.

Line 2
We create a constant called config and require the config dependency we installed in part 2. You will see this come into play in line 4 when you use this dependency to get access credentials stored in the config directory.

Line 4
The access credentials were stored under 'elastic' in config/default.json as shown below.
Image description

We use config to get credentials stored under 'elastic' and set it equal to a constant elasticConfig.
Image description

This allows us to securely store and access sensitive data and not store these in the source code.

Lines 6-14
Image description
We create a new instance of Elasticsearch Client with our access credentials(cloudID, username, and password) that points to our Elasticsearch cluster. We set this equal to a constant called client.

Lines 16-18
Image description

We use the ping() method on our Elasticsearch Client to check if the Elasticsearch cluster is up and available to process requests.

We have it set up so that if we receive a response(meaning we are connected to Elasticsearch), our terminal will display the message "You are connected to Elasticsearch!".

If an error occurs during this process, our terminal should display the message "Elasticsearch is not connected".

Line 20
We expose our Elasticsearch Client via Node.js module exports as this will be used in server.js.

Go to the server.js file within the server directory. In line 2, add the following snippet.



//in server/server.js
const client = require('./elasticsearch/client');


Enter fullscreen mode Exit fullscreen mode

Your server.js should look like the following screenshot.

Image description

In line 2, we require client.js from the elasticsearch directory. We created an instance of Elasticsearch Client in this file.

We do this as we will need to access our Elasticsearch Client from server.js.

Let's test if we are able to connect to Elastic Cloud as a user(basic authentication)!

Start your server by running the following command in the terminal.



//in terminal earthquake_app directory
npm start


Enter fullscreen mode Exit fullscreen mode

You will see that the server is listening on port 3001 and that you are connected to Elasticsearch!

Image description

Step 5: Generate an API key
Now that we have logged into Elastic Cloud as a user, we can generate an API key.

From this point on, the API key will be used to connect to Elastic Cloud.

Within the server directory, create a file called create-api-key.js.

Image description

In create-api-key.js, copy and paste the following code.



//in server/create-api-key.js
const client = require('./elasticsearch/client');

async function generateApiKeys(opts) {
  const body = await client.security.createApiKey({
    body: {
      name: 'earthquake_app',
      role_descriptors: {
        earthquakes_example_writer: {
          cluster: ['monitor'],
          index: [
            {
              names: ['earthquakes'],
              privileges: ['create_index', 'write', 'read', 'manage'],
            },
          ],
        },
      },
    },
  });
  return Buffer.from(`${body.id}:${body.api_key}`).toString('base64');
}

generateApiKeys()
  .then(console.log)
  .catch((err) => {
    console.error(err);
    process.exit(1);
  });



Enter fullscreen mode Exit fullscreen mode

Your create-api-key.js should look like the following.

Image description

Let's go over the code!

Heads up!
For reference purposes only, I have included screenshots of code that I will be explaining.

If you need to copy and paste the code, please refer to the code above or the GitHub repo for connecting to Elastic Cloud with API key.

Line 1
We require the instance of Elasticsearch Client we created in the previous step.

Lines 3-21
We define the generateApiKeys() function. This function creates an API key that gives access to our Elastic Cloud deployment.

Image description

Line 4
We define a constant body and set it equal to client.security.createAPIKey() method.

Lines 5-18
Within this method, we define what Elasticsearch cluster privileges are granted to the server connecting with the generated API key.

Line 6
We specify the name of the server using this API key.

Line 7
Under role_descriptors, we define a role(access privileges) granted to this server.

Line 8
We name this role as earthquakes_example_writer. You could name this role any way you see fit.

Line 9
We specify that this API key comes with cluster monitor privilege, which gives read-only access for determining cluster state.

Lines 12-13
This key also grants additional privileges such as create_index, write, read, and manage operations for an index named earthquakes.

Line 20
The security.createApiKey function(line 4) returns an id and api_key value which are concatenated and encoded in base64.

Image description

Lines 23-28
We call the generateApiKeys() function here which prints the API key in the terminal. If an error occurs, it prints the error instead.

Let's generate an API key by running the create-api-key.js file.

In the terminal, create a separate tab. cd into the earthquake_app directory and run the following command.



//in terminal within the earthquake_app directory
node server/create-api-key.js


Enter fullscreen mode Exit fullscreen mode

You will see the following message in the terminal along with your API key.

Image description

Step 6: Connect to Elastic Cloud via API key

We will start by storing the API key in the default.json file of our config directory.

Go to default.json file and add a comma at the end of *line 5. In line 6, copy and paste the following.*



//in config/default.json
"apiKey": "paste your generated API key here"


Enter fullscreen mode Exit fullscreen mode

Copy the API key from your terminal and paste it into *line 6 in default.json.

Your default.json file should look like this.
Image description

Open the client.js file in the elasticsearch directory.

Image description

Replace username and password(lines 11 and 12) with the following code.



//in elasticsearch/client.js
apiKey: elasticConfig.apiKey


Enter fullscreen mode Exit fullscreen mode

Your client.js file should now look like the following screenshot(lines 11).

Image description

Go back to the terminal tab where you are running the server.

You will see that the server is connected to Elasticsearch except that this time we are connected to Elastic Cloud deployment via the API key!
Image description

Now that our server is connected to our Elasticsearch cluster, it is time to think about the data we want to store in Elasticsearch.

Move on to Part 5 to plan for efficient data storage and search performance in Elasticsearch!

Top comments (2)

Collapse
 
webjose profile image
José Pablo Ramírez Vargas

Hello, I thought you might be interested in wj-config, which is what I think a better configuration package than config.

On the surface it works very similarly to config, but it has a great feature to create URL's out of configuration data that just can't be beaten. Cheers.

Collapse
 
nickie-f profile image
Nickie Ferrante

I followed exactly what this guide told, but got stuck at step 4: It said "Elastic search is not connected". Can anybody help me? Thanks.