DEV Community

Cover image for AI Environmental Bot Creation: A Step-by-Step Guide with Twilio, Node.js, Gemini, and Render
Karthikeyan
Karthikeyan

Posted on • Originally published at imkarthikeyans.hashnode.dev

AI Environmental Bot Creation: A Step-by-Step Guide with Twilio, Node.js, Gemini, and Render

Hello πŸ‘‹

In this blog, we will explore how to build a WhatsApp bot using Twilio, Node.js, Typescript, Render, and Gemini.

Pre-requisites :

Setting up the development environment

First create the directory environmental-bot and run this following command

npm init -y
Enter fullscreen mode Exit fullscreen mode

Install the following dependencies to setup our basic node server. Later we will add the typescript to our project.

npm i express dotenv
Enter fullscreen mode Exit fullscreen mode

Create the index.js file in the src folder and the following code

const express = require('express');
const dotenv = require('dotenv');

dotenv.config();

const app = express();
const port = process.env.PORT;

app.get('/', (req, res) => {
  res.send('Express + TypeScript Server');
});

app.listen(3000, () => {
  console.log('[server]: Server is running at http://localhost:3000');
});
Enter fullscreen mode Exit fullscreen mode

Now in your terminal run npm run dev and if you check the browser , you should see the following

data

Now , let's add the typescript to our project. Install the following dependencies

npm i -D typescript @types/express @types/node
Enter fullscreen mode Exit fullscreen mode

Once done, let's initalize the tsconfig for our project

tsconfig.json is a configuration file for TypeScript that specifies the root files and compiler options needed to compile a TypeScript project.

Run the following command in your terminal

npx tsc --init
Enter fullscreen mode Exit fullscreen mode

This will create a basic tsconfig.json file in your project's root directory. You need to enable the outDir option, which specifies the destination directory for the compiled output. Locate this option in the tsconfig.json file and uncomment it.

By default, the value of this option is set to the project’s root. Change it to dist, as shown below:

{
  "compilerOptions": {
    ...
    "outDir": "./dist"
    ...
  }
}
Enter fullscreen mode Exit fullscreen mode

Let's update the main field in the package.json file to dist/index.js as the TypeScript code will compile from the src directory to dist. It should look something like this

// package.json
{
  "name": "environmental-bot",
  "version": "1.0.0",
  "description": "",
  "main": "dist/index.js",
  "scripts": {
    "build": "npx tsc",
    "start": "node dist/index.js",
    "dev": "nodemon src/index.ts"
  },
    ...
}
Enter fullscreen mode Exit fullscreen mode

Next step is to add type changes to our index.js file and rename it to index.ts

// src/index.js -> src/index.ts

import express, { Express, Request, Response } from "express";
import dotenv from "dotenv";

dotenv.config();

const app: Express = express();
const port = process.env.PORT || 3000;

app.get("/", (req: Request, res: Response) => {
  res.send("Express + TypeScript Server");
});

app.listen(3000, () => {
  console.log('[server]: Server is running at http://localhost:3000');
});
Enter fullscreen mode Exit fullscreen mode

Now let's try stopping and restarting the server with same command npm run dev and when we do , we will get the following error

Cannot use import statement outside a module
Enter fullscreen mode Exit fullscreen mode

That is because Node.js does not natively support the execution of TypeScript files. To avoid that, let's use the package called ts-node . Let's first install the following dependencies and make some changes in the package.json script file.

npm i -D nodemon ts-node
Enter fullscreen mode Exit fullscreen mode

Nodemon automatically restarts your Node.js application when it detects changes in the source files, streamlining the development process.

The build command compiles the code into JavaScript and saves it in the dist directory using the TypeScript Compiler (tsc). The dev command runs the Express server in development mode using nodemon and ts-node.

One last step before the fun part , let's install concurrently and setup it in our nodemon.json .

concurrently allows you to run multiple commands or scripts simultaneously in a single terminal window, streamlining your workflow.

## terminal command 
npm i -D concurrently

## nodemon.json
{
  "watch": ["src"],
  "ext": "ts",
  "exec": "concurrently \"npx tsc --watch\" \"ts-node src/index.ts\""
}
Enter fullscreen mode Exit fullscreen mode

Setting up the twilio

To complete the setup process for Twilio:

  1. Log in to your Twilio account.

  2. Access your Twilio Console dashboard.

  3. Navigate to the WhatsApp sandbox:

* Select **Messaging** from the left-hand menu.

* Click on **Try It Out**.

* Choose **Send A WhatsApp Message**.
Enter fullscreen mode Exit fullscreen mode

From the sandbox tab, note the Twilio phone number and the join code. The WhatsApp sandbox allows you to test your application in a development environment without needing approval from WhatsApp.

data

Coding the bot

data

System Architecture for AI-Driven Environmental Bot

  1. WhatsApp Bot (UI): User sends their location via WhatsApp.

  2. OpenWeatherMap API: Fetches the air quality index (AQI) based on the user's location.

  3. Gemini AI: Analyzes the AQI data and generates personalized safety advice.

  4. Response: The AI-generated advice is sent back to the user via WhatsApp.

To enable integration with WhatsApp, your web application must be accessible through a public domain.

To achieve this, utilize ngrok to expose your local web server by executing the following command in a different tab on your terminal:

ngrok http 3000
Enter fullscreen mode Exit fullscreen mode

data

Add the following things to the .env file

OPENWEATHERMAPAPI=****
GEMINIAPIKEY=****
Enter fullscreen mode Exit fullscreen mode

Let's create a another endpoint /incoming and add it in the Sandbox settings in the twilio console. It should look something like in the below screenshot

data

Add the following code in the src/index.ts

import express, { Express, Request, Response } from "express";
import dotenv from "dotenv";
import twilio from 'twilio';

dotenv.config();

const app = express();
const port = process.env.PORT;
const accountSid = process.env.ACCOUNTID;
const authToken = process.env.AUTHTOKEN;
// const client = new twilio(accountSid, authToken);

app.get('/', (req: Request, res: Response) => {
  res.send('Express + TypeScript Server');
});

const MessagingResponse = twilio.twiml.MessagingResponse;

app.post('/incoming', (req, res) => {
  const message = req.body;
  console.log(`Received message from ${message.From}: ${message.Body}`);
  const twiml = new MessagingResponse();
  twiml.message(`You said: ${message.Body}`);
  res.writeHead(200, { 'Content-Type': 'text/xml' });
  res.end(twiml.toString());
});

app.listen(port, () => {
  console.log(`[server]: Server is running at http://localhost:${port}`);
});
Enter fullscreen mode Exit fullscreen mode

Now, when you connect your WhatsApp and send a message, you should receive a reply.

data

Next step , let's connect the openweathermap and get the AQI response. Let's update the code


// src/index.js
import express, { Express, Request, Response } from "express";
import dotenv from "dotenv";
import twilio from 'twilio';
import axios from 'axios'

dotenv.config();

const app = express();
const port = process.env.PORT;
const MessagingResponse = twilio.twiml.MessagingResponse;

async function getAirQuality(lat: number, lon: number) {
  const apiKey = process.env.OPENWEATHERMAPAPI;
  const url = `http://api.openweathermap.org/data/2.5/air_pollution?lat=${lat}&lon=${lon}&appid=${apiKey}`;
  const response = await axios.get(url);
  const airQualityIndex = response.data.list[0].main.aqi;
  return airQualityIndex;
}

app.get('/', (req: Request, res: Response) => {
  res.send('Express + TypeScript Server');
});

app.post('/incoming', async (req, res) => {
  const message = req.body;
  const {Latitude, Longtitude, From, Body} = message;
  const airQualityIndex = await getAirQuality(Latitude, Longtitude);
  console.log(`Air quality index in your places is ${airQualityIndex}`);
  const twiml = new MessagingResponse();
  twiml.message(`You said: ${message.Body}`);
  res.writeHead(200, { 'Content-Type': 'text/xml' });
  res.end(twiml.toString());
});

app.listen(port, () => {
  console.log(`[server]: Server is running at http://localhost:${port}`);
});
Enter fullscreen mode Exit fullscreen mode

Now, when we run the following code , and send a message in whatsapp , we will get the AQI response in the console.


{
  "coord":[
    50,
    50
  ],
  "list":[
    {
      "dt":1605182400,
      "main":{
        "aqi":1
      },
      "components":{
        "co":201.94053649902344,
        "no":0.01877197064459324,
        "no2":0.7711350917816162,
        "o3":68.66455078125,
        "so2":0.6407499313354492,
        "pm2_5":0.5,
        "pm10":0.540438711643219,
        "nh3":0.12369127571582794
      }
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

Now, for the final part, integrate Gemini to complete the bot.

Install the following dependencies

npm install @google/generative-ai
Enter fullscreen mode Exit fullscreen mode

Update the following in your src/index.ts file

// src/index.js
import express, { Express, Request, Response } from 'express';
import dotenv from 'dotenv';
import MessagingResponse from 'twilio/lib/twiml/MessagingResponse';
import axios from 'axios';
import { GoogleGenerativeAI } from '@google/generative-ai';
import cors from 'cors';

dotenv.config();

const app: Express = express();
const port = process.env.PORT;

const geminiAPIKey = process.env.GEMINIAPIKEY as string;
const genAI = new GoogleGenerativeAI(geminiAPIKey);
const model = genAI.getGenerativeModel({ model: "gemini-1.5-flash" });

async function getAirQuality(lat: number, lon: number) {
  const apiKey = process.env.OPENWEATHERMAPAPI;
  const url = `http://api.openweathermap.org/data/2.5/air_pollution?lat=${lat}&lon=${lon}&appid=${apiKey}`;
  const response = await axios.get(url);
  const airQualityIndex = response.data.list[0].main.aqi;

  return airQualityIndex;
}

const predictHazard = async (airQualityIndex: number) => {
  const prompt = `The air quality index is ${airQualityIndex}. The AQI scale is from 1 to 5, where 1 is good and 5 is very poor. Predict the potential hazard level and provide safety advice.`;
  const result = await model.generateContent(prompt);
  const response = await result.response;
  const text = response.text();
  console.log(text);
  return text;
}

app.use(cors());
app.use(express.urlencoded({ extended: false }));
app.use(express.json());

app.get('/', (req: Request, res: Response) => {
  res.send('Express + TypeScript Server');
});

app.post('/incoming', async (req, res) => {
  const { Latitude, Longitude, Body } = req.body;

  console.log(Latitude, Longitude);
  const airQuality = await getAirQuality(Latitude, Longitude);

  console.log("airQuality", airQuality);
  console.log(`Received message from ${Body}`);

  const alert = await predictHazard(airQuality);

  const twiml = new MessagingResponse();
  twiml.message(alert);
  res.writeHead(200, { 'Content-Type': 'text/xml' });
  res.end(twiml.toString());
});

app.listen(port, () => {
  console.log(`[server]: Server is running at http://localhost:${port}`);
});
Enter fullscreen mode Exit fullscreen mode

Quick code overview

  • AI and API Key Setup: Set up the Google Generative AI model and the OpenWeatherMap API key.

  • Air Quality Function: Define an async function getAirQuality to fetch air quality data from OpenWeatherMap using latitude and longitude.

  • Hazard Prediction Function: Define an async function predictHazard that uses the Google Generative AI model to generate safety advice based on the air quality index.

  • Middleware: Use cors, express.urlencoded, and express.json to handle requests.

There are many ways to deploy the node.js application, I have connected the github repository to render.

You can start by deploying it as a service, adding the necessary environment variables, and then deploying it. Once completed, add it to the Twilio sandbox settings.

data

Let's test it out.

Connect to the whatsapp sandbox and send a location message , you should get the response from the AI

data

![data](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s4jmu4bcqy34nvj3k06k.png

Conclusion

That's pretty much it. Thank you for taking the time to read the blog post. If you found the post useful , add ❀️ to it and let me know in the comment section if I have missed something.

Feedback on the blog is most welcome.

Social Links:

Twitter

Instagram

Top comments (3)

Collapse
 
geekcoding101 profile image
GeekCoding101.com

Hey, this is amazing! I would vote you to winner!
I did follow up your steps and it worked great!

I spotted a few things you might want to update so that can benefit others:

  1. A typo: Longitude should be Longtitude. I spent some time to figure it out.
  2. Looks like cors is not needed, I didn't use it at least and it worked well.
  3. Should use app.use(express.urlencoded({ extended: true })); in your code when first time introducing twilio. Otherwise it will failed.
  4. Please mention installing twilio and axios packages.
  5. If restarted ngrok, it will use a new random endpoint. So we need to update the endpoint in twilio.

Thanks!

Collapse
 
imkarthikeyan profile image
Karthikeyan

Thanks @geekcoding101 . will update them

Collapse
 
ghoshbishakh profile image
Bishakh Ghosh

Instead of ngrok, once can use Pinggy. It is easier to run.

One command:

ssh -p 443 -o ServerAliveInterval=30 -R0:localhost:3000 a.pinggy.io

Here is a guide: pinggy.io/quickstart/twilio/