Today, we’re going to implement serverless RESTful API services by using “Serverless Framework”. There are many cloud services provider offers serverless functionality like AWS Lambda, Azure Functions, and Google CloudFunctions but in this article, I’m sticking with AWS Lambda as a cloud service provider.
If you don’t know about the serverless idea then I strongly recommended that first checkout this video and come back once finished it.
Serverless Framework
The serverless framework is an open source CLI tool that allows us to build, configure and deploy serverless functions (In our case, AWS Lambda functions).
Without "Serverless Framework", we have to go manually on console then create and configure necessary resources. That’s okay when the project is small and functions are limited but as soon as the project grows then creating and configuring resources is a challenging task and in lots of case unmaintainable. Writing code on console and managing team workflow becomes a tedious job.
With a "Serverless Framework", we can quickly build, configure and deploy resources within few commands. We can store our code and configuration into a centralized repository so we can design proper workflow and developers can later write, reuse and refer other developers codebase.
There are lots of significant advantages of using a serverless framework instead of doing manually work.
In this article, we're going to build a serverless Pokemon RESTful API services with a "Serverless Framework". Checkout below table for reference.
The code for this article can be found here: https://github.com/sagar-gavhane/pokemon-app
# | ENDPOINT | METHOD | DESCRIPTION |
---|---|---|---|
1 | pokemon/ | GET | Get a list of all pokemon from the database |
2 | pokemon/{id} | GET | Get a specific pokemon. |
3 | pokemon/ | POST | Add new pokemon to the database. |
4 | pokemon/{id} | PUT | Update existing pokemon. |
5 | pokemon/{id} | DELETE | Delete existing pokemon. |
Prerequisites
Install the following tools and frameworks:
- Node.js 8.10 or above
- MySQL
- Visual Studio Code (preffered) or any code editor
- Postman
Next, create the project folder and initialize it using npm.
mkdir pokemon-app
cd pokemon-app
npm init -f
Dependencies
Install the following packages to work with "Serverless Framework"
- express - Fast, unopinionated, minimalist web framework for Node.js.
- body-parser - Parse incoming request bodies in a middleware before your handlers, available under the req.body property.
- mysql - A pure node.js JavaScript Client implementing the MySql protocol.
- serverless - Framework for operationalize serverless development.
- serverless-http - Plugin allows you to wrap express API for serverless use.
- serverless-offline - Plugin to emulate AWS Lambda and API Gateway for speed up local development.
First up, we’ll install the serverless CLI:
npm install -g serverless
Now, let's install plugins and libraries step by step.
npm install express body-parser mysql serverless-http --save # app dependancies
npm install serverless-offline --save-dev # development dependancies
App structure
Before we start writing the handler code, we’re going to structure the project folder and configure our tools.
Create the following structure at the root level:
/pokemon-app/
|--/configs
|----/dbConfig.js
|--/node_modules
|--.gitignore
|--index.js
|--package.json
|--serverless.yml
Make sure to list private files into .gitignore
file so that we don’t accidentally commit it to public repository. Copy paste raw material from https://www.gitignore.io/api/node to .gitignore
file.
serverless.yml
file serves as a manifest for our RESTful api service. Where we define our functions, events, and necessary resources. Later, with serverless CLI we configure and deploy our service to AWS infrastructure.
# serverless.yml
service: pokemon-service
provider:
name: aws
runtime: nodejs8.10
stage: dev
region: us-east-1
memorySize: 512
functions:
pokemonFunc:
handler: index.handler
events:
- http:
path: pokemon
method: get
- http:
path: pokemon/{id}
method: get
- http:
path: pokemon
method: post
- http:
path: pokemon/{id}
method: put
- http:
path: pokemon/{id}
method: delete
plugins:
- serverless-offline
We are doing a few things here:
-
service:
pokemon-service
is a name of the service. You can give any type name for your service. -
provider: This is where we specify the name of the
provider
we’re using (AWS as cloud service provider) and configurations specific to it. In our case, we’ve configured the runtime (Node.js) with 8.10 version and region tous-east-1
. -
functions: We specify the functions provided by our service, Here I'm specifying
pokemonFunc
as function name withhttp
events. We can also say that this is our AWS Lambda function.
We have to store our pokemon somewhere, for sake of simplicity I'm chosen MySQL but you can also use another type database. I have already created a database with name pokemon_db and inside a database created table pokemon_tb with id, name, height, weight, avatar, and createAt columns.
CREATE TABLE `pokemon_tb` (
`id` int(11) NOT NULL,
`name` varchar(255) NOT NULL,
`height` float NOT NULL,
`weight` float NOT NULL,
`avatar` varchar(255) NOT NULL,
`createdAt` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
ALTER TABLE `pokemon_tb` ADD PRIMARY KEY (`id`);
ALTER TABLE `pokemon_tb` MODIFY `id` int(11) NOT NULL AUTO_INCREMENT, AUTO_INCREMENT=1;
Rather than creating and managing connections every time, we configure pool connections once inside dbConfig.js
file and reused it multiple times.
// dbConfig.js
const mysql = require('mysql')
const pool = mysql.createPool({
host : 'localhost',
user : 'root',
password : '12345',
database : 'pokemon_app_db',
})
module.exports = pool
Writing the handler function
Let's focus on handling RESTful api route inside the index.js file with express. First, we imported the serverless-http
package at the top. Second, we exported a handler function which is our application wrapped in the serverless package.
Here, we're implementing basic five routes for handling crud
operation with pokemon (without any validation).
const express = require('express')
const serverless = require('serverless-http')
const bodyParser = require('body-parser')
const pool = require('./configs/dbConfig')
const app = express()
app.use(bodyParser.json())
app.use(bodyParser.urlencoded({ extended: true }))
// Handle pokemon GET route for all pokemon
app.get('/pokemon/', (req, res) => {
const query = 'SELECT * FROM pokemon_tb'
pool.query(query, (err, results, fields) => {
if (err) {
const response = { data: null, message: err.message, }
res.send(response)
}
const pokemons = [...results]
const response = {
data: pokemons,
message: 'All pokemons successfully retrieved.',
}
res.send(response)
})
})
// Handle pokemon GET route for specific pokemon
app.get('/pokemon/:id', (req, res) => {
const id = req.params.id
const query = `SELECT * FROM pokemon_tb WHERE id=${id}`
pool.query(query, (err, results, fields) => {
if (err) {
const response = { data: null, message: err.message, }
res.send(response)
}
const pokemon = results[0]
const response = {
data: pokemon,
message: `Pokemon ${pokemon.name} successfully retrieved.`,
}
res.status(200).send(response)
})
})
// Handle pokemon POST route
app.post('/pokemon/', (req, res) => {
const { name, height, weight, avatar } = req.body
const query = `INSERT INTO pokemon_tb (name, height, weight, avatar) VALUES ('${name}', '${height}', '${weight}', '${avatar}')`
pool.query(query, (err, results, fields) => {
if (err) {
const response = { data: null, message: err.message, }
res.send(response)
}
const { insertId } = results
const pokemon = { id: insertId, name, height, weight, avatar }
const response = {
data: pokemon,
message: `Pokemon ${name} successfully added.`,
}
res.status(201).send(response)
})
})
// Handle pokemon PUT route
app.put('/pokemon/:id', (req, res) => {
const { id } = req.params
const query = `SELECT * FROM pokemon_tb WHERE id=${id} LIMIT 1`
pool.query(query, (err, results, fields) => {
if (err) {
const response = { data: null, message: err.message, }
res.send(response)
}
const { id, name, height, weight, avatar } = { ...results[0], ...req.body }
const query = `UPDATE pokemon_tb SET name='${name}', height='${height}', weight='${weight}', avatar='${avatar}' WHERE id='${id}'`
pool.query(query, (err, results, fields) => {
if (err) {
const response = { data: null, message: err.message, }
res.send(response)
}
const pokemon = {
id,
name,
height,
weight,
avatar,
}
const response = {
data: pokemon,
message: `Pokemon ${name} is successfully updated.`,
}
res.send(response)
})
})
})
// Handler pokemon DELETE route
app.delete('/pokemon/:id', (req, res) => {
const { id } = req.params
const query = `DELETE FROM pokemon_tb WHERE id=${id}`
pool.query(query, (err, results, fields) => {
if (err) {
const response = { data: null, message: err.message }
res.send(response)
}
const response = {
data: null,
message: `Pokemon with id: ${id} successfully deleted.`,
}
res.send(response)
})
})
// Handle in-valid route
app.all('*', function(req, res) {
const response = { data: null, message: 'Route not found!!' }
res.status(400).send(response)
})
// wrap express app instance with serverless http function
module.exports.handler = serverless(app)
Terminal snapshot:
Get all pokemon:
Get pokemon by id:
Add new pokemon:
Update existing pokemon:
Delete existing pokemon:
Deployment
Deloying services with serverless framework is so simple, we require to just hit deploy command.
serverless deploy
I have not setup a MySQL database on my AWS account hence RESTful services will not work on my AWS infrastructure. Later, I will deploy RESTful services to AWS infrastructure.
Conclusion
Creating RESTful API with a serverless framework is pretty straightforward. For serverless, we have to switch our development workflow. I found that lots of companies are moving towards creating and managing micro-services architecture instead of the monolithic app. That's sound great.
Closing Note
Thanks for reading. I hope you like this article feel free to like, comment or share this article with your friends. For more depth understanding of serverless framework checkout official documentation and blog of serverless.com.
Top comments (39)
Oh, and one other thing: Is it possible to use mongoose+MongoDB instead of mysql+MySql? Any downside there?
If you were to use GraphQL, would that be instead of mysql?
Magnus I'm working right now in a project using the serverless framework, you can use mongoose+MongoDB (Cloud Atlas in my case) without any problems, what I like a lot about the serverless framework features is that you can handle a lot of cases in my case I'm working in a public API so you can configure custom authorizers and CORS management / rules. its very flexible framework.
Great, thanks for sharing Erick. I look forward to trying it.
For simplicity I choose MySQL but you can use any NOSQL database. There is single downside that for aws doesn't provide mongodb. If want to use mongodb then you have run ec2 instance for the same.
Check out mlab.com where you can run MongoDB in Serverless fashion. There is a free tier for running a single node MongoDB.
Wow!! It's good. I know that AWS Aurora provides serverless SQL engine. If MongoDB, providing serverless then its too good. Thanks for information.
Lambda + Cloud Atlas is a pretty good stack.
Hey Erick, I have not tried Cloud Atlas before it will be good if you send an email any link or reference for the same.
Hey Sagar, Sure I'll sent you an email.
Erick, I haven't not received email. You can DM me.
Hey you can delete this one now.
Hi, Sagar, i was trying to implement Nodejs App with Mysql to build it serverless following your article, i get this error.
Serverless: Configuration warning at 'provider.runtime': should be equal to one of the allowed values [dotnetcore2.1, dotnetcore3.1, go1.x, java11, java8, java8.al2, nodejs10.x, nodejs12.x, provided, provided.al2, python2.7, python3.6, python3.7, python3.8, ruby2.5, ruby2.7]
i use below versions
Angular:- 10.1.2
Node:- 12.18.0
Mysql:- 8.0.17
please help me with the above error, i have followed same steps as mentioned in your article. (excepts for db config password)
basically, you need to provide runtime in the serverless.yml file.
Hi Sagar,
Thank you for sharing, upon changing serverless.yml runtime to nodejs12.x, i am not getting the error i mentioned in previous comment.
I am attaching screenshots of calling API in postman i dont get response as shown in your article, and get pokemon by id API doesnt work at all.
Please let me knoew how to get data of API correctly.
dev-to-uploads.s3.amazonaws.com/i/...
dev-to-uploads.s3.amazonaws.com/i/...
Also, I tried to debug by putting console statements in index.js file API's but console statements are not printed in BE.
app.get('/pokemon/', (req, res) => {
const query = 'SELECT * FROM pokemon_tb'
pool.query(query, (err, results, fields) => {
if (err) {
const response = { data: null, message: err.message, }
console.log('***********', response);
res.send(response)
}
})
please tell me why ?
IDK exact reason behind this. you need crosscheck function with the article mention code blocks and also you've to debug the root cause of this.
Hi, i had crossed checked function code blocks with article code blocks, its the same.
Can please tell me why console statements arent printing, it should print right ?
Great walkthrough! I've covered this topic in more detail with several different databases and scenarios. Have a look if you're interested.
I'd love some feedback!
Thank you 🙏🙇
Hi Sagar First of all, thanks for the tutorial. It works perfect for me, but ... When I tried this code for the first time, I used my MySQL RDS variables and everything works fine, but when I use my localhost variables I get this error:
Serverless: GET /user (λ: testFunc)
Debug: internal, implementation, error
TypeError: Uncaught error: undefined is not iterable (cannot read property Symbol(Symbol.iterator))
at Query. (...\kpoapi-base\user.js:20:19)
at bound (domain.js:415:14)
at Query.runBound as _callback
at Query.Sequence.end (...\kpoapi-base\node_modules\mysql\lib\protocol\sequences\Sequence.js:83:24)
at ...\kpoapi-base\node_modules\mysql\lib\Pool.js:205:13
at Handshake.onConnect (...\kpoapi-base\node_modules\mysql\lib\Pool.js:58:9)
at bound (domain.js:415:14)
at Handshake.runBound as _callback
at Handshake.Sequence.end (...\kpoapi-base\node_modules\mysql\lib\protocol\sequences\Sequence.js:83:24)
at Handshake.ErrorPacket (...\kpoapi-base\node_modules\mysql\lib\protocol\sequences\Handshake.js:125:8)
at Protocol._parsePacket (...\kpoapi-base\node_modules\mysql\lib\protocol\Protocol.js:291:23)
at Parser._parsePacket (...\kpoapi-base\node_modules\mysql\lib\protocol\Parser.js:433:10)
at Parser.write (...\kpoapi-base\node_modules\mysql\lib\protocol\Parser.js:43:10)
at Protocol.write (...\kpoapi-base\node_modules\mysql\lib\protocol\Protocol.js:38:16)
at Socket. (...\kpoapi-base\node_modules\mysql\lib\Connection.js:91:28) at Socket. (...\kpoapi-base\node_modules\mysql\lib\Connection.js:522:10)
at Socket.emit (events.js:196:13)
at Socket.EventEmitter.emit (domain.js:494:23)
at addChunk (_stream_readable.js:290:12)
at readableAddChunk (_stream_readable.js:271:11)
at Socket.Readable.push (_stream_readable.js:226:10)
at TCP.onStreamRead (internal/stream_base_commons.js:166:17)
Obviously, I tried the connection using Workbench and I have no problem.
What could be happening?
I would appreciate any help from you. I hope you can help me.
Hi Carlos,thanks for your feedback. As per error stack, your trying to iterate over undefined.
Hi Sagar, thanks for your response.
The problem, finally, was the type of authentication configured in my localhost of MySQL. I changed it to "Use Legacy Authentication Method" and the problem was solved. Thank you very much, again for your contributions.
In this article, I have used MySQL because of simplicity but you can try any database which you feel comfortable.
You can also try latest serverless database features.
Reference links:
Hey, Thank you very much for the post!
I had a quick question - Considering lambdas are short lived (3 or 5 mins if I am not wrong) - do we benefit from keeping a ConnectionPool?
Once the lambda lifecycle ends, would we not need to create this connection pool again?
If we keep creating this connection pool, isn't the purpose of pooling defeated?
Thanks!
Hi Anirudh, if we create a connection pool for each lambda then we're not reusing pool code. For code reusability, I have defined dbConfig.js file.
Thanks, Sagar. Quick question. Isn't express used to create a web service so to say (I.e. the functionality of the server). Why do we need both Serverless and express? Probably a pretty basic question, it just confused me a bit.
Hey Magnus, good question.
We can create api in serverless without using express. But express is so powerful and creating api with express is so much simpler.
Lots of node developers are expert in express.
Hey, thank you for your article !
I'm also a bit confused with express. Does the express server starts everytime we receive a call ? Or is there something done with
module.exports.handler = serverless(app)
that does something that I don't understand yet ?Hi Sagar, its an excellent sample. I just had a question, have you tried serving images while using serverless-http package. I am able to respond the images but i dont see the images being rendered. I thing there is some confusion with binary data. If possible can u share a sample with the image respose also.
Thanks!
Hey Sagar. Thanks in advance for the shared knowledge. I have a concern about it, you built a complete API in just one lambda function, I’m not sure if it’s a good practice but AWS recommends separating and dividing responsibilities inside a lambda funcion code. What do you think?
Hi Trysm, you're absolutely right. To explains developer in the simplest way, I haven't included any best practices or validation. If it's production application then I will definitely use string interpolation and so many other stuff. Thanks for you're a suggestion.