DEV Community

myk_okoth_ogodo
myk_okoth_ogodo

Posted on

Go-lang Gorilla API + MongoDB in 30 minutes

Now hold your horses, before you start the thirty minute timer there is some house keeping we need to do first.One, we need to talk about No-SQL database family that MongoDB belongs to,two, we also need to touch on how to install MongoDB in our system (i am assuming you are using one of the Linux distros), then we will integrate this MongoDB database with a simple Gorilla-Mux API.

The code to this article can be found here.

No-SQL(Non-Relational Databases)

These are document-oriented databases that have a different data storing format to SQL database such as MySQL or PostgreSQL.There are various flavors of non-relational databases based on their data model. Main types are document databases like MongoDB, key-value databases like Redis, wide-column databases like Apache and Cassandra and finally graph databases like Orient-DB, Dgraph.

No-SQL databases give developers the freedom to store huge amounts of unstructured data. This is due to the fact that No-SQL databases are less structured or confined in format thus allowing for greater flexibility and adaptability in the type of data stored.

A popular example of an application that uses a No-SQL database would be Facebook's Messenger app that uses an Apache HBase to stream data to Hadoop clusters. Apache HBase is a wide-column database modeled on by Google's Big Table.

Characteristics of No-SQL(non-Relational) Databases:

  • Simple to integrate and use with apps for developers
  • Allows for Horizontal scaling or Scaling out.
  • Allows for Fast Querying
  • Have Flexible Schema.

Dive deeper into this topic at this article here.

Installing MongoDB

We first update our apt and install the MongoDB as shown below:


mykmyk@skynet:~$ sudo apt update

Enter fullscreen mode Exit fullscreen mode

then;

mykmyk@skynet:~$ sudo apt-get install mongodb

Enter fullscreen mode Exit fullscreen mode

Now, to confirm if your MongoBD database has installed
correctly, check that by doing the following:

mykmyk@skynet:~$ mongo
MongoDB shell version v3.6.3
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.6.3
Server has startup warnings: 
2022-07-23T22:58:19.824+0300 I STORAGE  [initandlisten] 
2022-07-23T22:58:19.824+0300 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2022-07-23T22:58:19.824+0300 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2022-07-23T22:58:30.835+0300 I CONTROL  [initandlisten] 
2022-07-23T22:58:30.835+0300 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2022-07-23T22:58:30.835+0300 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2022-07-23T22:58:30.835+0300 I CONTROL  [initandlisten] 
>
Enter fullscreen mode Exit fullscreen mode

If you ran into errors while you were installing MongoDB, i refer you to this well written article, it should guide sufficiently enough to get the database safely installed on your system.

To check the databases that are avaible in our MongoDB DMS:

> show databases;
admin   0.000GB
appdb   0.000GB
config  0.000GB
local   0.000GB
test    0.000GB

Enter fullscreen mode Exit fullscreen mode

The above command "show databases" lists all available databases , by default,admin,test, and local are the three that should be available before you create any new database;
We are going to create a database called, "booksdb", so while still in your MongoDB shell:

> use booksdb;
switched to db booksdb

Enter fullscreen mode Exit fullscreen mode

The command "use booksdb" above switches from the current database to the booksdb database that we just created. Its interesting to note that the "booksdb" wont show up if we query all the databases available using the "show databases" command. Go ahead and try it.

> use booksdb;
switched to db booksdb
> show databases;
admin   0.000GB
appdb   0.000GB
config  0.000GB
local   0.000GB
test    0.000GB
Enter fullscreen mode Exit fullscreen mode

The above phenomenon results from the internal implementation of MongoDB,a database is only created in MongoDB when actual initial data is inserted into it , so lets go ahead and do just that.

still on your MongoDB shell type in:

> db.books.insertOne({_id:5, title: 'A Song Of Ice and Fire',authors:['George R.R Martin','Phyllis Eisenstein'],publishdate:'August 1,1996',characters:['Jon Snow','Daenerys Targaryen','Eddard Star','Sansa Stark','Jammie Lannister','Rob Stark','Cerser Lannister'],publisher:{name:'Bantam Books',country:'United States',website:'www.randomhousebooks.com'}})
{ "acknowledged" : true, "insertedId" : 5 }

Enter fullscreen mode Exit fullscreen mode

The json we just inserted has an ID called _id, we could either provide that id while inserting our books object or we could leave it to MongoDB to insert it for us. Lets try and insert a book without specifying an id below and see what MongoDB will do:

> db.books.insertOne({title: 'A Feast Of Crows',authors:['George R.R Martin'],publishdate:'August 1,1996',characters:['Jon Snow','Daenerys Targaryen','Eddard Star','Sansa Stark','Jammie Lannister','Rob Stark','Cerser Lannister'],publisher:{name:'Bantam Books',country:'United States',website:'www.randomhousebooks.com'}})
{
    "acknowledged" : true,
    "insertedId" : ObjectId("62dda3f1457193537fd4d245")
}
> 
Enter fullscreen mode Exit fullscreen mode

As you can see above in the insert aknowledgment JSON response, insertId value has now changed to a long hash value that is generated internally by MongoDB, in our case it is "62dda3f1457193537fd4d245"

To search for books we use the find() keyword as shown below:

> db.books.find()
{ "_id" : 5, "title" : "A Song Of Ice and Fire", "authors" : [ "George R.R Martin", "Phyllis Eisenstein" ], "publishdate" : "August 1,1996", "characters" : [ "Jon Snow", "Daenerys Targaryen", "Eddard Star", "Sansa Stark", "Jammie Lannister", "Rob Stark", "Cerser Lannister" ], "publisher" : { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" } }
{ "_id" : ObjectId("62dda3f1457193537fd4d245"), "title" : "A Feast Of Crows", "authors" : [ "George R.R Martin" ], "publishdate" : "August 1,1996", "characters" : [ "Jon Snow", "Daenerys Targaryen", "Eddard Star", "Sansa Stark", "Jammie Lannister", "Rob Stark", "Cerser Lannister" ], "publisher" : { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" } }
> 

Enter fullscreen mode Exit fullscreen mode

The find() without any supplied arguments returns all the
documents in our "booksdb" collection. To return a single document we use the findOne(), if empty it returns latest document i.e:

> db.books.findOne()
{
    "_id" : 5,
    "title" : "A Song Of Ice and Fire",
    "authors" : [
        "George R.R Martin",
        "Phyllis Eisenstein"
    ],
    "publishdate" : "August 1,1996",
    "characters" : [
        "Jon Snow",
        "Daenerys Targaryen",
        "Eddard Star",
        "Sansa Stark",
        "Jammie Lannister",
        "Rob Stark",
        "Cerser Lannister"
    ],
    "publisher" : {
        "name" : "Bantam Books",
        "country" : "United States",
        "website" : "www.randomhousebooks.com"
    }
}
> 
Enter fullscreen mode Exit fullscreen mode

We can supply filtering parameters to our find() collection queries as shown below:

> db.books.find({title:{$eq: "A Feast Of Crows"}})
{ "_id" : ObjectId("62dda3f1457193537fd4d245"), "title" : "A Feast Of Crows", "authors" : [ "George R.R Martin" ], "publishdate" : "August 1,1996", "characters" : [ "Jon Snow", "Daenerys Targaryen", "Eddard Star", "Sansa Stark", "Jammie Lannister", "Rob Stark", "Cerser Lannister" ], "publisher" : { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" } }

Enter fullscreen mode Exit fullscreen mode

We can delete a document from a given collection using the deleteOne and deleteMany functions:

> db.books.deleteOne({"_id": ObjectId("62dda3f1457193537fd4d245")})
{ "acknowledged" : true, "deletedCount" : 1 }
Enter fullscreen mode Exit fullscreen mode

Next stage we concentrate on building up our very simple GorillaMux API:

Gorilla-Mux API

First things first,to allow our API to talk to the MongoDB database we will be using a database driver called mgo.

mgo- A Rich MongoDB driver for Go

This is MongoDB driver for go-lang that enables developers to build applications like in our case and have that API interface directly with the MongoDB database without having to go through the Mongo interactive shell. This diver acts as a wrapper around the MongoDB API.

Check out the documentation on this driver here, for a deeper understanding on its implementation and application.

mgo installation:

First lets create a folder for our application, create a folder called MongoGorilla:

mykmyk@skynet:~/go/src/github.com/myk4040okothogodo/GoMongo$ mkdir MongoGorilla

Enter fullscreen mode Exit fullscreen mode

then run go mod init, that create go.mod file to track the dependencies that we will use to build our API application like mgo driver above:

mykmyk@skynet:~/go/src/github.com/myk4040okothogodo/MongoGorilla$ go mod init
go: creating new go.mod: module github.com/myk4040okothogodo/MongoGorilla
mykmyk@skynet:~/go/src/github.com/myk4040okothogodo/MongoGorilla$ ll
total 12
drwxrwxr-x  2 mykmyk mykmyk 4096 Jul 23 23:24 ./
drwxrwxr-x 20 mykmyk mykmyk 4096 Jul 23 23:09 ../
-rw-rw-r--  1 mykmyk mykmyk   58 Jul 23 23:24 go.mod
Enter fullscreen mode Exit fullscreen mode

Now, let install our driver "mgo" as shown below:

mykmyk@skynet:~/go/src/github.com/myk4040okothogodo/MongoGorilla$ go get gopkg.in/mgo.v2
go: downloading gopkg.in/mgo.v2 v2.0.0-20190816093944-a6b53ec6cb22
go: added gopkg.in/mgo.v2 v2.0.0-20190816093944-a6b53ec6cb22
Enter fullscreen mode Exit fullscreen mode

Gorilla-Mux

Next we should add the gorilla/mux package , that will help use implement a very simple router in our API, to help match incoming request to respective handlers.
Dive deeper into the package and how it implements its dispatching features here.

To install the package and have it accessible in our project, while in our project root folder that houses the go.mod file, type in the following:

mykmyk@skynet:~/go/src/github.com/myk4040okothogodo/MongoGorilla$ go get -u github.com/gorilla/mux
go: downloading github.com/gorilla/mux v1.8.0
go: added github.com/gorilla/mux v1.8.0
mykmyk@skynet:~/go/src/github.com/myk4040okothogodo/MongoGorilla$ 

Enter fullscreen mode Exit fullscreen mode

API Building

Now lets write a simple program that talks to the MongoDB database using our "mgo" driver and inserts a book " A song of Ice and Fire" book record.

Create a file called "main.go" open it and add the following code shown below:

 1 package main                                                                                                                                                        
  2 
  3 import (
  4   "os"
  5   "os/signal"
  6   "context"
  7   "time"
  8   "net/http"
  9   "io/ioutil"
 10   "encoding/json"
 11   "github.com/hashicorp/go-hclog"
 12   "log"
 13   "github.com/gorilla/mux"
 14   mgo "gopkg.in/mgo.v2"
 15   "gopkg.in/mgo.v2/bson"
 16 
 17 )
 18 
 19 
 20 // The struct below holds our database session information
 21 type DBSession struct {
 22     session      *mgo.Session
 23     collection   *mgo.Collection                                                                                                                                    
 24 }                                                                                                                                                                   
 25                                                                                                                                                                     
 26                                                                                                                                                                     
 27 //Book struct holds book data                                                                                                                                       
 28 type Book struct {                                                                                                                                                  
 29     ID            bson.ObjectId    `json: "id"          bson:"_id,omitempty"`                                                                                       
 30     Title         string           `json: "title"       bson:"title"`                                                                                               
 31     Authors       []string         `json: "authors"     bson: "authors"`                                                                                            
 32     Genre         []string         `json: "genre"       bson: "genre"`                                                                                              
 33     PublishDate   string           `json: "publishdate" bson: "publishdate"`                                                                                        
 34     Characters    []string         `json: "characters"  bson: "characters"`                                                                                         
 35     Publisher     Publisher        `json: "publisher"   bson:"publisher"`                                                                                           
 36 }                                                                                                                                                                   
 37                                                                                                                                                                     
 38  
   39 // Publisher is nested in Movie
 40 type Publisher struct {
 41     Name     string   `json: "budget"    bson:"name"`
 42     Country  string   `json: "country"   bson:"country"`
 43     website  string   `json: "website"   bson:"website"`
 44 }
 45 
 46 
 47 // GetBook fetches a book with a given ID
 48 func (db *DBSession) GetBook (w http.ResponseWriter, r *http.Request){
 49     vars := mux.Vars(r)
 50 
 51     w.WriteHeader(http.StatusOK)
 52     var book Book
 53     err := db.collection.Find(bson.M{"_id": bson.ObjectIdHex(vars["id"])}).One(&book)
 54     if err != nil {
 55         w.Write([]byte(err.Error()))
 56     } else {
 57         w.Header().Set("Content-Type", "application/json")
 58         response, _ := json.Marshal(book)
 59         w.Write(response)
 60     }
 61 }
 62 
 63 
  64 //PostBook adds a new book to our MongoDB collection
 65 func (db *DBSession) PostBook (w http.ResponseWriter, r *http.Request){
 66     var book Book
 67     postBody, _ := ioutil.ReadAll(r.Body)
 68     json.Unmarshal(postBody, &book)
 69 
 70     //Create a Hash ID to insert
 71     book.ID = bson.NewObjectId()
 72     err := db.collection.Insert(book)
 73     if err != nil {
 74         w.Write([]byte(err.Error()))
 75     } else {
 76         w.Header().Set("Content-Type","application/json")
 77         response, _ := json.Marshal(book)
 78         w.Write(response)
 79     }
 80 }
 81 
 82 
 83 
 84 
 85 //UpdateBook modifies the data of an existing book resource
 86 func (db *DBSession) UpdateBook(w http.ResponseWriter, r *http.Request){
 87     vars := mux.Vars(r)
 88     var book Book
 89 
 90     putBody, _ := ioutil.ReadAll(r.Body)
 91     json.Unmarshal(putBody, &book)
 92     err := db.collection.Update(bson.M{"_id":                                                                                                                       
 93         bson.ObjectIdHex(vars["id"])}, bson.M{"$set": &book})
 94 
 95     if err != nil {
 96         w.WriteHeader(http.StatusOK)
 97         w.Write([]byte(err.Error()))
 98     } else {
 99         w.Header().Set("Content-Type","text")
100         w.Write([]byte("Update succesfully"))
101     }
102 }
103 
104 
105 //DeleteBook removes the data from the db                                                                                                                           
106 func (db *DBSession) DeleteBook (w http.ResponseWriter, r *http.Request){
107      vars := mux.Vars(r)
108      err := db.collection.Remove(bson.M{"_id":
109          bson.ObjectIdHex(vars["id"])})
110      if err != nil {
111          w.WriteHeader(http.StatusOK)
112          w.Write([]byte(err.Error()))
113      } else {
114          w.Header().Set("Content-Type", "text")
115          w.Write([]byte("Delete Succesfully"))
116      }
117 }
118 
119 func main() {
120     l := hclog.Default()
121     session, err := mgo.Dial("127.0.0.1")
122     c := session.DB("booksdb").C("books")
123     db := &DBSession{session: session, collection:c}
124     addr := "127.0.0.1:8000"
125     if err != nil {
126         panic(err)
127     }
128     defer session.Close()
129     
130     //logger := log.New(os.Stdout, "", log.Ldate | log.Ltime)
131     // Create a new router
132     r := mux.NewRouter()
133 
134     //Attach an elegant path with handler
135     r.HandleFunc("/api/books/{id:[a-zA-Z0-9]*}", db.GetBook).Methods("GET")
136     r.HandleFunc("/api/books", db.PostBook).Methods("POST")
137     r.HandleFunc("/api/books/{id:[a-zA-Z0-9]*}", db.UpdateBook).Methods("PUT")
138     r.HandleFunc("/api/books/{id:[a-zA-Z0-9]*}", db.DeleteBook).Methods("DELETE")
139 
140     srv := &http.Server{
141         Handler:       r,
142         Addr:          addr,
143         ErrorLog:      l.StandardLogger(&hclog.StandardLoggerOptions{}),
144         IdleTimeout:   time.Minute,
145         WriteTimeout:  15 * time.Second,
146         ReadTimeout:   15 * time.Second,
147     }                                                                                                                                                               
148 
149     //start the server
150     go func() {
151       l.Info("Starting server on port 8000 ")
152       err := srv.ListenAndServe()
153       if err != nil {
154           l.Error("Error starting the server :", "error", err)
155           os.Exit(1)
156       }        
157     }()
158 
159 
160     //Trap sigterm or interupt and gracefully shutdown the server
161     ch := make(chan os.Signal, 1)
162     signal.Notify(ch, os.Interrupt)
163     signal.Notify(ch, os.Kill)
164     
165     //Block untill a signal is received
166     sig := <- ch
167     log.Println("Got signal :", sig)
168 
169 
170     //gracefully shutdown the server, waiting max 30 for current operations to complete
171     ctx, _ := context.WithTimeout(context.Background(), 30 * time.Second)
172     srv.Shutdown(ctx)
173 
174 }                                                                                                                                                         
Enter fullscreen mode Exit fullscreen mode

Remember to run the command "go mod tidy", to have all the direct and indirect dependecies brought in.

In the above code we are first creating two structs , DBSession and Book struct;

DBSession structure will be used to hold our database session and collection properties of the mgo driver. Book structure will hold our book data, also note that the Book struct has a nested Publisher struct.

Next we define a receiver Pointer Method for our Book struct that we intend to have implement Handler interface.In this case we first define the method GetBook, that fetches a book from the database using an id that has been supplied as part of the request.It then returns this method as a response. We then define the second method, called PostBook that does just what its title suggests, it takes a book sent in as part of the Request body and then attempts to insert this book into our database. We the created another handler method called UpdateBook, that takes the updated values of a book that already exists in our database and tries to alter one of its values. Our Final Handler is the DeleteBook Method that is used the data pertaining to a book that we have stored in our database. We identify this particular book using an id that is sent in as part of the request.

We then create our main function, in this we instantiate a session with our database that we will use for all our subsequent queries and updates, we then create a new router using the NewRouter() method of gorilla/mux, we then attach our handlers to this router to enable dispatching of the various request to their subsequent handlers. We then launch our server as a go-routine , we then create a channel to listen and respond to server terminating signals.

To add a new book to our database and to test our POST endpoint lets make a post request to our API, open your terminal from any location and make the following curl request:

mykmyk@skynet:~/go/src/github.com/myk4040okothogodo/MongoGorilla$ curl -X POST \
> http://localhost:8000/api/books \
> -H 'cache-control: no-cache' \
> -H 'content-type: application/json' \
> -H 'postman-token: 6ef9507e-65b3-c3dd-4748-3a2a3e055c9c' \
> -d '{"title" :"A Dance with Dragons","authors":[ "George R.R Martin", "Phyllis Eisenstein"],"publishdate":"August 6,1998","characters":[ "Jon Snow", "Daenerys Targaryen", "Eddard Star", "Sansa Stark", "Jammie Lannister", "Rob Stark", "Cerser Lannister" ],"publisher": { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" }}'

Enter fullscreen mode Exit fullscreen mode

You will receive a reply from the API server that will look like:

{"ID":"62ddc5e89c99ea1313ae2568","Title":"A Dance with Dragons","Authors":["George R.R Martin","Phyllis Eisenstein"],"Genre":null,"PublishDate":"August 6,1998","Characters":["Jon Snow","Daenerys Targaryen","Eddard Star","Sansa Stark","Jammie Lannister","Rob Stark","Cerser Lannister"],"publisher": { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" }}
Enter fullscreen mode Exit fullscreen mode

Go to your database and confirm that there us a new entry, whose id has been auto-generated by MongoDB for us, it should look like:

{ "_id" : ObjectId("62ddc5e87071de3d88044140"), "id" : ObjectId("62ddc5e89c99ea1313ae2568"), "title" : "A Dance with Dragons", "authors" : [ "George R.R Martin", "Phyllis Eisenstein" ], "genre" : [ ], "publishdate" : "August 6,1998", "characters" : [ "Jon Snow", "Daenerys Targaryen", "Eddard Star", "Sansa Stark", "Jammie Lannister", "Rob Stark", "Cerser Lannister" ], "publisher" : { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" }}

Enter fullscreen mode Exit fullscreen mode

Lets test our GetBook endpoint, we are go supply the id that was returned when we added our book using the PostBook endpoint above, i.e in my case "62ddc5e89c99ea1313ae2568":

mykmyk@skynet:~/go/src/github.com/myk4040okothogodo/MongoGorilla$ curl -X GET \
> http://localhost:8000/api/books/62ddc5e89c99ea1313ae2568 \
> -H 'cache-control: no-cache' \
> -H 'postman-token: 00282916-e7f8-5977-ea34-d8f89aeb43e2'
Enter fullscreen mode Exit fullscreen mode

It should return:

{"ID":"62ddc5e89c99ea1313ae2568","Title":"A Dance with Dragons","Authors":["George R.R Martin","Phyllis Eisenstein"],"Genre":[],"PublishDate":"August 6,1998","Characters":["Jon Snow","Daenerys Targaryen","Eddard Star","Sansa Stark","Jammie Lannister","Rob Stark","Cerser Lannister"],"publisher" : { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" }}
Enter fullscreen mode Exit fullscreen mode

To test the delete endpoint:

curl -X DELETE http://localhost:8000/api/books/62ddc5e87071de3d88044140
Enter fullscreen mode Exit fullscreen mode

it should return:

Delete Successfully
Enter fullscreen mode Exit fullscreen mode

Now if we check our database using the find(), command we find that, the specific book "A Dance with Dragons" has been removed i.e:

> db.books.find()
{ "_id" : 5, "title" : "A Song Of Ice and Fire", "authors" : [ "George R.R Martin", "Phyllis Eisenstein" ], "publishdate" : "August 1,1996", "characters" : [ "Jon Snow", "Daenerys Targaryen", "Eddard Star", "Sansa Stark", "Jammie Lannister", "Rob Stark", "Cerser Lannister" ], "publisher" : { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" } }
{ "_id" : ObjectId("62dda3f1457193537fd4d245"), "title" : "A Feast Of Crows", "authors" : [ "George R.R Martin" ], "publishdate" : "August 1,1996", "characters" : [ "Jon Snow", "Daenerys Targaryen", "Eddard Star", "Sansa Stark", "Jammie Lannister", "Rob Stark", "Cerser Lannister" ], "publisher" : { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" } }

Enter fullscreen mode Exit fullscreen mode

To test the Put endpoint, that we will use to alter specific values of of book data:

I want to change the publisher to the book "A Feast of Crows" from "Bantam Books" to "Voyager" and i want to remove one of the authors "Phyllis Eisenstein"

the books currently looks like:

{ "_id" : 62dda3f1457193537fd4d245, "title" : "A Feast Of Crows", "authors" : [ "George R.R Martin", "Phyllis Eisenstein" ], "publishdate" : "August 1,1996", "characters" : [ "Jon Snow", "Daenerys Targaryen", "Eddard Star", "Sansa Stark", "Jammie Lannister", "Rob Stark", "Cerser Lannister" ], "publisher" : { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" } }
Enter fullscreen mode Exit fullscreen mode

Can you figure out how to alter the data, hint:

 curl -X PUT http://localhost:8000/api/books/.................
Enter fullscreen mode Exit fullscreen mode

If you get stuck put it down in the comments and i will be sure to help.

Conclusion
In this article we have mainly looked at MongoDB, how to install and add data to it using its interactive shell, then we talked about building a simple gorilla-mux api and having it interface with our database to store, delete and alter the details of books send using HTTP request methods like PUT, POST, GET and DELETE.

Goodbye,see you soon.

Top comments (0)