Sitaram Rathi

Posted on Jan 4 • Edited on May 11

Uploading and Serving Images from MongoDB in Golang

👨🏻‍💻 Introduction

In this blog, we will delve into the fascinating realm of handling images in a Golang application, leveraging the power of the Gin framework for RESTful API development, MongoDB as a robust NoSQL database, and the mongo-driver library for seamless interaction with MongoDB. To store images efficiently, we'll explore the intricacies of GridFS, a specification within MongoDB for storing large files as separate chunks.

For complete code please visit here, don't forget to ⭐ it if it's useful to you: serve-image

1. Gin Framework:

Gin, a high-performance web framework for Golang, will serve as the backbone of our RESTful API. Known for its speed and minimalistic design, Gin provides essential routing and middleware functionalities, making it an ideal choice for building scalable and efficient web applications. We'll explore how to set up routes, handle HTTP requests, and integrate middleware for image-related operations.

2. MongoDB and mongo-driver:

MongoDB, a document-oriented NoSQL database, will be our data powerhouse. We'll utilize the mongo-driver library to seamlessly connect our Golang application to MongoDB. This section will cover essential database interactions, including creating collections, storing metadata, and efficiently querying for image-related data. Understanding these fundamentals is crucial for building a robust image storage and retrieval system.

3. GridFS for Image Storage:

GridFS, an integral part of MongoDB, enables us to store large files, such as images, by dividing them into smaller, manageable chunks. Each chunk is stored as a separate document, allowing for efficient retrieval and storage. We'll delve into the details of using GridFS with the mongo-driver in Golang, including uploading images, managing chunks, and retrieving images seamlessly.

🔭 Methodology:

In this section, we'll walk through the step-by-step process of creating REST endpoints for uploading and serving images in a Golang application using Gin, MongoDB, and GridFS.

1. Image Uploading Endpoint:

Receive Image via Form File:
To initiate the image uploading process, our Golang application will expose a REST endpoint that accepts image files sent as a Form file from the client.
Connect to MongoDB:
We'll establish a connection to MongoDB using the mongo-driver library, ensuring seamless interaction with our database.
Initialize GridFS Upload Stream:
To handle large file uploads efficiently, we'll open an upload stream in the MongoDB GridFS bucket. This stream will enable us to upload image data in manageable chunks.
Write Image Data in Chunks:
As the image data is received, we'll write it into the GridFS upload stream in chunks. This method optimizes the storage and retrieval of large files in MongoDB.
Return FileId in Response:
Upon successful upload, the response to the client will include the MongoDB primitive ID (FileId) associated with the uploaded image. This ID will serve as a reference for later retrieval.

2. Image Serving Endpoint:

Receive FileId in Request:
The serving endpoint will expect a request containing the FileId, which uniquely identifies the desired image in the MongoDB GridFS bucket.
Retrieve Image Data from MongoDB:
Using the received FileId, we'll fetch the corresponding image data from the MongoDB GridFS bucket.
Send Binary Data in Response:
The retrieved binary image data will be sent as the response body with appropriate content-type headers, ensuring compatibility with various clients.

⚙️ Project Setup:

Step 1: Setting Up the Project

Create a Golang project and initialize it with Go modules:



go mod init image-server

Install the required packages using the following commands:



go install github.com/gin-gonic/gin

go install github.com/joho/godotenv

go install go.mongodb.org/mongo-driver

Step 2: Environment Configuration

Create a .env file in the root of your project and define two variables:



MONGOURI=<your_mongodb_connection_string>

PORT=8001

🚀 Endpoints:

Certainly! Here are the explanations for both endpoints in Markdown format:

1. Upload Image:

`http://localhost:8001/upload`

Method:

POST

Purpose:

Accepts a POST request to upload an image to the MongoDB database using GridFS.

Functionality:

Form File Extraction:
- The endpoint expects a POST request with an image file included in the form data. The file is retrieved using c.Request.FormFile("image").
GridFS Bucket Creation:
- A GridFS bucket named "photos" is created within the "image-server" database using the gridfs.NewBucket() function.
Image Data Buffering:
- The image data is read into a buffer using io.Copy(buf, file).
Filename Generation:
- A unique filename is generated based on the current timestamp and the original filename.
Upload Stream Opening:
- An upload stream is opened in the GridFS bucket for the specified filename using bucket.OpenUploadStream(filename,).
Image Data Writing:
- The image data in the buffer is written to the upload stream in chunks using uploadStream.Write(buf.Bytes()).
Response Construction:
- The function constructs a JSON response containing the FileId and fileSize and sends it back to the client using c.JSON(http.StatusOK, map[string]interface{}{"fileId": fileId, "fileSize": fileSize}).

2. Get Image:

`http://localhost:8001/image/:imageId`

Method:

GET

Purpose:

Accepts a GET request to serve an image based on the provided imageId.

Functionality:

ImageId Extraction:
- The endpoint expects a GET request with the imageId as a path parameter, which is extracted using imageId := strings.TrimPrefix(c.Request.URL.Path, "/image/").
ObjectID Conversion:
- The extracted imageId is converted to a MongoDB ObjectID using primitive.ObjectIDFromHex(imageId).
GridFS Bucket Creation:
- A GridFS bucket named "photos" is created within the "image-server" database using gridfs.NewBucket().
Image Data Retrieval:
- The image data associated with the ObjectID is retrieved from the GridFS bucket using bucket.DownloadToStream(objID, &buf).
Content-Type Determination:
- The content-type header is set based on the detected content type of the image using contentType := http.DetectContentType(buf.Bytes()).
Response Construction:
- The function sets appropriate headers (Content-Type and Content-Length) and writes the image data to the HTTP response using c.Writer.Write(buf.Bytes()), effectively serving the image.

These explanations provide a detailed understanding of the both the endpoints for image upload and serving in from the server. You can also checkout the PostMan documentation where you can better understand the working of both these endpoints.

🤖 Code:

Paste the below code in main.go file.



package main

import (

    "bytes"

    "context"

    "encoding/json"

    "io"

    "log"

    "net/http"

    "os"

    "strconv"

    "strings"

    "time"

<span class="s">"github.com/gin-gonic/gin"</span>
<span class="s">"github.com/joho/godotenv"</span>
<span class="s">"go.mongodb.org/mongo-driver/bson/primitive"</span>
<span class="s">"go.mongodb.org/mongo-driver/mongo"</span>
<span class="s">"go.mongodb.org/mongo-driver/mongo/gridfs"</span>
<span class="s">"go.mongodb.org/mongo-driver/mongo/options"</span>



    

    




)

// To get mongodb connection URI from .env file

func EnvMongoURI() string {

    err := godotenv.Load()

    if err != nil {

        log.Fatal("Error loading .env file")

    }

    return os.Getenv("MONGOURI")

}

// To connect to mongodb

func ConnectDB() mongo.Client {

    ctx, cancel := context.WithTimeout(context.Background(), 30time.Second)

    defer cancel()

<span class="c">// mongo.Connect return mongo.Client method</span>
<span class="n">uri</span> <span class="o">:=</span> <span class="n">EnvMongoURI</span><span class="p">()</span>
<span class="n">client</span><span class="p">,</span> <span class="n">err</span> <span class="o">:=</span> <span class="n">mongo</span><span class="o">.</span><span class="n">Connect</span><span class="p">(</span><span class="n">ctx</span><span class="p">,</span> <span class="n">options</span><span class="o">.</span><span class="n">Client</span><span class="p">()</span><span class="o">.</span><span class="n">ApplyURI</span><span class="p">(</span><span class="n">uri</span><span class="p">))</span>
<span class="k">if</span> <span class="n">err</span> <span class="o">!=</span> <span class="no">nil</span> <span class="p">{</span>
    <span class="n">log</span><span class="o">.</span><span class="n">Fatal</span><span class="p">(</span><span class="s">"Error: "</span> <span class="o">+</span> <span class="n">err</span><span class="o">.</span><span class="n">Error</span><span class="p">())</span>
<span class="p">}</span>

<span class="c">//ping the database</span>
<span class="n">err</span> <span class="o">=</span> <span class="n">client</span><span class="o">.</span><span class="n">Ping</span><span class="p">(</span><span class="n">ctx</span><span class="p">,</span> <span class="no">nil</span><span class="p">)</span>
<span class="k">if</span> <span class="n">err</span> <span class="o">!=</span> <span class="no">nil</span> <span class="p">{</span>
    <span class="n">log</span><span class="o">.</span><span class="n">Fatal</span><span class="p">(</span><span class="s">"Error: "</span> <span class="o">+</span> <span class="n">err</span><span class="o">.</span><span class="n">Error</span><span class="p">())</span>
<span class="p">}</span>
<span class="n">log</span><span class="o">.</span><span class="n">Println</span><span class="p">(</span><span class="s">"Connected to MongoDB"</span><span class="p">)</span>
<span class="k">return</span> <span class="n">client</span>



    

    




}

// Client instance

var DB *mongo.Client = ConnectDB()

var name = "photos"

var opt = options.GridFSBucket().SetName(name)

// Getting database collections

func GetCollection(client mongo.Client, collectionName string) mongo.Collection {

    collection := client.Database("image-server").Collection(collectionName)

    return collection

}

// Upload image handler

func uploadImage() gin.HandlerFunc {

    return func(c *gin.Context) {

        file, header, err := c.Request.FormFile("image")

        if err != nil {

            log.Fatal(err)

            c.JSON(http.StatusBadRequest, err.Error())

            return

        }

        defer file.Close()

    <span class="n">bucket</span><span class="p">,</span> <span class="n">err</span> <span class="o">:=</span> <span class="n">gridfs</span><span class="o">.</span><span class="n">NewBucket</span><span class="p">(</span>
        <span class="n">DB</span><span class="o">.</span><span class="n">Database</span><span class="p">(</span><span class="s">"image-server"</span><span class="p">),</span> <span class="n">opt</span><span class="p">,</span>
    <span class="p">)</span>
    <span class="k">if</span> <span class="n">err</span> <span class="o">!=</span> <span class="no">nil</span> <span class="p">{</span>
        <span class="n">log</span><span class="o">.</span><span class="n">Fatal</span><span class="p">(</span><span class="n">err</span><span class="p">)</span>
        <span class="n">c</span><span class="o">.</span><span class="n">JSON</span><span class="p">(</span><span class="n">http</span><span class="o">.</span><span class="n">StatusBadRequest</span><span class="p">,</span> <span class="n">err</span><span class="o">.</span><span class="n">Error</span><span class="p">())</span>
        <span class="k">return</span>
    <span class="p">}</span>

    <span class="n">buf</span> <span class="o">:=</span> <span class="n">bytes</span><span class="o">.</span><span class="n">NewBuffer</span><span class="p">(</span><span class="no">nil</span><span class="p">)</span>
    <span class="k">if</span> <span class="n">_</span><span class="p">,</span> <span class="n">err</span> <span class="o">:=</span> <span class="n">io</span><span class="o">.</span><span class="n">Copy</span><span class="p">(</span><span class="n">buf</span><span class="p">,</span> <span class="n">file</span><span class="p">);</span> <span class="n">err</span> <span class="o">!=</span> <span class="no">nil</span> <span class="p">{</span>
        <span class="n">log</span><span class="o">.</span><span class="n">Fatal</span><span class="p">(</span><span class="n">err</span><span class="p">)</span>
        <span class="n">c</span><span class="o">.</span><span class="n">JSON</span><span class="p">(</span><span class="n">http</span><span class="o">.</span><span class="n">StatusBadRequest</span><span class="p">,</span> <span class="n">err</span><span class="o">.</span><span class="n">Error</span><span class="p">())</span>
        <span class="k">return</span>
    <span class="p">}</span>

    <span class="n">filename</span> <span class="o">:=</span> <span class="n">time</span><span class="o">.</span><span class="n">Now</span><span class="p">()</span><span class="o">.</span><span class="n">Format</span><span class="p">(</span><span class="n">time</span><span class="o">.</span><span class="n">RFC3339</span><span class="p">)</span> <span class="o">+</span> <span class="s">"_"</span> <span class="o">+</span> <span class="n">header</span><span class="o">.</span><span class="n">Filename</span>
    <span class="n">uploadStream</span><span class="p">,</span> <span class="n">err</span> <span class="o">:=</span> <span class="n">bucket</span><span class="o">.</span><span class="n">OpenUploadStream</span><span class="p">(</span>
        <span class="n">filename</span><span class="p">,</span>
    <span class="p">)</span>
    <span class="k">if</span> <span class="n">err</span> <span class="o">!=</span> <span class="no">nil</span> <span class="p">{</span>
        <span class="n">log</span><span class="o">.</span><span class="n">Fatal</span><span class="p">(</span><span class="n">err</span><span class="p">)</span>
        <span class="n">c</span><span class="o">.</span><span class="n">JSON</span><span class="p">(</span><span class="n">http</span><span class="o">.</span><span class="n">StatusBadRequest</span><span class="p">,</span> <span class="n">err</span><span class="o">.</span><span class="n">Error</span><span class="p">())</span>
        <span class="k">return</span>
    <span class="p">}</span>
    <span class="k">defer</span> <span class="n">uploadStream</span><span class="o">.</span><span class="n">Close</span><span class="p">()</span>

    <span class="n">fileSize</span><span class="p">,</span> <span class="n">err</span> <span class="o">:=</span> <span class="n">uploadStream</span><span class="o">.</span><span class="n">Write</span><span class="p">(</span><span class="n">buf</span><span class="o">.</span><span class="n">Bytes</span><span class="p">())</span>
    <span class="k">if</span> <span class="n">err</span> <span class="o">!=</span> <span class="no">nil</span> <span class="p">{</span>
        <span class="n">log</span><span class="o">.</span><span class="n">Fatal</span><span class="p">(</span><span class="n">err</span><span class="p">)</span>
        <span class="n">c</span><span class="o">.</span><span class="n">JSON</span><span class="p">(</span><span class="n">http</span><span class="o">.</span><span class="n">StatusBadRequest</span><span class="p">,</span> <span class="n">err</span><span class="o">.</span><span class="n">Error</span><span class="p">())</span>
        <span class="k">return</span>
    <span class="p">}</span>

    <span class="n">fileId</span><span class="p">,</span> <span class="n">_</span> <span class="o">:=</span> <span class="n">json</span><span class="o">.</span><span class="n">Marshal</span><span class="p">(</span><span class="n">uploadStream</span><span class="o">.</span><span class="n">FileID</span><span class="p">)</span>
    <span class="k">if</span> <span class="n">err</span> <span class="o">!=</span> <span class="no">nil</span> <span class="p">{</span>
        <span class="n">log</span><span class="o">.</span><span class="n">Fatal</span><span class="p">(</span><span class="n">err</span><span class="p">)</span>
        <span class="n">c</span><span class="o">.</span><span class="n">JSON</span><span class="p">(</span><span class="n">http</span><span class="o">.</span><span class="n">StatusBadRequest</span><span class="p">,</span> <span class="n">err</span><span class="o">.</span><span class="n">Error</span><span class="p">())</span>
        <span class="k">return</span>
    <span class="p">}</span>
    <span class="n">c</span><span class="o">.</span><span class="n">JSON</span><span class="p">(</span><span class="n">http</span><span class="o">.</span><span class="n">StatusOK</span><span class="p">,</span> <span class="k">map</span><span class="p">[</span><span class="kt">string</span><span class="p">]</span><span class="k">interface</span><span class="p">{}{</span><span class="s">"fileId"</span><span class="o">:</span> <span class="n">strings</span><span class="o">.</span><span class="n">Trim</span><span class="p">(</span><span class="kt">string</span><span class="p">(</span><span class="n">fileId</span><span class="p">),</span> <span class="s">`"`</span><span class="p">),</span> <span class="s">"fileSize"</span><span class="o">:</span> <span class="n">fileSize</span><span class="p">})</span>
<span class="p">}</span>



    

    




}

// Serving image over http REST handler

func serveImage() gin.HandlerFunc {

    return func(c *gin.Context) {

        imageId := strings.TrimPrefix(c.Request.URL.Path, "/image/")

        objID, err := primitive.ObjectIDFromHex(imageId)

        if err != nil {

            log.Fatal(err)

            c.JSON(http.StatusBadRequest, err.Error())

            return

        }

    <span class="n">bucket</span><span class="p">,</span> <span class="n">_</span> <span class="o">:=</span> <span class="n">gridfs</span><span class="o">.</span><span class="n">NewBucket</span><span class="p">(</span>
        <span class="n">DB</span><span class="o">.</span><span class="n">Database</span><span class="p">(</span><span class="s">"image-server"</span><span class="p">),</span> <span class="n">opt</span><span class="p">,</span>
    <span class="p">)</span>

    <span class="k">var</span> <span class="n">buf</span> <span class="n">bytes</span><span class="o">.</span><span class="n">Buffer</span>
    <span class="n">dStream</span><span class="p">,</span> <span class="n">err</span> <span class="o">:=</span> <span class="n">bucket</span><span class="o">.</span><span class="n">DownloadToStream</span><span class="p">(</span><span class="n">objID</span><span class="p">,</span> <span class="o">&amp;</span><span class="n">buf</span><span class="p">)</span>
    <span class="k">if</span> <span class="n">err</span> <span class="o">!=</span> <span class="no">nil</span> <span class="p">{</span>
        <span class="n">log</span><span class="o">.</span><span class="n">Fatal</span><span class="p">(</span><span class="n">err</span><span class="p">)</span>
        <span class="n">c</span><span class="o">.</span><span class="n">JSON</span><span class="p">(</span><span class="n">http</span><span class="o">.</span><span class="n">StatusBadRequest</span><span class="p">,</span> <span class="n">err</span><span class="o">.</span><span class="n">Error</span><span class="p">())</span>
        <span class="k">return</span>
    <span class="p">}</span>

    <span class="n">log</span><span class="o">.</span><span class="n">Printf</span><span class="p">(</span><span class="s">"File size to download: %v</span><span class="se">\n</span><span class="s">"</span><span class="p">,</span> <span class="n">dStream</span><span class="p">)</span>
    <span class="n">contentType</span> <span class="o">:=</span> <span class="n">http</span><span class="o">.</span><span class="n">DetectContentType</span><span class="p">(</span><span class="n">buf</span><span class="o">.</span><span class="n">Bytes</span><span class="p">())</span>

    <span class="n">c</span><span class="o">.</span><span class="n">Writer</span><span class="o">.</span><span class="n">Header</span><span class="p">()</span><span class="o">.</span><span class="n">Add</span><span class="p">(</span><span class="s">"Content-Type"</span><span class="p">,</span> <span class="n">contentType</span><span class="p">)</span>
    <span class="n">c</span><span class="o">.</span><span class="n">Writer</span><span class="o">.</span><span class="n">Header</span><span class="p">()</span><span class="o">.</span><span class="n">Add</span><span class="p">(</span><span class="s">"Content-Length"</span><span class="p">,</span> <span class="n">strconv</span><span class="o">.</span><span class="n">Itoa</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">buf</span><span class="o">.</span><span class="n">Bytes</span><span class="p">())))</span>

    <span class="n">c</span><span class="o">.</span><span class="n">Writer</span><span class="o">.</span><span class="n">Write</span><span class="p">(</span><span class="n">buf</span><span class="o">.</span><span class="n">Bytes</span><span class="p">())</span>
<span class="p">}</span>



    

    




}

// Home endpoint to check server status

func homePage() gin.HandlerFunc {

    return func(c *gin.Context) {

        c.JSON(http.StatusOK, "Home page of Image Server")

    }

}

func Route(router *gin.Engine) {

    //All routes will be added here

    router.GET("/", homePage())

    router.POST("/upload", uploadImage())

    router.GET("/image/:imageId", serveImage())

}

func main() {

    port := os.Getenv("PORT")

    if port == "" {

        port = "8001"

    }

    router := gin.Default()

    Route(router)

<span class="n">router</span><span class="o">.</span><span class="n">Run</span><span class="p">(</span><span class="s">":"</span> <span class="o">+</span> <span class="n">port</span><span class="p">)</span>



    

    




}

😄 About Me

I’m Sitaram Rathi, a Full-stack Developer focused on back-end development. I'm pursuing my B.Tech+M.Tech Dual Degree in Computer Science & Engineering from NIT Hamirpur. I love working on projects and problems which make me push my limits and learn something new. You can connect with me here.

DEV Community

Uploading and Serving Images from MongoDB in Golang

👨🏻‍💻 Introduction

1. Gin Framework:

2. MongoDB and mongo-driver:

3. GridFS for Image Storage:

🔭 Methodology:

1. Image Uploading Endpoint:

2. Image Serving Endpoint:

⚙️ Project Setup:

Step 1: Setting Up the Project

Step 2: Environment Configuration

🚀 Endpoints:

1. Upload Image:

`http://localhost:8001/upload`

Method:

Purpose:

Functionality:

2. Get Image:

`http://localhost:8001/image/:imageId`

Method:

Purpose:

Functionality:

🤖 Code:

😄 About Me

Top comments (0)

Read next

𝗖𝗣𝗨 𝗧𝗵𝗿𝗼𝘁𝘁𝗹𝗶𝗻𝗴 𝗶𝗻 𝗞𝘂𝗯𝗲𝗿𝗻𝗲𝘁𝗲𝘀 ☸️

Decoding URL Encoding: Unveiling the Mystery Behind % Symbols

React Router- Params

𝗛𝗼𝘄 𝗕𝗮𝗰𝗸𝗲𝗻𝗱 𝗔𝗰𝗰𝗲𝗽𝘁 𝗖𝗼𝗻𝗻𝗲𝗰𝘁𝗶𝗼𝗻𝘀.