Prathamesh More

Posted on Jul 5, 2021

How to Create a YouTube Downloader with Node.js and React.js

#javascript #node #react #youtubedownloader

The basic flow of the app:

User will provide a YouTube video link
The backend server will push this video link in the queue to process the download
When the job popped from the queue for processing, the backend emits the event for the client
The client listens to the event and shows appropriate messages
Users will able to download videos from a server

We will use Socket.io for the emitting events and processing and handling jobs will use the Bull package.

Let's begin,

Install required software and packages on your local machine

Software Requirements:

Node.js - Node.js® is a JavaScript runtime built on Chrome's V8 JavaScript engine.
Postman - A collaboration platform for API development.

Require packages:

Backend packages:

npm i typescript express mongoose cors express-validator mongoose morgan socket.io ytdl-core bull dotenv

Frontend packages:

npm i axios js-file-download socket.io-client

Setup Backend:

We will be using the MongoDB database so make sure you install it locally or you can use free cloud service from MongoDB.

Set up Redis database with Upstash:

Upstash is a serverless database for Redis. With servers/instances, you usually pay per hour or at a fixed price. With serverless, you pay-per-request.

This means you're not charged when the database isn't in use. Upstash configures and manages the database for you.

Start by creating an account on Upstash.

Now set up the Redis database instance

Let's initialize TypeScript based Node.js project,



tsc --init
then do
npm init -y

Don't forget to add .env file and its content.

Create a new src directory in the root directory of the project as shown in the above image.

Create a simple server and connect to the local or remote MongoDB database:



import { config } from "dotenv";
config();
import http from "http";
import express, { Request, Response } from "express";
import { Server } from "socket.io";
import mongoose from "mongoose";
import cors from "cors";
import path from "path";
import morgan from "morgan";
import { SocketInit } from "./socket.io";

const app = express();

const server = http.createServer(app);

export const io = new Server(server, {
  cors: { origin: "*" },
});

new SocketInit(io);

mongoose
  .connect(process.env.MONGO_DB, {
    useNewUrlParser: true,
    useUnifiedTopology: true,
  })
  .then(() => {
    console.log("Connected to database");
  })
  .catch((error) => {
    throw error;
  });

app.use(morgan("dev"));
app.use(express.json());
app.use(express.urlencoded({ extended: true }));
app.use(cors());

app.get("/", (req: Request, res: Response) => {
  res.status(200).send('ok')
});

server.listen(3000, () => {
  console.log("Server running up 3000");
});

Now, create a mongoose model for store video metadata, this file will reside in src/models.



import mongoose from "mongoose";

export interface VideoDoc extends mongoose.Document {
  title: string;
  file: string;
  thumbnail: string;
}

const videoSchema = new mongoose.Schema(
  {
    title: {
      type: String,
      required: true,
    },
    file: {
      type: String,
      required: true,
    },
    thumbnail: {
      type: String,
    },
  },
  { timestamps: true }
);

export const Video = mongoose.model<VideoDoc>("video", videoSchema);

REST APIs



REST APIs Routes
1. GET => /api/donwloads => Get all downloads
2. GET => /api/donwloads/:id => Get a single download
3. POST => /api/downloads => Push new download
4. DELETE => /api/downloads/:id => Remove a single download
5. GET => /api/downloads/:id/downloadfile => Download a single file

Let's implement controllers and routes for APIs,



import express, { Request, Response, NextFunction } from "express";
import fs from "fs/promises";
import { Video } from "../models/video";
const downloadsRouter = express.Router();

downloadsRouter.get(
  "/api/downloads",
  async (req: Request, res: Response, next: NextFunction) => {
    const videos = await Video.find().sort({ createdAt: -1 });
    res.status(200).send(videos);
  }
);

downloadsRouter.get(
  "/api/downloads/:id/downloadfile",
  async (req: Request, res: Response, next: NextFunction) => {
    const { id } = req.params;
    const video = await Video.findById(id);

    if (!video) {
      res.status(404).send("Video not found");
    }
    const { file } = video;

    res.status(200).download(file);
  }
);

downloadsRouter.post(
  "/api/downloads",
  body("youtubeUrl").isURL(),
  async (req: Request, res: Response, next: NextFunction) => {
    //Will implement
  }
);

downloadsRouter.delete(
  "/api/downloads/:id",
  async (req: Request, res: Response, next: NextFunction) => {
    const { id } = req.params;

    const video = await Video.findByIdAndDelete(id);

    if (video) {
      await fs.unlink(video.file!);
    }
    res.status(200).send(video);
  }
);

export { downloadsRouter };

Now here comes the most important task,

This is section will implement a download queue using Bull Queue.

However every queue instance will require new Redis connections.

This queue will process all downloads one by one.

On each job process, we are emitting events for the client.



import Bull from "bull";
import ytdl from "ytdl-core";
import fs from "fs";
import { Video } from "../models/video";
import { Events } from "../utils";
import { SocketInit } from "../socket.io";

const downloadQueue = new Bull("download queue", {
  redis: {
    host: process.env.REDIS_HOST!,
    port: parseInt(process.env.REDIS_PORT!),
    password: process.env.REDIS_PASSWORD
  },
});

downloadQueue.process((job, done) => {
  return new Promise(async (resolve, reject) => {
    const { youtubeUrl } = job.data;

    //Get singleton instance
    const socket = SocketInit.getInstance();

    const info = await ytdl.getBasicInfo(youtubeUrl);

    console.log(info.videoDetails.thumbnails[0].url);

    const thumbnail = info.videoDetails.thumbnails[0].url;

    //Appending some randome string at the end of file name so it should be unique while storing on server's disk
    const title =
      info.videoDetails.title +
      " by " +
      info.videoDetails.author.name +
      "-" +
      new Date().getTime().toString();

    ytdl(youtubeUrl)
      .pipe(fs.createWriteStream(`${process.cwd()}/downloads/${title}.mp4`))
      .on("finish", async () => {
        socket.publishEvent(Events.VIDEO_DOWNLOADED, title);

        const file = `${process.cwd()}/downloads/${title}.mp4`;

        const video = new Video({
          title,
          file,
          thumbnail,
        });

        await video.save();

        done();

        resolve({ title });
      })
      .on("ready", () => {
        socket.publishEvent(Events.VIDEO_STARTED, title);
      })
      .on("error", (error) => {
        socket.publishEvent(Events.VIDEO_ERROR, error);
        done(error);
        reject(error);
      });
  });
});

export { downloadQueue };



export enum Events {
  VIDEO_DOWNLOADED = "VIDEO_DOWNLOADED",
  VIDEO_STARTED = "VIDEO_STARTED",
  VIDEO_ERROR = "VIDEO_ERROR",
}

Whenever a user tries to download a video, we first push that job i.e. link in download queue.

Then we request for Socket.io instance and video's metadata like title and thumbnail.



//Get existing instance
const socket = SocketInit.getInstance();
const info = await ytdl.getBasicInfo(youtubeUrl);
const thumbnail = info.videoDetails.thumbnails[0].url;

Using ytdl package, we start downloading the file and storing it in a directory called downloads in the root of the project.

When the download starts we emit event VIDEO_STARTED with a title as data.

When the download completes we emit event VIDEO_DOWNLOADED.

When the download gets failed due to some reason like private video or copyright content, we emit event VIDEO_ERROR.

Now import this queue module in the controller, also we added some validation on the request body.



import express, { Request, Response, NextFunction } from "express";
import fs from "fs/promises";
import { body, validationResult } from "express-validator";
import { downloadQueue } from "../queues/download-queue";
import { Video } from "../models/video";
const downloadsRouter = express.Router();

downloadsRouter.get(
  "/api/downloads",
  async (req: Request, res: Response, next: NextFunction) => {
    const videos = await Video.find().sort({ createdAt: -1 });
    res.status(200).send(videos);
  }
);

downloadsRouter.get(
  "/api/downloads/:id/downloadfile",
  async (req: Request, res: Response, next: NextFunction) => {
    const { id } = req.params;
    const video = await Video.findById(id);

    if (!video) {
      res.status(404).send("Video not found");
    }
    const { file } = video;

    res.status(200).download(file);
  }
);

downloadsRouter.post(
  "/api/downloads",
  body("youtubeUrl").isURL(),
  async (req: Request, res: Response, next: NextFunction) => {
    try {
      const errors = validationResult(req);
      if (!errors.isEmpty()) {
        return res.status(400).json({ errors: errors.array() });
      }
      const { youtubeUrl } = req.body;
      await downloadQueue.add({ youtubeUrl });
      res.status(200).send("Downloading");
    } catch (error) {
      throw error;
    }
  }
);

downloadsRouter.delete(
  "/api/downloads/:id",
  async (req: Request, res: Response, next: NextFunction) => {
    const { id } = req.params;

    const video = await Video.findByIdAndDelete(id);


    if (video) {
      await fs.unlink(video.file!);
    }
    res.status(200).send(video);
  }
);

export { downloadsRouter };

Finally we can add this controller in server.ts file,



import { config } from "dotenv";
config();
import http from "http";
import express, { Request, Response } from "express";
import { Server } from "socket.io";
import mongoose from "mongoose";
import cors from "cors";
import path from "path";
import morgan from "morgan";
import { SocketInit } from "./socket.io";
import { downloadsRouter } from "./routes/downloads";

const app = express();

const server = http.createServer(app);

export const io = new Server(server, {
  cors: { origin: "*" },
});

new SocketInit(io);

mongoose
  .connect(process.env.MONGO_DB, {
    useNewUrlParser: true,
    useUnifiedTopology: true,
  })
  .then(() => {
    console.log("Connected to database");
  })
  .catch((error) => {
    throw error;
  });

app.use(morgan("dev"));
app.use(express.json());
app.use(express.urlencoded({ extended: true }));
app.set("view engine", "ejs");
app.use(express.static(path.join(__dirname, "views")));
app.use(cors());
app.use(downloadsRouter);

app.get("/", (req: Request, res: Response) => {
  res.render("index");
});

server.listen(3000, () => {
  console.log("Server running up 3000");
});

Finally, change scripts in package.json



"scripts": {   
   "start": "ts-node src/server.ts",
   "dev": "ts-node-dev src/server.ts"
}

Now test with Postman,

POST => /api/downloads

GET => /api/downloads

Setup Frontend:

Create boilerplate code for React by running the following command:

npx create-react-app fronend && cd frontend

The folder structure looks like after running the command,

Then we just added Components directory, we have there three components

Now add Bootstrap for UI:

Design basic nav bar:



import React from "react";

export default function Navbar() {
  return (
    <header class="pb-3 mb-4 border-bottom">
      <a
        href="/"
        class="d-flex align-items-center text-dark text-decoration-none"
      >
        <svg
          xmlns="http://www.w3.org/2000/svg"
          width="50"
          height="50"
          fill="currentColor"
          class="bi bi-youtube"
          viewBox="0 0 16 16"
        >
          <path d="M8.051 1.999h.089c.822.003 4.987.033 6.11.335a2.01 2.01 0 0 1 1.415 1.42c.101.38.172.883.22 1.402l.01.104.022.26.008.104c.065.914.073 1.77.074 1.957v.075c-.001.194-.01 1.108-.082 2.06l-.008.105-.009.104c-.05.572-.124 1.14-.235 1.558a2.007 2.007 0 0 1-1.415 1.42c-1.16.312-5.569.334-6.18.335h-.142c-.309 0-1.587-.006-2.927-.052l-.17-.006-.087-.004-.171-.007-.171-.007c-1.11-.049-2.167-.128-2.654-.26a2.007 2.007 0 0 1-1.415-1.419c-.111-.417-.185-.986-.235-1.558L.09 9.82l-.008-.104A31.4 31.4 0 0 1 0 7.68v-.123c.002-.215.01-.958.064-1.778l.007-.103.003-.052.008-.104.022-.26.01-.104c.048-.519.119-1.023.22-1.402a2.007 2.007 0 0 1 1.415-1.42c.487-.13 1.544-.21 2.654-.26l.17-.007.172-.006.086-.003.171-.007A99.788 99.788 0 0 1 7.858 2h.193zM6.4 5.209v4.818l4.157-2.408L6.4 5.209z" />
        </svg>
        <span className="fs-4">YouTube Downloader</span>
      </a>
    </header>
  );
}

Now integrate all download APIs in Home.js component,

Here we making connection with server using socketio-client for events and aslo making HTTP request for data.



import React, { useEffect, useState } from "react";
import axios from "axios";
import toast, { Toaster } from "react-hot-toast";
import { io } from "socket.io-client";
import Videos from "./Videos";

const notify = (msg, { success }) => {
  if (success) {
    return toast.success(msg);
  }
  return toast.error(msg);
};

const socket = io("http://localhost:3000/");

export default function Home() {
  const [videos, setVideos] = useState([]);

  useEffect(() => {
    socket.on("VIDEO_DOWNLOADED", (data) => {
      notify(`${data} Downloaded`, { success: true });
      window.location.reload();
    });

    socket.on("VIDEO_STARTED", (data) => {
      notify(`Download Started ${data}`, { success: true });
    });

    axios
      .get("http://localhost:3000/api/downloads")
      .then((res) => {
        setVideos(res.data);
      })
      .catch((error) => {
        console.log(error);
      });
  }, []);

  const downloadVideo = (event) => {
    event.preventDefault();

    const youtubeUrl = event.target.elements.youtubeUrl.value;

    axios
      .post("http://localhost:3000/api/downloads", { youtubeUrl })
      .then((res) => {
        notify("Fetching video details...", { success: true });
      })
      .catch((error) => {
        notify("Something went wrong", { success: false });
      });
  };

  return (
    <div>
      <div class="p-5 mb-4 bg-light rounded-3">
        <div class="container-fluid py-5">
          <h1 class="display-5 fw-bold">
            Download your favorite Youtube videos
          </h1>
        </div>
        <form onSubmit={downloadVideo}>
          <div>
            <label for="youtubeUrl" class="form-label">
              Enter link
            </label>
            <input type="url" id="youtubeUrl" class="form-control" required />
            <div id="urlHelpBlock" class="form-text">
              E.g. https://www.youtube.com/watch?v=PCicKydX5GE
            </div>
            <br />
            <button type="submit" class="btn btn-primary btn-lg">
              Download
            </button>
            <Toaster />
          </div>
        </form>
      </div>
      <h3>Downloaded videos</h3>
      <div style={{ margin: 10 }} className="row">
        {videos.map((video) => {
          return <Videos video={video} />;
        })}
      </div>
    </div>
  );
}

Now let's implement Video.js component to render each single video and related operation,



import axios from "axios";
import React from "react";
const FileDownload = require("js-file-download");

export default function VideoDownloader(props) {
  console.log(props);
  const { video } = props;
  const { _id, title, thumbnail } = video;

  const downloadVideo = async (event) => {
    const videoId = event.target.id;
    const filename = event.target.title;
    console.log(filename);
    axios
      .get("http://localhost:3000/api/downloads/" + videoId + "/downloadfile", {
        responseType: "blob",
      })
      .then((response) => {
        FileDownload(response.data, `${filename}.mp4`);
      });
  };

  const removeVideo = async (event) => {
    const videoId = event.target.title;
    axios
      .delete("http://localhost:3000/api/downloads/" + videoId)
      .then((respsonse) => {
        window.location.reload();
      });
  };

  return (
    <div className="card" style={{ width: "18rem" }}>
      <img src={thumbnail} class="card-img-top" alt="thumbnail" />
      <div className="card-body">
        <h6 className="card-text">{title}</h6>
        <button
          id={_id}
          className="btn btn-success rounded"
          style={{ width: "100px" }}
          onClick={downloadVideo}
          title={title}
        >
          Download
        </button>
        <button
          title={_id}
          className="btn btn-danger rounded"
          onClick={removeVideo}
        >
          Delete
        </button>
      </div>
    </div>
  );
}

Now let's run both frontend and backend code,

Backend will run on 3000 port => npm run dev

Frontend will run on 3001 port => npm start

Source Code

Check out Upstash for production.

Top comments (7)

Utkarsh Kumar Raut • Jul 6 '21

npm install "thanks-for-share"

Kamal Hossain • Jan 2 '22

Hey there,
Thanks for sharing this.
I have two qustions, first ytdl-core has download speed issues, and sometimes the download stuck after downloading some part of it, do you have any solution plan for it?
Second, why did you created the the queue? Why is that important?
Thak you.

Prathamesh More • Nov 14 '22

No solution
Queue will help to download files in the background. If server went down, and restarted it will all list of pending downloads in queue. Once server came back, it will start processing remaining downloads.