Creating a Face Swapping Application with Python and OpenCV

#python #beginners #tutorial #opencv

Introduction

Hello! 😎

In this tutorial I will teach you how to create a face-swapping application using Python, OpenCV and dlib.
Face swapping involves taking the face from one image and seamlessly blending it onto another face in a different image.
This tutorial is beginner-friendly and will guide you through the entire process. By the end you'll have a working face-swapping application and a good understanding of some essential image processing techniques.

Requirements

For this tutorial you will need to have the following installed:

Python
Pip (Python package installer)

Setting Up the Environment

First we will need to create a virtual environment for the project.
Create a new directory that will house our project via the following command:

mkdir face_swap && cd face_swap

Next create the virtual environment and activate it:

python3 -m venv env
source env/bin/activate

Now we need to install the packages required by this project, create a new file called "requirements.txt" and populate it with the following:

opencv-python
dlib
imutils
numpy

To install the required packages run the following command:

pip install -r requirements.txt

Additionally, you need to download the pre-trained shape predictor model for facial landmarks from dlib. Download the file from the following link and extract it into your project directory.

http://dlib.net/files/shape_predictor_68_face_landmarks.dat.bz2

Done! Now we are ready to write the code! 😆

Writing the Face Swapping Code

First create a new file called "main.py", we will start by importing the necessary libraries and defining a function to apply an affine transform:

import cv2
import dlib
import numpy as np
import imutils
from imutils import face_utils
import argparse

def apply_affine_transform(src, src_tri, dst_tri, size):
    warp_mat = cv2.getAffineTransform(np.float32(src_tri), np.float32(dst_tri))
    dst = cv2.warpAffine(src, warp_mat, (size[0], size[1]), None, flags=cv2.INTER_LINEAR, borderMode=cv2.BORDER_REFLECT_101)
    return dst

Next, we will define a function to warp the triangles from the source image to the destination image like so:

def warp_triangle(img1, img2, t1, t2):
    r1 = cv2.boundingRect(np.float32([t1]))
    r2 = cv2.boundingRect(np.float32([t2]))

    t1_rect = []
    t2_rect = []
    t2_rect_int = []

    for i in range(0, 3):
        t1_rect.append(((t1[i][0] - r1[0]), (t1[i][1] - r1[1])))
        t2_rect.append(((t2[i][0] - r2[0]), (t2[i][1] - r2[1])))
        t2_rect_int.append(((t2[i][0] - r2[0]), (t2[i][1] - r2[1])))

    img1_rect = img1[r1[1]:r1[1] + r1[3], r1[0]:r1[0] + r1[2]]

    size = (r2[2], r2[3])
    img2_rect = apply_affine_transform(img1_rect, t1_rect, t2_rect, size)

    mask = np.zeros((r2[3], r2[2], 3), dtype=np.float32)
    cv2.fillConvexPoly(mask, np.int32(t2_rect_int), (1.0, 1.0, 1.0), 16, 0)

    img2[r2[1]:r2[1] + r2[3], r2[0]:r2[0] + r2[2]] = img2[r2[1]:r2[1] + r2[3], r2[0]:r2[0] + r2[2]] * (1 - mask) + img2_rect * mask

Now, we will write the main face_swap function that will handle the face detection, landmark extraction and face swapping.

def face_swap(image1_path, image2_path):
    detector = dlib.get_frontal_face_detector()
    predictor = dlib.shape_predictor("shape_predictor_68_face_landmarks.dat")

    image1 = cv2.imread(image1_path)
    image2 = cv2.imread(image2_path)

    gray1 = cv2.cvtColor(image1, cv2.COLOR_BGR2GRAY)
    gray2 = cv2.cvtColor(image2, cv2.COLOR_BGR2GRAY)

    rects1 = detector(gray1, 1)
    rects2 = detector(gray2, 1)

    if len(rects1) == 0 or len(rects2) == 0:
        print("Error: Could not detect faces in one or both images.")
        return

    shape1 = predictor(gray1, rects1[0])
    shape2 = predictor(gray2, rects2[0])

    points1 = face_utils.shape_to_np(shape1)
    points2 = face_utils.shape_to_np(shape2)

    hullIndex = cv2.convexHull(points2, returnPoints=False)
    hull1 = points1[hullIndex[:, 0]]
    hull2 = points2[hullIndex[:, 0]]

    rect = (0, 0, gray2.shape[1], gray2.shape[0])
    subdiv = cv2.Subdiv2D(rect)
    subdiv.insert(hull2.tolist())
    triangles = subdiv.getTriangleList()
    triangles = np.array(triangles, dtype=np.int32)

    indexes_triangles = []
    for t in triangles:
        pts = [(t[0], t[1]), (t[2], t[3]), (t[4], t[5])]
        indices = []
        for pt in pts:
            ind = np.where((hull2 == pt).all(axis=1))
            if len(ind[0]) == 0:
                continue
            indices.append(ind[0][0])
        indexes_triangles.append(indices)

    img2_new_face = np.zeros_like(image2)

    for indices in indexes_triangles:
        t1 = [hull1[indices[0]], hull1[indices[1]], hull1[indices[2]]]
        t2 = [hull2[indices[0]], hull2[indices[1]], hull2[indices[2]]]

        warp_triangle(image1, img2_new_face, t1, t2)

    mask = np.zeros_like(gray2)
    cv2.fillConvexPoly(mask, np.int32(hull2), (255, 255, 255))

    r = cv2.boundingRect(np.float32([hull2]))
    center = (r[0] + int(r[2] / 2), r[1] + int(r[3] / 2))
    output = cv2.seamlessClone(img2_new_face, image2, mask, center, cv2.NORMAL_CLONE)

    cv2.imwrite("output.jpg", output)

Next to wrap up the code for the application, we will add the command line argument parsing and the main function in order to run our script:

def main():
    parser = argparse.ArgumentParser(description="Face Swapping Application")
    parser.add_argument("image1", type=str, help="Path to the first image (source face)")
    parser.add_argument("image2", type=str, help="Path to the second image (destination face)")

    args = parser.parse_args()

    face_swap(args.image1, args.image2)

if __name__ == "__main__":
    main()

That's the end of the code, now we can actually run our application! 👀

Running the Application

In order to run our application, you will need to images in order to perform the face swap. Once you have two images run the above script with the following command:

python main.py [image1] [image2]

Replace image1 and image2 with the paths to your images. The script will detect faces in the images, swap them, and then save the new image as "output.jpg".

Once the command has run checkout the new "output.jpg". 🥸

Original:

Output:

Conclusion

In this tutorial I have shown how to use Python, OpenCV and dlib to swap faces. This example is pretty simple, so it may not work right with multiple faces. I hope this tutorial has taught you something as I certainly had fun making it.

If you know of any ways to further refine the face swapping, please tell me.

As always the code for this example can be found on my Github:
https://github.com/ethand91/face-swap

Happy Coding! 😎

Like my work? I post about a variety of topics, if you would like to see more please like and follow me.
Also I love coffee.