Computer Vision Project - Face Recognition and Detection

Click on the above“Beginner Learning Vision”, select to add “Star” or “Pin”

Heavyweight content delivered first time Computer Vision Project - Face Recognition and Detection

Author | Cat Eating Fish python @CSDN

Editor | 3D Vision Developer Community

Contents

Introduction

content

1. Project Introduction and Preface

2. Recognition and Detection Methods

Methods in this article
Project Analysis

3. Complete Code and Effect Display

Computer Vision Project - Face Recognition and Detection

1. Project Introduction and Preface

Face recognition, as a biometric identification technology, has advantages such as non-invasiveness, non-contact, friendliness, and convenience.The general process of face recognition mainly includes face detection, face cropping, face alignment, feature extraction, and face recognition.Face detection removes interference from the acquired image, extracts face information, obtains the location of the face image, and the success rate of detection is mainly affected by factors such as image quality, lighting, and occlusion.The picture below shows the entire face detection process.

2. Recognition and Detection Methods

Traditional recognition methods (1) Face recognition based on point cloud data (2) 3D face recognition based on facial features

Deep learning recognition methods (1) Face recognition based on depth maps (2) Face recognition based on RGB-3DMM (3) Face recognition based on RGB-D

🌙Methods in this Article

Overview of key point localization: Generally, there are 5 key points in a face, including two eyes, one nose, and two mouth corners. It can also be detailed into 68 key points, which provides a more comprehensive overview. Our research focuses on 68 key point localization.

The above image shows the 68 key points we located on the face, where the order must be strictly followed. The sequence from point 1 to 68 cannot be incorrect.

🌙Project Analysis

We use the machine learning framework dlib for this project. First, when specifying parameters, we need to find the 68 key points for face localization in dlib. Set the 68 key points for face localization.

from collections import OrderedDict
import numpy as np
import argparse
import dlib
import cv2

First, we import the libraries. The dlib library can be downloaded from this website http://dlib.net/files/. Then we import the parameters.

ap = argparse.ArgumentParser()
ap.add_argument("-p", "--shape-predictor", required=True,  help="path to facial landmark predictor")
ap.add_argument("-i", "--image", required=True,  help="path to input image")
args = vars(ap.parse_args())

Here we need to set the parameters,

–shape-predictor shape_predictor_68_face_landmarks.dat –image images/lanqiudui.jpg. If there are multiple faces in one image, we will detect and crop the corresponding ROI areas separately. Our overall approach is to first detect the location of the face area, then detect the position of the nose relative to the face frame, for example, the left eye of the person is at (0.2w, 0.2h) in the face frame.

FACIAL_LANDMARKS_68_IDXS = OrderedDict([
  ("mouth", (48, 68)),
  ("right_eyebrow", (17, 22)),
  ("left_eyebrow", (22, 27)),
  ("right_eye", (36, 42)),
  ("left_eye", (42, 48)),
  ("nose", (27, 36)),
  ("jaw", (0, 17))
])

This defines the positions of the 68 key points relative to the face frame, corresponding to the mouth, left eye, right eye, left eyebrow, right eyebrow, nose, and jaw.

FACIAL_LANDMARKS_5_IDXS = OrderedDict([
  ("right_eye", (2, 3)),
  ("left_eye", (0, 1)),
  ("nose", (4))
])

If it is 5-point localization, then we need to locate the left eye, right eye, and nose. Points 0, 1, 2, 3, and 4 correspond to the 5 points.

detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor(args["shape_predictor"])

Load the face detection and key point localization. The detector is the default face detector. Then, by passing in parameters, it returns the 4-point coordinates of the face detection rectangle. The predictor takes a region of the image as input and outputs a series of points (point location) to indicate the pose of the object in that region. It returns the trained face 68 feature point detector.

image = cv2.imread(args["image"])
(h, w) = image.shape[:2]
width=500
r = width / float(w)
dim = (width, int(h * r))
image = cv2.resize(image, dim, interpolation=cv2.INTER_AREA)
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

Here we read in the data, then process it to extract h and w, where we set the width of the image to 500, and then proportionally set h. Then we perform a resize operation, and finally convert it to a grayscale image.

rects = detector(gray, 1)

Here we call the detector face frame detector, which needs to use the grayscale image for detection, and the 1 is the number of resampling. The returned result is the 4-point coordinates of the face detection rectangle. Then we iterate through the detection boxes.

for (i, rect) in enumerate(rects):  # Perform key point localization on the face frame
  # Convert to ndarray
  shape = predictor(gray, rect)
  shape = shape_to_np(shape)

This returns the 68 key point locations. The shape_to_np function is as follows.

def shape_to_np(shape, dtype="int"):
  # Create 68*2
  coords = np.zeros((shape.num_parts, 2), dtype=dtype)
  # Iterate through each key point
  # Get coordinates
  for i in range(0, shape.num_parts):
    coords[i] = (shape.part(i).x, shape.part(i).y)
  return coords

The shape_to_np function serves to obtain the coordinates of the key point locations.

for (name, (i, j)) in FACIAL_LANDMARKS_68_IDXS.items():
    clone = image.copy()
    cv2.putText(clone, name, (10, 30), cv2.FONT_HERSHEY_SIMPLEX,
      0.7, (0, 0, 255), 2)
    # Draw points based on positions
    for (x, y) in shape[i:j]:
      cv2.circle(clone, (x, y), 3, (0, 0, 255), -1)
    # Extract ROI area
    (x, y, w, h) = cv2.boundingRect(np.array([shape[i:j]]))
    roi = image[y:y + h, x:x + w]
    (h, w) = roi.shape[:2]
    width=250
    r = width / float(w)
    dim = (width, int(h * r))
    roi = cv2.resize(roi, dim, interpolation=cv2.INTER_AREA)
    # Display each part
    cv2.imshow("ROI", roi)
    cv2.imshow("Image", clone)
    cv2.waitKey(0)

Here the dictionary FACIAL_LANDMARKS_68_IDXS.items() extracts both the key and value from the dictionary. Then it iterates through these areas, displays which area it is, and draws a circle in that area. Afterward, it extracts the ROI area and displays it. The latter part shows the width and height displayed proportionally. Then it shows the result.

output = visualize_facial_landmarks(image, shape)
cv2.imshow("Image", output)
cv2.waitKey(0)

Finally, display all areas. The visualize_facial_landmarks function is:

def visualize_facial_landmarks(image, shape, colors=None, alpha=0.75):
  # Create two copies
  # overlay and one for the final output image
  overlay = image.copy()
  output = image.copy()
  # Set some color areas
  if colors is None:
    colors = [(19, 199, 109), (79, 76, 240), (230, 159, 23),
      (168, 100, 168), (158, 163, 32),
      (163, 38, 32), (180, 42, 220)]
  # Iterate through each area
  for (i, name) in enumerate(FACIAL_LANDMARKS_68_IDXS.keys()):
    # Get the coordinates of each point
    (j, k) = FACIAL_LANDMARKS_68_IDXS[name]
    pts = shape[j:k]
    # Check position
    if name == "jaw":
      # Connect with lines
      for l in range(1, len(pts)):
        ptA = tuple(pts[l - 1])
        ptB = tuple(pts[l])
        cv2.line(overlay, ptA, ptB, colors[i], 2)
    # Calculate convex hull
    else:
      hull = cv2.convexHull(pts)
      cv2.drawContours(overlay, [hull], -1, colors[i], -1)
  # Overlay on the original image, can specify ratio
  cv2.addWeighted(overlay, alpha, output, 1 - alpha, 0, output)
  return output

This function computes the cv2.convexHull, which means the following image.

This function cv2.addWeighted is used for image overlay.

src1, src2: The two images to be fused and overlaid, requiring the same size and channel number

alpha: The weight of src1

beta: The weight of src2

gamma: The correction factor for brightness, set to 0 if no correction is needed

dst: Optional parameter, variable to save the output result, default value is None

dtype: Optional parameter, the depth of the output image array, i.e., the number of bits for a single pixel value (e.g., RGB is represented by three bytes, so it is 24 bits), the default value None indicates that it remains consistent with the source image.

dst = src1 × alpha + src2 × beta + gamma; The above formula can be understood as the resulting image = Image 1 × Coefficient 1 + Image 2 × Coefficient 2 + Brightness adjustment amount.

3. Complete Code and Effect Display

from collections import OrderedDict
import numpy as np
import argparse
import dlib
import cv2
ap = argparse.ArgumentParser()
ap.add_argument("-p", "--shape-predictor", required=True,  help="path to facial landmark predictor")
ap.add_argument("-i", "--image", required=True,  help="path to input image")
args = vars(ap.parse_args())
FACIAL_LANDMARKS_68_IDXS = OrderedDict([
  ("mouth", (48, 68)),
  ("right_eyebrow", (17, 22)),
  ("left_eyebrow", (22, 27)),
  ("right_eye", (36, 42)),
  ("left_eye", (42, 48)),
  ("nose", (27, 36)),
  ("jaw", (0, 17))
])
FACIAL_LANDMARKS_5_IDXS = OrderedDict([
  ("right_eye", (2, 3)),
  ("left_eye", (0, 1)),
  ("nose", (4))
])
def shape_to_np(shape, dtype="int"):
  # Create 68*2
  coords = np.zeros((shape.num_parts, 2), dtype=dtype)
  # Iterate through each key point
  # Get coordinates
  for i in range(0, shape.num_parts):
    coords[i] = (shape.part(i).x, shape.part(i).y)
  return coords
def visualize_facial_landmarks(image, shape, colors=None, alpha=0.75):
  # Create two copies
  # overlay and one for the final output image
  overlay = image.copy()
  output = image.copy()
  # Set some color areas
  if colors is None:
    colors = [(19, 199, 109), (79, 76, 240), (230, 159, 23),
      (168, 100, 168), (158, 163, 32),
      (163, 38, 32), (180, 42, 220)]
  # Iterate through each area
  for (i, name) in enumerate(FACIAL_LANDMARKS_68_IDXS.keys()):
    # Get the coordinates of each point
    (j, k) = FACIAL_LANDMARKS_68_IDXS[name]
    pts = shape[j:k]
    # Check position
    if name == "jaw":
      # Connect with lines
      for l in range(1, len(pts)):
        ptA = tuple(pts[l - 1])
        ptB = tuple(pts[l])
        cv2.line(overlay, ptA, ptB, colors[i], 2)
    # Calculate convex hull
    else:
      hull = cv2.convexHull(pts)
      cv2.drawContours(overlay, [hull], -1, colors[i], -1)
  # Overlay on the original image, can specify ratio
  cv2.addWeighted(overlay, alpha, output, 1 - alpha, 0, output)
  return output
# Load face detection and key point localization
detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor(args["shape_predictor"])
# Read input data, preprocess
image = cv2.imread(args["image"])
(h, w) = image.shape[:2]
width=500
r = width / float(w)
dim = (width, int(h * r))
image = cv2.resize(image, dim, interpolation=cv2.INTER_AREA)
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
# Face detection
rects = detector(gray, 1)
# Iterate through the detected boxes
for (i, rect) in enumerate(rects):  # Perform key point localization on the face frame
  # Convert to ndarray
  shape = predictor(gray, rect)
  shape = shape_to_np(shape)
  # Iterate through each part
  for (name, (i, j)) in FACIAL_LANDMARKS_68_IDXS.items():
    clone = image.copy()
    cv2.putText(clone, name, (10, 30), cv2.FONT_HERSHEY_SIMPLEX,
      0.7, (0, 0, 255), 2)
    # Draw points based on positions
    for (x, y) in shape[i:j]:
      cv2.circle(clone, (x, y), 3, (0, 0, 255), -1)
    # Extract ROI area
    (x, y, w, h) = cv2.boundingRect(np.array([shape[i:j]]))
    roi = image[y:y + h, x:x + w]
    (h, w) = roi.shape[:2]
    width=250
    r = width / float(w)
    dim = (width, int(h * r))
    roi = cv2.resize(roi, dim, interpolation=cv2.INTER_AREA)
    # Display each part
    cv2.imshow("ROI", roi)
    cv2.imshow("Image", clone)
    cv2.waitKey(0)
  # Display all areas
  output = visualize_facial_landmarks(image, shape)
  cv2.imshow("Image", output)
  cv2.waitKey(0)

Finally, the faces of 7 people were detected in sequence, and the key points were located.

This article is for academic sharing only. If there is any infringement, please contact to delete the article.

Good news! 
The Beginner Learning Vision Knowledge Planet is now open to the public 👇👇👇 





Download 1: OpenCV-Contrib Extension Module Chinese Version Tutorial
Reply "Extension Module Chinese Tutorial" in the background of the "Beginner Learning Vision" public account to download the first OpenCV extension module tutorial in Chinese, covering over twenty chapters including extension module installation, SFM algorithms, stereo vision, object tracking, biological vision, super-resolution processing, etc. 

Download 2: Python Vision Practical Project 52 Lectures
Reply "Python Vision Practical Project" in the background of the "Beginner Learning Vision" public account to download 31 practical vision projects including image segmentation, mask detection, lane line detection, vehicle counting, eyeliner addition, license plate recognition, character recognition, emotion detection, text content extraction, and face recognition to help quickly learn computer vision. 

Download 3: OpenCV Practical Project 20 Lectures
Reply "OpenCV Practical Project 20 Lectures" in the background of the "Beginner Learning Vision" public account to download 20 practical projects based on OpenCV to achieve advanced learning of OpenCV. 

Group Chat

Welcome to join the reader group of the public account to communicate with peers. Currently, there are WeChat groups for SLAM, 3D vision, sensors, autonomous driving, computational photography, detection, segmentation, recognition, medical imaging, GAN, algorithm competitions, etc. (will gradually be subdivided later). Please scan the WeChat ID below to join the group, and note: "Nickname + School/Company + Research Direction", for example: "Zhang San + Shanghai Jiao Tong University + Vision SLAM". Please follow the format for notes, otherwise, you will not be approved. After successful addition, you will be invited to the relevant WeChat group based on your research direction. Please do not send advertisements in the group, otherwise, you will be asked to leave the group. Thank you for your understanding~

Computer Vision Project – Face Recognition and Detection

Leave a Comment Cancel reply