Build a Realtime Object Detection Web App in 30 Minutes

Image Credit: https://github.com/tensorflow/models/tree/master/research/object_detection

TENSORFLOW.JS

Tensorflow.js is an open-source library enabling us to define, train and run machine learning models in the browser, using Javascript. I will use the Tensorflow.js framework in Angular to build a Web App that detects multiple objects on a webcam video feed.

COCO-SSD MODEL

First, we have to select the pre-trained model which we are going to use for object detection. Tensorflow.js provides several pre-trained models for classification, pose estimation, speech recognition and object detection purposes. Check out all the Tensoflow.js pre-trained models for more information.

COCO-SSD model, which is a pre-trained object detection model that aims to localize and identify multiple objects in an image, is the one we will use for object detection.

Photo by Brooke Cagle on Unsplash

The original ssd_mobilenet_v2_coco model size is 187.8 MB and can be downloaded from the TensorFlow model zoo. Compared to the original model, the Tensorflow.js version of the model is very lightweight and optimized for browser execution.

The default object detection model for Tensorflow.js COCO-SSD is ‘lite_mobilenet_v2’ which is very small in size, under 1MB, and fastest in inference speed. If you would like better classification accuracy you can use ‘mobilenet_v2’ , in this case, the size of the model increases to 75 MB, which is not suitable for the web-browser experience.

‘model.detect’takes image or video inputs directly from HTML, so you do not need to convert the inputs into tensors before using them. It returns an array of classes, probability scores as well as bounding box coordinates for the detected objects.

model.detect(
 img: tf.Tensor3D | ImageData | HTMLImageElement |
   HTMLCanvasElement | HTMLVideoElement, maxDetectionSize: number
)
[{
 bbox: [x, y, width, height],
 class: "person",
 score: 0.8380282521247864
}, {
 bbox: [x, y, width, height],
 class: "kite",
 score: 0.74644153267145157
}]

Photo by Marc Kleen on Unsplash

ANGULAR WEB APP INITIALIZING

After we all clear about the model, it is time to use the Angular command-line interface to initialize the Angular web application.

npm install -g @angular/cli
ng new TFJS-ObjectDetection
cd TFJS-ObjectDetection

Then I will use the NMP package manager to load Tensorflow.js and COCO-SSD library.

TFJS-ObjectDetection npm install @tensorflow/tfjs --save
TFJS-ObjectDetection npm install @tensorflow-models/coco-ssd --save

It is all set now. So we can start coding. I will start by importing the COCO-SSD model in ‘app.component.ts’ .

import { Component, OnInit } from '@angular/core';//import COCO-SSD model as cocoSSD
import * as cocoSSD from '@tensorflow-models/coco-ssd';