Vehicle color recognition (VCR) is vital in intelligent traffic management and criminal investigation assistance. However, the existing vehicle color data sets only cover 13 classes, which cannot meet the current actual demand. Besides, although much effort is devoted to VCR, it suffers from the problem of class imbalance in data sets.

After investigating the issue, a research team led by Mingdi Hu published its new research in Frontiers of Computer Science.

The team proposes a novel VCR method. They present a new VCR data set with 24 vehicle classes, Vehicle Color-24, and propose the SMNN-MSFF model with multi-scale feature fusion and smooth modulation.

The former aims to extract feature information from local to global, and the latter could increase the loss of the images of tail class instances for training with class-imbalance. Extensive ablation studies demonstrate that each module of the proposed method is effective, especially the smooth modulation, which efficiently assists feature learning of the minority or tail classes.

Comprehensive experimental evaluation on Vehicle Color-24 and three previous representative data sets demonstrated that the proposed SMNN-MSFF outperformed state-of-the-art VCR methods.

Within the research, the team built a new data set with 24 vehicle colors, called Vehicle Color-24. The colors include red, dark-red, pink, orange, dark-orange, red-orange, yellow, lemon-yellow, earthy-yellow, green, dark-green, grass-green, cyan, blue, dark-blue, purple, black, white, silver-gray, gray, dark-gray, champagne, brown and dark-brown. Vehicle Color-24 can make up for the current needs of practical vehicle traffic management and criminal vehicle tracking applications.

Then, the researchers proposed a novel vehicle color recognition method based on SMNN-MSFF. This algorithm focuses attention on the color distribution imbalance nature existing in any data set. The loss function fine-tunes the network so that the algorithm can better capture the characteristics of small-scale classes than focal loss through ablation experiments.

This network also adds an FPN module to extract edge and corner information, which aids in extracting vehicle shape features and local location information to assist in vehicle recognition. Additionally, this backbone network is designed with only 42 layers, which belong to a lightweight network, to relieve the pressure of storage and increase the possibility of implementation in practical applications.

The experimental results show that the map of this method in the paper is 94.96% accurate in recognizing 24 types of colors. The proposed SMNN-MSFF outperforms state-of-the-art VCR methods, and better meets the requirements for fine classification of vehicle colors.

However, since the actual environment can be affected by unpredictable factors and the long-tail effect exists in vehicle color distribution, further efforts to improve the fine recognition of vehicle color are still required. Future work will continue studying the solution of class imbalance owing to diverse vehicle colors, and the vehicle color data set must will require characteristics of the long-tail distribution.