License Plate Detection and Recognition for Degraded CCTV Videos
	2011. 06. 01 ~

1. Introduction

l Intelligent transportation systems (ITSs) aim at improving transportation safety and productivity.

l ITSs make use of advanced technologies such automatic license plate recognition (LPR) that has recently attracted considerable amount of research attention.

ž LPR applications: tolling, border control, traffic control, marketing tools, and airport parking.

l Challenges in LPR in CCTV: degraded video frames

ž The size of license plate (LP) region is under 80x20 pixels.

ž In such a case, the size of each plate character is usually 8x12 pixels.

ž Blur and noise: blur usually comes from motion of vehicles while noise is generated from dust and vehicle exhaust.

l We propose an LPR system for degraded vehicle images in real-world applications.

ž We propose a character segmentation method using prior-knowledge.

ž We propose an enhanced character feature which exploits Gabor wavelet with color information, which is robust to low-resolution video frames.

ž We propose simple and effective multiple-frame-based character recognition based on probabilistic voting.

2. License Plate Detection

Fig. 1. License plate detection from CCTV image by using color and texture information

(a)

(b)

Fig. 2. An example of color partition.

(a) Illustration for HSV color space and color partition.

(b) White, Black, and Gray colors represent background, foreground of LP region, and background of LP region.

l There are two types of LPs i.e., white-black type and green-white type.

l Based on color partition, we count the color transitions between pixels of LP foreground color and pixels of LP background color.

l If the counted transition number (within a 15x3 window) is higher than the pre-determined threshold value, then the center pixel of the window is regarded as LP candidate.

3. Character Segmentation

Fig. 3. Procedure of extracting LP character regions from detected LP region.

Fig. 4. Initial LP character detection using SVM classifier

l Initial LP character detection: We first examine every candidate character region by using support vector machine (SVM) classifier.

ž The only character regions which has higher confidence value than a threshold value is regarded as the initial LP characters.

l We detect the remaining LP character regions using the assumption that missing character should be in the left or right side from the initial LP characters.

4. Character Recognition

Fig. 5. Color Gabor feature extraction from LP character region.

(a)

(b)

Fig. 6. Multiple-frame-based LP character recognition.

(a) Example of combining the results from multiple frames.

(b) Probabilistic voting for LP character recognition.

l We first select most frequently occurred candidates (e.g., 7624 and 6241 in Fig. 6(a)) among all possible 4bit string.

l To select the most probable candidate for the vehicle identification number, among the candidates in the above step, we use probabilistic voting.

ž 4 confidence values from 4bit string are summed and compared to find the candidate with the highest probability.

5. Experiment

5.1 License Plate Detection Results

<Experimental setup>

l Datasets

ž Dataset 1: Contains 203 vehicle images with LPs of green-white type (see Fig. 7(a))

ž Dataset 2: Contains 224 vehicle images with LPs of white-black type (see Fig. 7(b))

l Experimental condition

ž Vehicle images were captured by using a CCTV camera

ž Camera resolution: 640x480

ž LP size: over 80 pixels in horizontal size

ž Time for image acquisition: 3 PM

(a)

(b)

Fig. 7. Example vehicle images used in experiment.

(a) for green-white LP. (b) for white-black LP.

<Experimental results>

Table 1. License plate detection performance.

Data	Feature	Plate Detection Rate
Dataset 1	Green	95.07% (193/203)
Dataset 2	White	90.18% (202/224)
Total		92.51% (395/427)

5.2 License Plate Recognition Results

<Experimental setup>

l Datasets

ž Dataset 1: Contains 656 character images from LPs of green-white type

ž Dataset 2: Contains 737 character images from LPs of white-black type

<Experimental results>

Table 2. Character recognition performance.

Character feature	Dataset 1	Dataset 2
Raw pixel data	87.48% (587/656)	86.16% (635/737)
Binary feature	90.09% (591/656)	87.38% (644/737)
Local Binary Pattern	60.67% (398/656)	58.89% (434/737)
Gabor wavelet	96.95% (636/656)	95.25% (702/737)
Color Gabor wavelet (proposed)	97.87% (642/656)	95.39% (703/737)

Table 3. Comparison between single-frame-based approach and multiple-frame-based approach for 4 digit LP recognition.

Character feature	Using single-frame-based approach (baseline)	Using multiple-frame-based approach (proposed)
Dataset 1	54.92% (106/193)	78.95% (30/38)
Dataset 2	53.96% (109/202)	76.36% (42/55)

6. Conclusions

l In this research, we proposed an improved LPR system that can be effectively applied to real-world environment.

l For LP character segmentation, we introduce ROI calibration after initial character detection to find missing characters.

ž We achieved improved character segmentation result even if the vehicle images were blurred and noise-corrupted.

l In license plate recognition, we proposed to incorporate the vehicle images of same vehicle to perform probabilistic voting, yielding successful LPR result compared to the conventional single frame-based approach.

* Contact Person: Prof. Yong Man Ro (ymro@kaist.ac.kr)

1. Hyun-Seok Min, Seung Ho Lee, Wesley De Neve and Yong Man Ro “Sparse Representation-based Super-Resolution for Improved License Plate Recognition for Low-Resolution CCTV Forensics,” International Workshop on Digital-Forensic and Watermarking (IWDW), 2013.

2. 신욱진, 이승호, 민현석, 최재영, 노용만, “기울기와 크기 변화에 강인한 차량 번호판 검출에 관한 연구,” 한국멀티미디어학회 추계학술발표대회, 2011.

l Video1: License plate detection and recognition in CCTV video .

l Video2: Intelligent Surveillance System Using the Proposed License Plate Recognition .