Keywords: Image Enhancement for Night-Vision (IENV), License Plate Recognition (LPR), License Plate Country Identification (LPCI), Vehicle Color Recognition (VCR), Vehicle Make Model Recognition (VMMR), Vehicle Body Style Recognition (VBSR), Vehicle Direction Tracking (VDT) and Vehicle Speed Estimation (VSE)
Have you ever seen a deep learning based ANPR/ALPR (Automatic Number/License Plate Recognition) engine running at 64fps on a $99 ARM device (Khadas VIM3, 720p video resolution)?
UltimateALPR is the fastest ANPR/ALPR implementation you’ll find on the market. Being fast is important but being accurate is crucial.
We use state of the art deep learning techniques to offer unmatched accuracy and precision. As a comparison this is #33 times faster than OpenALPR on Android.
(see benchmark section for more information).
No need for special or dedicated GPUs, everything is running on CPU with SIMD ARM NEON optimizations, fixed-point math operations and multithreading.
This opens the doors for the possibilities of running fully featured ITS (Intelligent Transportation System) solutions on a camera without soliciting a cloud.
Being able to run all ITS applications on the device will significantly lower the cost to acquire, deploy and maintain such systems.
Please check Device-based versus Cloud-based solution section for more information about how this would reduce the cost.
The next video shows the Recognizer sample running on Windows:
The code is accelerated on CPU, GPU, VPU and FPGA, thanks to CUDA, NVIDIA TensorRT and Intel OpenVINO.
In addition to License Plate Recognition (LPR) we support Image Enhancement for Night-Vision (IENV), License Plate Country Identification (LPCI), Vehicle Color Recognition (VCR), Vehicle Make Model Recognition (VMMR), Vehicle Body Style Recognition (VBSR), Vehicle Direction Tracking (VDT) and Vehicle Speed Estimation (VSE).
On high-end NVIDIA GPUs like the Tesla V100 the frame rate is 315 fps which means 3.17 millisecond inference time. On high-end CPUs like Intel Xeon the maximum frame rate could be up to 237fps, thanks to OpenVINO. On low-end CPUs like the Raspberry Pi 4 the average frame rate is 12fps.
Don’t take our word for it, come check our implementation. No registration, license key or internet connection is required, just clone the code and start coding/testing. Everything runs on the device, no data is leaving your computer.
The code released here comes with many ready-to-use samples for Android, Raspberry Pi, Linux and Windows to help you get started easily.
You can also check our online cloud-based implementation (no registration required) to check out the accuracy and precision before starting to play with the SDK.
Please check full documentation at https://www.doubango.org/SDKs/anpr/docs/
Getting started
The SDK works on many platforms and comes with support for many programming languages but the next sections focus on Android, Raspberry Pi, Linux and Windows.
Android
The next sections are about Android and Java API.
Sample applications (Android)
The source code comes with #4 Android sample applications: Benchmark, VideoParallel, VideoSequential and ImageSnap.
Benchmark (Android)
This application is used to check everything is ok and running as fast as expected.
The information about the maximum frame rate (237fps on Intel Xeon, 64fps on Khadas VIM3 and 12fps on Raspberry Pi 4) could be checked using this application.
It’s open source and doesn’t require registration or license key.
VideoParallel (Android)
This application should be used as reference code by any developer trying to add ultimateALPR to their products. It shows how to detect and recognize license plates in realtime using live video stream from the camera.
Please check Parallel versus sequential processing section for more info about parellel mode.
VideoSequential (Android)
Same as VideoParallel but working on sequential mode which means slower. This application is provided to ease comparing the modes: Parallel versus Sequential.
ImageSnap (Android)
This application reads and display the live video stream from the camera but only recognize an image from the stream on demand.
Trying the samples (Android)
To try the sample applications on Android:
-
Open Android Studio and select “Open an existing Android Studio project”
-
Navigate to ultimateALPR-SDK/samples, select android folder and click OK
-
Select the sample you want to try (e.g. videoparallel) and press run. Make sure to have the device on landscape mode for better experience.
Adding the SDK to your project (Android)
The SDK is distributed as an Android Studio module and you can add it as reference or you can also build it and add the AAR to your project. But, the easiest way to add the SDK to your project is by directly including the source.
In your build.gradle file add:
android {
# This is the block to add within "android { } " section
sourceSets {
main {
jniLibs.srcDirs += ['path-to-your-ultimateALPR-SDK/binaries/android/jniLibs']
java.srcDirs += ['path-to-your-ultimateALPR-SDK/java/android']
assets.srcDirs += ['path-to-your-ultimateALPR-SDK/assets/models']
}
}
}
Using the Java API (Android)
It’s hard to be lost when you try to use the API as there are only 3 useful functions: init, process and deInit.
The C++ API is defined here.
import org.doubango.ultimateAlpr.Sdk.ULTALPR_SDK_IMAGE_TYPE;
import org.doubango.ultimateAlpr.Sdk.UltAlprSdkEngine;
import org.doubango.ultimateAlpr.Sdk.UltAlprSdkParallelDeliveryCallback;
import org.doubango.ultimateAlpr.Sdk.UltAlprSdkResult;
final static String CONFIG = "{" +
"\"debug_level\": \"info\"," +
"\"gpgpu_enabled\": true," +
"\"openvino_enabled\": true," +
"\"openvino_device\": \"CPU\"," +
"\"detect_minscore\": 0.1," +
"\"detect_quantization_enabled\": true," +
"\"pyramidal_search_enabled\": true," +
"\"pyramidal_search_sensitivity\": 0.28," +
"\"pyramidal_search_minscore\": 0.5," +
"\"pyramidal_search_quantization_enabled\": true," +
"\"klass_lpci_enabled\": true," +
"\"klass_vcr_enabled\": true," +
"\"klass_vmmr_enabled\": true," +
"\"recogn_score_type\": \"min\"," +
"\"recogn_minscore\": 0.3," +
"\"recogn_rectify_enabled\": false," +
"\"recogn_quantization_enabled\": true" +
"}";
/**
* Parallel callback delivery function used to notify about new results.
* This callback will be called few milliseconds (before next frame is completely processed)
* after process function is called.
*/
static class MyUltAlprSdkParallelDeliveryCallback extends UltAlprSdkParallelDeliveryCallback {
@Override
public void onNewResult(UltAlprSdkResult result) { }
}
final MyUltAlprSdkParallelDeliveryCallback mCallback = new MyUltAlprSdkParallelDeliveryCallback(); // set to null to disable parallel mode
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
// Initialize the engine
assert UltAlprSdkEngine.init(
getAssets(),
CONFIG,
mCallback
).isOK();
}
// Camera listener: https://developer.android.com/reference/android/media/ImageReader.OnImageAvailableListener
final ImageReader.OnImageAvailableListener mOnImageAvailableListener = new ImageReader.OnImageAvailableListener() {
@Override
public void onImageAvailable(ImageReader reader) {
try {
final Image image = reader.acquireLatestImage();
if (image == null) {
return;
}
// ANPR/ALPR recognition
final Image.Plane[] planes = image.getPlanes();
final UltAlprSdkResult result = UltAlprSdkEngine.process(
ULTALPR_SDK_IMAGE_TYPE.ULTALPR_SDK_IMAGE_TYPE_YUV420P,
planes[0].getBuffer(),
planes[1].getBuffer(),
planes[2].getBuffer(),
image.getWidth(),
image.getHeight(),
planes[0].getRowStride(),
planes[1].getRowStride(),
planes[2].getRowStride(),
planes[1].getPixelStride()
);
assert result.isOK();
image.close();
} catch (final Exception e) {
e.printStackTrace();
}
}
};
@Override
public void onDestroy() {
// DeInitialize the engine
assert UltAlprSdkEngine.deInit().isOK();
super.onDestroy();
}
Again, please check the sample applications for Android, Raspberry Pi, Linux and Windows and full documentation for more information.
Raspberry Pi (Raspbian OS), Linux, NVIDIA Jetson, Windows and others
Sample applications (Raspberry Pi, Linux, NVIDIA Jetson, Windows and others)
The source code comes with #2 C++ sample applications: Benchmark and Recognizer. These sample applications can be used on all supported platforms: Android, Windows, Raspberry Pi, iOS, OSX, Linux…
Benchmark (Raspberry Pi, Linux, NVIDIA Jetson, Windows and others)
This application is used to check everything is ok and running as fast as expected.
The information about the maximum frame rate (237fps on Intel Xeon, 47fps on Snapdragon 855, 152fps on Jetson NX, 64fps on Khadas VIM3, 30fps on Jetson nano and 12fps on Raspberry Pi 4) could be checked using this application.
It’s open source and doesn’t require registration or license key.
For more information on how to build and run this sample please check samples/c++/benchmark.
Recognizer (Raspberry Pi, Linux, NVIDIA Jetson, Windows and others)
This is a command line application used to detect and recognize a license plate from any JPEG/PNG/BMP image.
For more information on how to build and run this sample please check:
Using the C++ API
The C++ API is defined at https://www.doubango.org/SDKs/anpr/docs/cpp-api.html.
#include <ultimateALPR-SDK-API-PUBLIC.h> // Include the API header file
// JSON configuration string
// More info at https://www.doubango.org/SDKs/anpr/docs/Configuration_options.html
static const char* __jsonConfig =
"{"
"\"debug_level\": \"info\","
"\"debug_write_input_image_enabled\": false,"
"\"debug_internal_data_path\": \".\","
""
"\"num_threads\": -1,"
"\"gpgpu_enabled\": true,"
"\"openvino_enabled\": true,"
"\"openvino_device\": \"CPU\","
""
"\"detect_roi\": [0, 0, 0, 0],"
"\"detect_minscore\": 0.1,"
""
"\"pyramidal_search_enabled\": true,"
"\"pyramidal_search_sensitivity\": 0.28,"
"\"pyramidal_search_minscore\": 0.3,"
"\"pyramidal_search_min_image_size_inpixels\": 800,"
""
"\"klass_lpci_enabled\": true,"
"\"klass_vcr_enabled\": true,"
"\"klass_vmm_enabled\": true,"
""
"\"recogn_minscore\": 0.3,"
"\"recogn_score_type\": \"min\""
"}";
// Local variable
UltAlprSdkResult result;
// Initialize the engine (should be done once)
ULTALPR_SDK_ASSERT((result = UltAlprSdkEngine::init(
__jsonConfig
)).isOK());
// Processing (detection + recognition)
// Call this function for every video frame
const void* imageData = nullptr;
ULTALPR_SDK_ASSERT((result = UltAlprSdkEngine::process(
ULTMICR_SDK_IMAGE_TYPE_RGB24,
imageData,
imageWidth,
imageHeight
)).isOK());
// DeInit
// Call this function before exiting the app to free the allocate resources
// You must not call process() after calling this function
ULTALPR_SDK_ASSERT((result = UltAlprSdkEngine::deInit()).isOK());
Again, please check the sample applications for more information on how to use the API.
Technical questions
Please check our discussion group or twitter account
评论