InsightFace-REST
GitHub Issues
WARNING: Latest update may cause troubles with previously compiled Numba functions.
If you met any errors concerning ‘modules not found’ Run following command in repo root to remove __pycache__:
find . | grep -E "(__pycache__|\.pyc$)" | sudo xargs rm -rf
A production-ready REST API for face detection and recognition pipelines. Built with FastAPI for serving and
optimized with NVIDIA TensorRT for high-performance inference. Supports both GPU-accelerated and CPU-based deployments.
Code is heavily based on API
code
in official DeepInsight InsightFace
repository.
Key features:
- 🚀 High-Performance Inference: Up to 820 fps on RTX4090 with default settings. And even faster with lighter models.
- 🐳 Dockerized Deployment: Ready for NVIDIA GPU systems.
- 🤖 Automatic Model Download: Self-contained setup with Google Drive downloads
- 🔄 Batch Processing: Supported for both detection and recognition models
- 📦 Model Variety: SCRFD and YoloV5 detectors + PyTorch recognition models
- ⚙️ Multi-Backend Support:
- TensorRT for GPU acceleration
- ONNX Runtime for CPU inference
- 📈 Optimized Postprocessing: 2x faster SCRFD implementation
Prerequisites
- Docker (latest stable version)
- NVIDIA Container Toolkit
- NVIDIA GPU Drivers (535 or newer)
- Compatible NVIDIA GPU (for GPU acceleration)
Quick Start with Docker
-
Clone repository:
git clone https://github.com/SthPhoenix/InsightFace-REST
-
Move to compose dir:
cd InsightFace-REST/compose
-
Copy example.env to .env, modify configs if needed:
cp example.env .env
-
Build image:
docker compose build
-
Run docker compose:
docker compose up
-
Access API documentation: http://localhost:18081/docs
For other deployment options refer to README.md in compose directory.
List of supported models:
Detection:
Note: Performance metrics measured on NVIDIA RTX2080 SUPER + Intel Core i7-5820K (3.3Ghz * 6 cores) for
api/src/test_images/lumia.jpg with force_fp16=True, det_batch_size=1 and max_size=640,640.
Detection time include inference, pre- and postprocessing, but does not include image reading, decoding and resizing.
Note 2: SCRFD family models requires input image shape dividable by 32, i.e 640x640, 1024x768.
Recognition:
Other:
* - Models will be downloaded from Google Drive, which might be inaccessible in some regions like China.
** - custom models retrained for this repo. Original SCRFD models have bug
(deepinsight/insightface#1518) with
detecting large faces occupying >40% of image. These models are retrained with Group Normalization instead of
Batch Normalization, which fixes bug, though at cost of some accuracy.
Models accuracy on WiderFace benchmark:
| Model |
Easy |
Medium |
Hard |
| scrfd_10g_gnkps |
95.51 |
94.12 |
82.14 |
| scrfd_2.5g_gnkps |
93.57 |
91.70 |
76.08 |
| scrfd_500m_gnkps |
88.70 |
86.11 |
63.57 |
*** - custom model retrained for 112x112 input size to remove excessive resize operations and
improve performance.
API usage:
For example of API usage example please refer to
demo_client.py code.
⚠️ Known issues :
- Google Drive downloads may fail in restricted regions
Contributors & References
Special thanks to InsightFace authors and these projects:
Changelog:
(a bit deprecated…)
2021-11-06 v0.7.0.0
Since a lot of updates happened since last release version is updated straight to v0.7.0.0
Comparing to previous release (v0.6.2.0) this release brings improved performance for SCRFD based detectors.
Here is performance comparison on GPU Nvidia RTX 2080 Super for scrfd_10g_gnkps detector paired with
glintr100 recognition model (all tests are using src/api_trt/test_images/Stallone.jpg, 1 face per image):
| Num workers |
Client threads |
FPS v0.6.2.0 |
FPS v0.7.0.0 |
Speed-up |
| 1 |
1 |
56 |
103 |
83.9% |
| 1 |
30 |
72 |
128 |
77.7% |
| 6 |
30 |
145 |
179 |
23.4% |
Additions:
- Added experimental support for msgpack serializer: helps reduce network traffic for embeddings for ~2x.
- Output names no longer required for detection models when building TRT engine - correct output order is now extracted
from onnx models.
- Detection models now can be exported to TRT engine with batch size > 1 - inference code doesn’t support it yet, though
now they could be used in Triton Inference Server without issues.
Model Zoo:
- Added support for WebFace600k based recognition models from InsightFace repo:
w600k_r50 and w600k_mbf
- Added md5 check for models to allow automatic re-download if models have changed.
- All
scrfd based models now supports batch dimension/
Improvements:
- 1.5x-2x faster SCRFD re-implementation with Numba: 4.5 ms. vs 10 ms. for
lumia.jpg example with
scrfd_10g_gnkps and threshold = 0.3 (432 faces detected)).
- Move image normalization step to GPU with help of CuPy (4x lower data transfer from CPU to GPU, about 6%
inference speedup, and some computations offloaded from CPU).
- 4.5x Faster
face_align.norm_crop implementation with help of Numba and removal of unused computations.
(Cropping 432 faces from lumia.jpg example tooks 45 ms. vs 205 ms.).
- Face crops are now extracted only when needed - when face data or embeddings are requested, improving
detection only performance.
- Added Numba njit cache to reduce subsequent starts time.
- Logging timings rounded to ms for better readability.
- Minor refactoring
Fixes:
- Since gender/age estimation model is currently not supported exclude it from models preparing step.
2021-09-09 v0.6.2.0
REST-API
- Use async
httpx lib for retrieving images by urls instead of urllib3 (which caused
performance drop in multi-GPU environment under load due to excessive usage of opened sockets)
- Update draft Triton Infernce Server support to use CUDA shared memory.
- Minor refactoring for future change of project structure.
2021-08-07 v0.6.1.0
REST-API
- Dropped support of MXNet inference backend and automatic MXNet->ONNX models conversion,
since all models are now distributed as ONNX by default.
2021-06-16 v0.6.0.0
REST-API
- Added support for newer InsightFace face detection SCRFD models:
scrfd_500m_bnkps, scrfd_2.5g_bnkps, scrfd_10g_bnkps
- Released custom trained SCRFD models:
scrfd_500m_gnkps, scrfd_2.5g_gnkps, scrfd_10g_gnkps
- Added support for newer InsightFace face recognition model
glintr100
- Models auto download switched to Google Drive.
- Default models switched to
glintr100 and scrfd_10g_gnkps
2021-05-08 v0.5.9.9
REST-API
- Added JPEG decoding using PyTurboJPEG - increased decoding speed for large
JPEGs for about 2x.
- Support for batch inference of
genderage model.
- Support for limiting number of faces for recognition using
limit_faces parameter
in extract endpoint.
- New
/multipart/draw_detections endpoint, supporting image upload using multipart
form data.
- Support for printing face sizes and scores on image by
draw_detections endpoints.
- More verbose timings for
extract endpoint for debug and logging purposes.
2021-03-27 v0.5.9.8
REST-API
- Added v2 output format: more verbose and more suitable for logging.
Use
'api_ver':'2' in request body. In future versions this parameter
will be moved to path, like /v2/extract, and will be default output
format.
REST-API & conversion scripts:
- MXNet version in dockerfiles locked to 1.6.0, since version 1.8.0
causes missing libopenblas.0 exception.
2021-03-01 v0.5.9.7
REST-API & conversion scripts:
- Fixed issue with building TensorRT engine with batch > 1 and FP16
support, which caused FP32 inference instead of FP16.
- Moved to tensorrt:21.02 base image and removed workarounds for 20.12
image.
- Changed behaviour of
force_fp16 flag. Now model with FP16 precision
is build only when set to True. Otherwise FP32 will be used even on
GPUs with fast FP16 support.
2021-03-01 v0.5.9.6
REST-API:
- Add flag
embed_only to /extract endpoint. When set to true
input images are processed as face crops, omitting detection phase.
Expects 112x112 face crops.
- Added flag
draw_landmarks to /draw_detections endpoint.
2021-02-13
REST-API:
- Added Dockerfile for CPU-only inference with ONNXRuntime backend.
- Added flag to return landmarks with
/extract endpoint
2020-12-26
REST-API & conversion scripts:
- Added support for
glint360k_r100FC_1.0 and glint360k_r100FC_0.1
face recognition models.
2020-12-26
REST-API:
- Base image updated to
TensorRT:20.12-py3.
- Added temporary fixes for TensortRT 7.2.2 ONNX parsing.
- Added support for
r50-arcface-msfdrop75 face recognition model.
Conversion scripts:
- Same updates as for REST-API
2020-12-06
REST-API:
- Added draft support for batch inference of ArcFace model.
Conversion scripts:
- Added draft support for batch inference of ArcFace model.
2020-11-20
REST API:
- Pure MXNet version removed from master branch.
- Added models bootstrapping before running workers, to prevent race
condition for building TRT engine.
- Applied changes from conversion scripts (see below)
Conversion scripts:
- Reshape ONNX models in memory to prevent writing temp files.
- TRT engine builder now takes input name and shape, required for
building optimization profiles, from ONNX model intself.
2020-11-07
Conversion scripts:
- Added support for building TensorRT engine with batch input.
- Added support for RetinaFaceAntiCov model (mnet_cov2, must be manually
downloaded
and unpacked to
models/mxnet/mnet_cov2)
REST API:
- Added support for RetinaFaceAntiCov v2
- Added support for FP16 precision (
force_fp16 flag in
deploy_trt.sh)
2020-10-22
Conversion scripts:
REST API:
- Added TensorRT version in
src/api_trt
- Added Dockerfile (
src/Dockerfile_trt)
- Added deployment script
deploy_trt.sh
- Added Centerface detector
TensorRT version contains MXNet and ONNXRuntime compiled for CPU for
testing and conversion purposes.
2020-10-16
Conversion scripts:
- Added conversion of MXNet models to ONNX using Python
- Added conversion of ONNX to TensorRT using Python
- Added demo inference scripts for ArcFace and Retinaface using ONNX and
TensorRT backends
REST API:
2020-09-28
- REST API code refactored to FastAPI
- Detection/Recognition code is now based on official Insightface Python
package.
- TensorFlow MTCNN replaced with PyTorch version
- Added RetinaFace detector
- Added InsightFace gender/age detector
- Added support for GPU inference
- Resize function refactored for fixed image proportions (significant
speed increase and memory usage optimization)
![dependabot[bot]](https://avatars.githubusercontent.com/in/29110?v=4)
评论