Hand Lights
Augmented reality application that creates artificial lights on the finger tips of hands. The effect resembles a gloving performance, which involves a person wearing gloves with LED lights on the tips. Hand keypoint detection is the core component of this project.
NOTE: Currently, the code only supports the use of a CPU, which takes takes about 10 seconds to process each frame. In a future release, I plan to add GPU support which should allow the application to run in real time.
Demos
Original Video: Void Dreamers - Gloving Vietnam
Technical Specs
Python 3.6.3
OpenCV-python 3.4.1.15
NumPy 1.14.2
Getting Started
If you prefer to gain some background knowledge on hand keypoint detection before diving into the code of this project, I would highly recommend reading Vikas Gupta’s tutorial.
Since we are not using the GPU, the setup is very minimal for this project.
- Install or update Python, OpenCV, and NumPy.
- Cloan or download this project.
- Download OpenPose’s hand keypoint detection model (pose_iter_102000.caffemodel) and place it into the
hand_lights/caffe_model folder. It can be downloaded from this repo’s release or by running OpenPose’s getModels script.
The following examples will demonstrate several ways to run the python program. Set the verbose flag to True if you would like to see the progress.
Example #1 - All default parameter values
python hand_lights.py \
--input_video_path media/videos/vertical.mp4 \
--output_video_path media/videos/vertical_output.mp4
Example #2 - Darken background
python hand_lights.py \
--input_video_path media/videos/vertical.mp4 \
--output_video_path media/videos/vertical_output.mp4 \
--background_alpha 0.3
Example #3 - Single color
python hand_lights.py \
--input_video_path media/videos/vertical.mp4 \
--output_video_path media/videos/vertical_output.mp4 \
--light_color red
Example #4 - All colors
python hand_lights.py \
--input_video_path media/videos/vertical.mp4 \
--output_video_path media/videos/vertical_output.mp4 \
--light_color all
Example #5 - Light size
python hand_lights.py \
--input_video_path media/videos/vertical.mp4 \
--output_video_path media/videos/vertical_output.mp4 \
--light_radius_frame_height_ratio 0.05
Example #6 - Draw text
python hand_lights.py \
--input_video_path media/videos/draw_initials.mp4 \
--output_video_path media/videos/draw_initials_output.mp4 \
--fingers index \
--max_hands_detected 1 \
--light_duration_n_secs 12 \
--light_same_alpha True \
--mirror True
How It Works
Detecting Finger Tips
The hand keypoint detection model from OpenPose is used to detect the finger tips of each hand on each frame.
Save Detections
The locations of the finger tips that are detected in each frame will be added to a circular queue. The queue will keep track of the detections from the past n number of frames. On every new frame, the entire queue will be used to draw the lights.
Fade Older Lights
The most recent finger tips will have fully opaque lights. As the lights become older, they will become more transparent.
Smooth Overlapping Edges
A Gaussian Blur is used to smoothen out the overlapping edges between light circles.
Python Script
To recreate the last example, use the following script.
python hand_lights.py \
--input_video_path media/videos/vertical.mp4 \
--output_video_path media/videos/vertical_output.mp4 \
--light_duration_n_secs 0.5 \
--light_radius_frame_height_ratio 0.03
评论