scopeInfinity/Video2Description 开源项目中文解读：技术栈、适用场景与上手建议

362

Apache License 2.0

项目介绍

scopeInfinity/Video2Description 是一个托管在 GitHub 上的开源项目，主要使用 Python 编写，采用 Apache License 2.0 许可证，目前获得 362 个 Star、67 次 Fork。

仓库全名: scopeInfinity/Video2Description
开源许可证: Apache License 2.0
Star 数: 362
Fork 数: 67
未关闭 Issue: 1

README 摘要

Video Captioning Generate caption for the given video clip Branch : VideoCaption Status : Ongoing Setup Setup anaconda environment, use environment.yml Install keras with tensorflow backend. Install ffmpeg Configure, build and install ffmpeg from source with shared libraries Set environment variable Reference https://conda.io/docs/user-guide/tasks/manage-environments.html#saving-environment-variables Install opencv…

完整内容见 GitHub 原始 README 或本页。

常见问题

Video2Description 用什么语言开发？

主要使用 Python 编写，采用 Apache License 2.0 许可证，目前有 362 个 Star。

Video2Description 可以免费商用吗？

如何获取 Video2Description 的源码？

可以直接克隆仓库：git clone https://github.com/scopeInfinity/Video2Description.git；安装与运行步骤见仓库 README。

Video2Description 最近还在维护吗？

仓库最近一次更新为 2022年5月4日，当前有 1 个未关闭 Issue；Star 数为 362，可作为社区活跃度的参考。

关于

Video to Text: Natural language description generator for some given video. [Video Captioning]

描述取自 GitHub 官方 API，中文内容由 GitRead 中文社区编辑团队整理。

简要分析

热度分

858

价值分

273

活跃状态

活跃

audio-processing cnn-keras deep-neural-networks image-captioning lstm-neural-networks video-captioning video-processing video-to-text

GitHub

语言占比

Dockerfile

HTML

Python

Shell

Release

Pre-Trained Models2020年3月15日

贡献者

scopeInfinity

README

Video Captioning

Generate caption for the given video clip

Branch : VideoCaption

Status : Ongoing

Setup

Setup anaconda environment, use environment.yml
- Install keras with tensorflow backend.
Install ffmpeg
- Configure, build and install ffmpeg from source with shared libraries

git clone 'https://github.com/FFmpeg/FFmpeg.git'
cd FFmpeg
./configure --prefix=~/V2D_local/ --enable-shared
make
make install

Set environment variable
- Reference https://conda.io/docs/user-guide/tasks/manage-environments.html#saving-environment-variables

export PATH=/home/gagan.cs14/V2D_local/bin:$PATH
export LD_LIBRARY_PATH=$/home/gagan.cs14/V2D_local/lib/:$LD_LIBRARY_PATH

Install opencv

conda install opencv -c conda-forge

Data Directory

File	Reference
/path/to/data_dir/VideoDataset/videodatainfo_2017.json	http://ms-multimedia-challenge.com/2017/dataset
/path/to/data_dir/VideoDataset/videos/[0-9]+.mp4	Download videos based on above dataset
/path/to/data_dir/glove/glove.6B.100d.txt	https://nlp.stanford.edu/projects/glove/

Working Directory

File	Content
/path/to/working_dir/glove.dat	Pickle Dumped Glove Embedding
/path/to/working_dir/vocab.dat	Pickle Dumped Vocabulary Words

Download Dataset

Execute python videohandler.py from VideoDataset Directory

Training Methods

Try Iterative Learning
Try Random Learning

Evaluation

Prerequisite

cd /path/to/eval_dir/
git clone 'https://github.com/tylin/coco-caption.git' cococaption
ln /path/to/working_dir/cocoeval.py cococaption/

Evaluate

# One can do changes in parser.py for numbers of test examples to be considered in evaluation
python parser.py predict save_all_test
python /path/to/eval_dir/cocoeval.py <results file>.txt

Sample Evaluation while training

Commit	Training	CIDEr	Bleu_4	Bleu_3	Bleu_2	Bleu_1	ROUGE_L	METEOR	Model Filename
3ccf5d5	CPU 15hrs	0.1258	0.2535	0.3763	0.51817	0.6792	0.4619	0.1895	res_mcnn_rand_b100_s500_model.dat_model1_3ccf5d5

Image Captioning

Generate caption for the given images

Branch : onehot_gen

Commit : 898f15778d40b67f333df0a0e744a4af0b04b16c

Trained Model : https://drive.google.com/open?id=1qzMCAbh_tW3SjMMVSPS4Ikt6hDnGfhEN

Categorical Crossentropy Loss : 0.58

Video2Description