mmpose/demo/docs/en/2d_animal_demo.md

2D Animal Pose Demo

We provide a demo script to test a single image or video with top-down pose estimators and animal detectors. Assume that you have already installed mmdet with version >= 3.0.

2D Animal Pose Image Demo

python demo/topdown_demo_with_mmdet.py \
    ${MMDET_CONFIG_FILE} ${MMDET_CHECKPOINT_FILE} \
    ${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
    --input ${INPUT_PATH} --det-cat-id ${DET_CAT_ID} \
    [--show] [--output-root ${OUTPUT_DIR}] [--save-predictions] \
    [--draw-heatmap ${DRAW_HEATMAP}] [--radius ${KPT_RADIUS}] \
    [--kpt-thr ${KPT_SCORE_THR}] [--bbox-thr ${BBOX_SCORE_THR}] \
    [--device ${GPU_ID or CPU}]

The pre-trained animal pose estimation model can be found from model zoo. Take animalpose model as an example:

python demo/topdown_demo_with_mmdet.py \
    demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py \
    https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \
    configs/animal_2d_keypoint/topdown_heatmap/animalpose/td-hm_hrnet-w32_8xb64-210e_animalpose-256x256.py \
    https://download.openmmlab.com/mmpose/animal/hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth \
    --input tests/data/animalpose/ca110.jpeg \
    --show --draw-heatmap --det-cat-id=15

Visualization result:

If you use a heatmap-based model and set argument --draw-heatmap, the predicted heatmap will be visualized together with the keypoints.

The augement --det-cat-id=15 selected detected bounding boxes with label 'cat'. 15 is the index of category 'cat' in COCO dataset, on which the detection model is trained.

COCO-animals In COCO dataset, there are 80 object categories, including 10 common animal categories (14: 'bird', 15: 'cat', 16: 'dog', 17: 'horse', 18: 'sheep', 19: 'cow', 20: 'elephant', 21: 'bear', 22: 'zebra', 23: 'giraffe').

For other animals, we have also provided some pre-trained animal detection models (1-class models). Supported models can be found in detection model zoo.

To save visualized results on disk:

python demo/topdown_demo_with_mmdet.py \
    demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py \
    https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \
    configs/animal_2d_keypoint/topdown_heatmap/animalpose/td-hm_hrnet-w32_8xb64-210e_animalpose-256x256.py \
    https://download.openmmlab.com/mmpose/animal/hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth \
    --input tests/data/animalpose/ca110.jpeg \
    --output-root vis_results --draw-heatmap --det-cat-id=15

To save predicted results on disk:

python demo/topdown_demo_with_mmdet.py \
    demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py \
    https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \
    configs/animal_2d_keypoint/topdown_heatmap/animalpose/td-hm_hrnet-w32_8xb64-210e_animalpose-256x256.py \
    https://download.openmmlab.com/mmpose/animal/hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth \
    --input tests/data/animalpose/ca110.jpeg \
    --output-root vis_results --save-predictions --draw-heatmap --det-cat-id=15

To run demos on CPU:

python demo/topdown_demo_with_mmdet.py \
    demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py \
    https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \
    configs/animal_2d_keypoint/topdown_heatmap/animalpose/td-hm_hrnet-w32_8xb64-210e_animalpose-256x256.py \
    https://download.openmmlab.com/mmpose/animal/hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth \
    --input tests/data/animalpose/ca110.jpeg \
    --show --draw-heatmap --det-cat-id=15 --device cpu

2D Animal Pose Video Demo

Videos share the same interface with images. The difference is that the ${INPUT_PATH} for videos can be the local path or URL link to video file.

For example,

python demo/topdown_demo_with_mmdet.py \
    demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py \
    https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \
    configs/animal_2d_keypoint/topdown_heatmap/animalpose/td-hm_hrnet-w32_8xb64-210e_animalpose-256x256.py \
    https://download.openmmlab.com/mmpose/animal/hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth \
    --input demo/resources/<demo_dog.mp4> \
    --output-root vis_results --draw-heatmap --det-cat-id=16

The original video can be downloaded from Google Drive.

2D Animal Pose Demo with Inferencer

The Inferencer provides a convenient interface for inference, allowing customization using model aliases instead of configuration files and checkpoint paths. It supports various input formats, including image paths, video paths, image folder paths, and webcams. Below is an example command:

python demo/inferencer_demo.py tests/data/ap10k \
    --pose2d animal --vis-out-dir vis_results/ap10k

This command infers all images located in tests/data/ap10k and saves the visualization results in the vis_results/ap10k directory.

In addition, the Inferencer supports saving predicted poses. For more information, please refer to the inferencer document.

Speed Up Inference

Some tips to speed up MMPose inference:

1. set model.test_cfg.flip_test=False in animalpose_hrnet-w32.
2. use faster human bounding box detector, see MMDetection.