Eval COCO Demo

import sys
from pathlib import Path
path = Path("../src").resolve()
sys.path.extend([str(path)])

%matplotlib inline
from matplotlib import pyplot as plt
from pycocotools.coco import COCO
from pycocotools.cocoeval import COCOeval
import numpy as np


plt.rcParams['figure.figsize'] = (10.0, 8.0)

annType = ['segm', 'bbox', 'keypoints']
annType = annType[1]  # specify type here
prefix = 'person_keypoints' if annType == 'keypoints' else 'instances'
print(f'Running demo for *{annType}* results.')

Running demo for *bbox* results.

初始化 COCO ground truth api：

dataDir = '/media/pc/data/4tb/lxw/tests/datasets/coco'
dataType = 'val2014'
annFile = f'{dataDir}/annotations/{prefix}_{dataType}.json'
cocoGt = COCO(annFile)

loading annotations into memory...
Done (t=5.08s)
creating index...
index created!

初始化 COCO 检测 api：

dataDir = '../results/' # 检测结果根目录
dataType = 'val2014'
prefix = "instances"
resFile = f"{dataDir}/{prefix}_{dataType}_fakebbox100_results.json"
cocoDt = cocoGt.loadRes(resFile)

Loading and preparing results...
DONE (t=0.05s)
creating index...
index created!

imgIds = sorted(cocoGt.getImgIds())
imgIds = imgIds[0:100]
imgId = imgIds[np.random.randint(100)]

运行评估：

cocoEval = COCOeval(cocoGt, cocoDt, annType)
cocoEval.params.imgIds  = imgIds
cocoEval.evaluate()
cocoEval.accumulate()
cocoEval.summarize()

Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=0.34s).
Accumulating evaluation results...
DONE (t=0.40s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.505
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.697
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.573
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.586
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.519
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.501
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.387
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.594
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.595
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.640
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.566
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.564