Inference Accuracy

Qualcomm® Neural Processing SDK inference classification precision is measured against several popular public models. Based on our measurement, the accuracy score do not vary per chipset.

Qualcomm classification precision metrics

The following clasification precision scores are computed by comparing the Qualcomm® Neural Processing SDK inference result with the ground truth:

  • mAP: mean average presion

  • Top-1 error rate: chance the highest-probability predicted class is not the real class

  • Top-5 error rate: chance the real class is not contained in the 5 classed with highest probablity

Mean Average Precision Calculation

mAP (mean Average Precision) is the Average Precision across all categories. Each AveP (Average Precision) is calculated by:

../images/AP_formula.png

Where:

  • k is the rank in the sequence of retrieved documents

  • n is the number of retrieved documents

  • P(k) is the precision at cut-off k in the list. The precision is calculated by tp/(tp+fp) where tp is true positives, fp is false positives.

  • rel(k) is an indicator function equaling 1 if the item at rank k is a relevant document, zero otherwise.

  • the precision score is zero if no relevant documents get retrieved.

python code example to calculate AP:
for j in range(len(img_sorted)):
    if img_sorted[j] in anno_imgs:
        count += 1.0
        AP += count/rank
    rank += 1.0
if (count == 0):
    AP = 0
else:
    AP = AP/count