Duplicate test case descriptions #262

New issue

Closed

opened 2019-09-16 12:59:04 -05:00 by gilles-peskine-arm · 2 comments

gilles-peskine-arm commented

2019-09-16 12:59:04 -05:00

(Migrated from github.com)

Description

We have 1420 duplicated test case descriptions, i.e. two test cases in the same test suite that have the exact same description.

This doesn't break any tooling yet, but it makes the outcome file introduced by https://github.com/ARMmbed/mbedtls/pull/2843 imprecise. It also makes it hard to figure out which test case is failing when looking at logs.

https://github.com/ARMmbed/mbedtls/pull/2843 adds a script tests/scripts/check-test-cases.py that checks this issue and fixes the problem in the tests that are present in mbedtls/development.

Issue request type

[ ] Question
[ ] Enhancement
[x] Bug

### Description We have 1420 duplicated test case descriptions, i.e. two test cases in the same test suite that have the exact same description. This doesn't break any tooling yet, but it makes the outcome file introduced by https://github.com/ARMmbed/mbedtls/pull/2843 imprecise. It also makes it hard to figure out which test case is failing when looking at logs. https://github.com/ARMmbed/mbedtls/pull/2843 adds a script `tests/scripts/check-test-cases.py` that checks this issue and fixes the problem in the tests that are present in `mbedtls/development`. ### Issue request type [ ] Question [ ] Enhancement [x] Bug

ciarmcom commented

2019-09-16 15:41:47 -05:00

(Migrated from github.com)

Internal Jira reference: https://jira.arm.com/browse/IOTCRYPT-916

*Internal Jira reference: https://jira.arm.com/browse/IOTCRYPT-916*

gilles-peskine-arm commented

2019-09-20 04:32:18 -05:00

(Migrated from github.com)

The scope of this issue is solely to make test case descriptions unique. Making them correct and helpful is out of scope. Reviewing the correctness of the non-unique test descriptions is in scope for https://github.com/ARMmbed/mbed-crypto/issues/271.