Fix copypasta in test case descriptions #271

New issue

Open

opened 2019-09-20 04:30:38 -05:00 by gilles-peskine-arm · 2 comments

gilles-peskine-arm commented

2019-09-20 04:30:38 -05:00

(Migrated from github.com)

Description

We evidently have a lot of copypasta in test case descriptions. https://github.com/ARMmbed/mbedtls/pull/2843 uncovered 1420 instances of test cases that have the same description. Sometimes the reason is test descriptions that aren't fully precise and could use some additional indications such as “… using this/that algorithm” or “… with an input of this/that length”. Sometimes there are just test cases with different input data and #1, #2, … is adequate. Sometimes test cases were copied and pasted and the test data was changed but not the test description, and the description is incorrect.

https://github.com/ARMmbed/mbed-crypto/pull/270 added a disambiguator [#1], [#2], … to non-unique test descriptions. Not all incorrect test descriptions are formerly non-unique, but having been non-unique has a strong correlation with being imprecise or incorrect. The comments in https://github.com/ARMmbed/mbed-crypto/pull/270 contain some analysis of some of the suspicious test descriptions.

Definition of done for this issue: no test case description contains [# anymore. All the descriptions that currently contain [# have been edited to be corrected, or to have a plain # without brackets if they were correct and sufficiently precise.

I expect this to be done partly organically as we modify or expand tests, and partly by working on one module at a time.

Issue request type

[ ] Question
[ ] Enhancement
[x] Bug

### Description We evidently have a lot of copypasta in test case descriptions. https://github.com/ARMmbed/mbedtls/pull/2843 uncovered 1420 instances of test cases that have the same description. Sometimes the reason is test descriptions that aren't fully precise and could use some additional indications such as “… using this/that algorithm” or “… with an input of this/that length”. Sometimes there are just test cases with different input data and `#1`, `#2`, … is adequate. Sometimes test cases were copied and pasted and the test data was changed but not the test description, and the description is incorrect. https://github.com/ARMmbed/mbed-crypto/pull/270 added a disambiguator `[#1]`, `[#2]`, … to non-unique test descriptions. Not all incorrect test descriptions are formerly non-unique, but having been non-unique has a strong correlation with being imprecise or incorrect. The comments in https://github.com/ARMmbed/mbed-crypto/pull/270 contain some analysis of some of the suspicious test descriptions. **Definition of done for this issue**: no test case description contains `[#` anymore. All the descriptions that currently contain `[#` have been edited to be corrected, or to have a plain `#` without brackets if they were correct and sufficiently precise. I expect this to be done partly organically as we modify or expand tests, and partly by working on one module at a time. ### Issue request type [ ] Question [ ] Enhancement [x] Bug

ciarmcom commented

2019-09-20 07:19:10 -05:00

(Migrated from github.com)