Inconsistent syntax expectations in data files #284

New issue

Open

opened 2026-03-04 05:45:43 -06:00 by gilles-peskine-arm · 0 comments

gilles-peskine-arm commented

2026-03-04 05:45:43 -06:00

(Migrated from github.com)

We have two pieces of code that parse .data files containing unit tests:

framework/scripts/mbedtls_framework/test_suite_preprocessor.py, moved from framework/scripts/generate_test_code.py in https://github.com/Mbed-TLS/mbedtls-framework/pull/282, used to generate .datax files which are then used to run the unit tests.
framework/scripts/mbedtls_framework/collect_test_cases.py, used in particular to list the test cases in outcome analysis, so it's what we use to check that all test cases are executed.

These have independent parsers, and they make different assumptions when we deviate from our habits. In particular:

Description 1
function1:argument1
Description 2
function2:argument2

without a blank line is treated as two test cases when generating .datax file, but “Description 2” is ignored for coverage analysis, so we may miss some test cases.

We have two pieces of code that parse `.data` files containing unit tests: * `framework/scripts/mbedtls_framework/test_suite_preprocessor.py`, moved from `framework/scripts/generate_test_code.py` in https://github.com/Mbed-TLS/mbedtls-framework/pull/282, used to generate `.datax` files which are then used to run the unit tests. * `framework/scripts/mbedtls_framework/collect_test_cases.py`, used in particular to list the test cases in outcome analysis, so it's what we use to check that all test cases are executed. These have independent parsers, and they make different assumptions when we deviate from our habits. In particular: ``` Description 1 function1:argument1 Description 2 function2:argument2 ``` without a blank line is treated as two test cases when generating `.datax` file, but “Description 2” is ignored for coverage analysis, so we may miss some test cases.

No milestone

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

gh-Mbed-TLS/mbedtls-framework#284

No description provided.

Rows
Columns