gh-Mbed-TLS/mbedtls-framework

Watch

Fork 0

Move misc. "easy" scripts to the framework #87

New issue

Open

opened 2024-12-02 04:07:56 -06:00 by mpg · 3 comments

mpg commented

2024-12-02 04:07:56 -06:00

(Migrated from github.com)

Move the following files to the framework:

tests/scripts/psa_collect_statuses.py
tests/scripts/run-metatests.sh
tests/scripts/run_demos.py
tests/scripts/test_zeroize.gdb

These are the scripts that are:

currently identical between 3.6 and development;
not covered by a previous issue.

To be discussed: should we also move programs/test/zeroize.c and programs/test/metatest.c?

Note: as usual for this EPIC, the work is only to move the files to the framework repo and adapt the paths everywhere they're used in mbedtls 3.6 and development. If further changes are needed to adapt to tf-psa-crypto, they're out of scope and will be covered by separate tasks.

Move the following files to the framework: - [ ] tests/scripts/psa_collect_statuses.py - [ ] tests/scripts/run-metatests.sh - [ ] tests/scripts/run_demos.py - [x] tests/scripts/test_zeroize.gdb These are the scripts that are: - currently identical between 3.6 and development; - not covered by a previous issue. To be discussed: should we also move `programs/test/zeroize.c` and `programs/test/metatest.c`? Note: as usual for this EPIC, the work is only to move the files to the framework repo and adapt the paths everywhere they're used in mbedtls 3.6 and development. If further changes are needed to adapt to tf-psa-crypto, they're out of scope and will be covered by separate tasks.

gilles-peskine-arm commented

2024-12-02 05:26:24 -06:00

(Migrated from github.com)

tests/scripts/psa_collect_statuses.py

I wrote that a while ago to solve a short-term problem (coverage of error cases and comparing that with what the specification draft says, to identify missing cases in the specification and missing coverage in our tests). I got it merged because it seemed like it could be a useful way to measure this aspect of test coverage, but I don't think we've made any use of it since then. So we could remove it and the corresponding code, if that's easier than moving it to the framework.

Today I would use the PSA wrapper generator framework for this, and I even have code to do that (also logging parameters, not just return values) in https://github.com/Mbed-TLS/mbedtls/pull/8559 (I used that partly to validate the PSA wrapper generator framework and partly because I needed to log the coverage of parameters).

programs/test/zeroize.c

It's very tightly coupled with test_zeroize.gdb so I think both should be in the same repository. And I can't think of a reason why they should be different in different branches, so it makes sense for both to be in the framework.

programs/test/metatest.c?

In principle this could have branch-specific code. But right now I can't think of anything branch-specific that I'd want to add even if I had plenty of spare time. It's mostly about checking that we're catching the kinds of failure we expect to catch from the test framework as well as from environment features such as sanitizers. So I think it belongs together with tests/src, in the framework.

> `tests/scripts/psa_collect_statuses.py` I wrote that a while ago to solve a short-term problem (coverage of error cases and comparing that with what the specification draft says, to identify missing cases in the specification and missing coverage in our tests). I got it merged because it seemed like it could be a useful way to measure this aspect of test coverage, but I don't think we've made any use of it since then. So we could remove it and the corresponding code, if that's easier than moving it to the framework. Today I would use the PSA wrapper generator framework for this, and I even have code to do that (also logging parameters, not just return values) in https://github.com/Mbed-TLS/mbedtls/pull/8559 (I used that partly to validate the PSA wrapper generator framework and partly because I needed to log the coverage of parameters). > `programs/test/zeroize.c` It's very tightly coupled with `test_zeroize.gdb` so I think both should be in the same repository. And I can't think of a reason why they should be different in different branches, so it makes sense for both to be in the framework. > `programs/test/metatest.c?` In principle this could have branch-specific code. But right now I can't think of anything branch-specific that I'd want to add even if I had plenty of spare time. It's mostly about checking that we're catching the kinds of failure we expect to catch from the test framework as well as from environment features such as sanitizers. So I think it belongs together with `tests/src`, in the framework.

mpg commented

2024-12-02 05:31:34 -06:00

(Migrated from github.com)

My main question about programs is are we ready to have C executable programs (as opposed to libraries / object files) in the framework, in terms of how our build system(s) work? Cc @ronald-cron-arm

Other than that, I'm also inclined to have them in the framework. But if that's not easy, we might want to post-pone it (or at least split it to a separate task).

My main question about programs is are we ready to have C executable programs (as opposed to libraries / object files) in the framework, in terms of how our build system(s) work? Cc @ronald-cron-arm Other than that, I'm also inclined to have them in the framework. But if that's not easy, we might want to post-pone it (or at least split it to a separate task).

ronald-cron-arm commented

2024-12-02 05:50:55 -06:00

(Migrated from github.com)

My main question about programs is are we ready to have C executable programs (as opposed to libraries / object files) in the framework, in terms of how our build system(s) work? Cc @ronald-cron-arm

We have not done this so far thus we definitely need a specific task for that.

> My main question about programs is are we ready to have C executable programs (as opposed to libraries / object files) in the framework, in terms of how our build system(s) work? Cc @ronald-cron-arm We have not done this so far thus we definitely need a specific task for that.

No milestone

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

gh-Mbed-TLS/mbedtls-framework#87

No description provided.

Rows
Columns