generate_test_cases.py: OpFail: detect which categories are supported #264

New issue

Open

opened 2026-01-08 08:28:12 -06:00 by gilles-peskine-arm · 0 comments

gilles-peskine-arm commented

2026-01-08 08:28:12 -06:00

(Migrated from github.com)

generate_test_cases.py in the framework generates operation failure test cases. The test cases call functions for each category of algorithms that need to be defined in test_suite_psa_crypto_op_fail.function in each consuming branch.

Different consuming branches may support different sets of categories. For example, Mbed TLS 3.6 and TF-PSA-Crypto 1.1 don't support XOF, KEM or key wrapping operations, but we indend to add those to TF-PSA-Crypto soon after 1.1. The test generator generate_test_psa.py parses headers to determine which algorithms and key types are supported, but we don't have any code that tells us which algorithm categories are supported.

I see three possible solutions:

Split generate_psa_tests.py with most of the code in the framework, but a little part in each consuming branch which customizes the categories that OpFail considers.
Detect which categories are supported by doing more header inspection. We could check if PSA_ALG_CATEGORY_xxx is defined, for example (but that might not work if we want to define an algorithm before we add support for it in the API, which is something we will do for XOF in TF-PSA-Crypto 1.1).
Decide that a category is supported if at least one algorithm in that category is supported. That might not work if we want to define an algorithm before we add support for it in the API, which is something we will do for XOF in TF-PSA-Crypto 1.1.

I started to work on this in https://github.com/gilles-peskine-arm/mbedtls-framework/tree/psa_crypto_op_fail-more_categories-framework-wip with the at-least-one-supported-algorithm approach.

`generate_test_cases.py` in the framework generates operation failure test cases. The test cases call functions for each category of algorithms that need to be defined in `test_suite_psa_crypto_op_fail.function` in each consuming branch. Different consuming branches may support different sets of categories. For example, Mbed TLS 3.6 and TF-PSA-Crypto 1.1 don't support XOF, KEM or key wrapping operations, but we indend to add those to TF-PSA-Crypto soon after 1.1. The test generator `generate_test_psa.py` parses headers to determine which algorithms and key types are supported, but we don't have any code that tells us which algorithm categories are supported. I see three possible solutions: * Split `generate_psa_tests.py` with most of the code in the framework, but a little part in each consuming branch which customizes the categories that `OpFail` considers. * Detect which categories are supported by doing more header inspection. We could check if `PSA_ALG_CATEGORY_xxx` is defined, for example (but that might not work if we want to define an algorithm before we add support for it in the API, which is something we will do for XOF in TF-PSA-Crypto 1.1). * Decide that a category is supported if at least one algorithm in that category is supported. That might not work if we want to define an algorithm before we add support for it in the API, which is something we will do for XOF in TF-PSA-Crypto 1.1. I started to work on this in https://github.com/gilles-peskine-arm/mbedtls-framework/tree/psa_crypto_op_fail-more_categories-framework-wip with the at-least-one-supported-algorithm approach.

No milestone

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

gh-Mbed-TLS/mbedtls-framework#264

No description provided.

Rows
Columns