Merge mbedtls/development-psa, mbedtls/development into development #51
No reviewers
Labels
No labels
api-spec
bug
CLA requested
CLA valid
compliance
DO NOT MERGE
duplicate
enhancement
feature branch
good first issue
HEADER FORMATTING ISSUE
help wanted
Information
invalid
Jira status: AWAITING REVIEW
Jira status: CLOSED
Jira status: DONE
Jira status: IN PROGRESS
Jira status: IN REVIEW
Jira status: OPEN
Jira status: REOPENED
Jira status: RESOLVED
mirrored
mirror internally
needs: backports
needs: ci
needs: design review
needs: preceding PR
needs: review
needs: work
notabug
open for community contribution
Product Backlog
question
ready for merge
security
stale
type: bug
type: enhancement
type: question
web pages
wontfix
writing
No milestone
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
gh-ARMmbed/mbed-crypto!51
Loading…
Reference in a new issue
No description provided.
Delete branch "update-dev-tls-dev-crypto-merge"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Merge Mbed TLS's merge of https://github.com/ARMmbed/mbedtls/tree/development and https://github.com/ARMmbed/mbedtls/tree/development-psa into Mbed Crypto. This finally aligns the crypto implementation that supports parameter validation with the crypto implementation that supports using TLS on PSA with the crypto implementation in Mbed Crypto which supports using null zero-length buffers in RSA. The source for this merge is the current version of https://github.com/ARMmbed/mbedtls/pull/2395.
This is a step along the way to aligning the crypto implementation in all three branches
https://github.com/ARMmbed/mbedtls/pull/2395 cannot be merged to the Mbed TLS development branch until Mbed Crypto has an up to date crypto implementation, which this PR brings.
After this PR is merged, Mbed TLS will update the submodule pointer in PR https://github.com/ARMmbed/mbedtls/pull/2395 and merge it to development. development-psa will no longer have a reason to exist.
See the merge commit description for an explanation of what merge conflicts were resolved and how. Additional commits are added on top to implement new features in Mbed Crypto like parameter validation support in examples and to remove use of recently deprecated constants.
Rebase to add new error code handling in psa_crypto.c in commit psa: Use new generic error codes and complete updating the PSA examples for parameter validation support in commit psa: Enable use of PSA examples with CHECK_PARAMS
I did the merge myself and found the following differences.
In
library/cipher.cI think https://github.com/ARMmbed/mbed-crypto/pull/44 was unintentionally reverted.In
library/psa_crypto.cI get the following diff between this PR and my version:The version in this PR diverges from the version in mbed-crypto/development in a way that I don't think is intended.
configs/config-default.handconfigs/config-psa-crypto.hwhere it's not immediately clear to me which version is correct, as I'm not sure how different or similar toinclude/mbedtls/config.hthose files are supposed to be.@Patater There are a number of tests failing in the CI, which I did not really investigate, plus one test that seems to be hanging in both jobs (
all_sh-test_use_psa_crypto_full_cmake_asan) and might require killing.I still had a superficial look at the failures and here are my first impressions:
all_sh-test_check_params_silentseem to fail only intest_suite_psa_cryptoso might be unrelated to that particular build, and perhaps more related to the change inlibrary/psa_crypto.cthat I mentioned above.all_sh-test_check_params_without_platformbuild error:probably unrelated to parameter validation but more related to building with full config except
MBEDTLS_PLATFORM_C, I suspect a missing#includein case the platform module is not enabled.all_sh-test_full_cmake_clang,all_sh-test_m32_o0,all_sh-test_m32_o1,all_sh-test_mx32,all_sh-test_no_64bit_multiplication,all_sh-test_no_udbl_divisionalso only fail intest_suite_psa_crypto, see above (hopefully same root cause for all of them)all_sh-test_no_platformfails to build with:probably missing a standard
#includewhen the platform module is not enabled.I tool the liberty of force-stopping the CI jobs as both seemed stuck in
all_sh-test_use_psa_crypto_full_cmake_asanand I didn't want to waste CI resources. I have no idea what the problem is so far.Fixed
These are from https://github.com/ARMmbed/mbed-crypto/pull/48, which was merged after this branch was created. If you want to try to reproduce my merge, use
2a0f48ae1fas a base. When this PR is merged, those changes won't be removed.configs/config-psa-crypto.hshould be identical toinclude/mbedtls/config.hin Mbed Crypto, as its the default we use in Mbed Crypto.configs/config-default.hshould be identical toinclude/mbedtls/config.hin Mbed TLS.It's interesting to compare
configs/config-default.hwithconfigs/config-psa-crypto.h, to see what options we use differently between the libraries. Specifically, we enable persistent keys by default, where Mbed TLS doesn't yet. We also enable the presence of PSA APIs by default.We'll look into those. Thanks taking a look!
Rebased on latest mbed-crypto/development branch.
Previous branch at https://github.com/Patater/mbed-crypto/tree/update-dev-tls-dev-crypto-merge-1
Added a commit to fix "no platform" tests.
Tests verified working with new branch:
If you are attempting to reproduce the merge yourself, the current base is
1fb011f2a6.@Patater Thanks for fixing! I'm now unable to find any issues by just inspecting the code, but the CI is still failing in
all_sh-test_use_psa_crypto_full_cmake_asan.I only looked at a few failures in
ssl-opt.shbut I suspect all are the same. The call topsa_crypto_init()returns15which isPSA_ERROR_INSUFFICIENT_ENTROPY.I really have no idea why this is happening all the time in the
USE_PSA_CRYPTObuild but not the others. As the error seems to be coming from a call to a PSA API, I'm sure you guys will have more ideas than me about it.Yup, look into what's happening today and see if I can't make a fix.
I suspect this has to do with the way the entropy module and the PSA subsystem are initialized. Each works on its own, but in an application that calls both
mbedtls_platform_setupandplatform_crypto_init, depending on how you do it, it's possible to cause it to fail or to go into an infinite loop. See a discussion in https://github.com/ARMmbed/mbed-os-example-tls/pull/232#discussion_r249813184One issue, which I think we need to resolve on the TLS side, is the change to main_test.function to make
main()callpsa_crypto_init()unconditionally. This will at least break PSA tests, where we check forpsa_crypto_init()to fail in certain conditions (e.g. in our entropy injection tests). I'll leave a comment on the TLS PR.I made the merge myself (https://github.com/gilles-peskine-arm/mbed-crypto/tree/prc_51-merge-gilles) and found a few differences, sometimes my mistake, sometimes yours, sometimes indifferent.
include/mbedtls/cipher.h: just whitespace, yours is better.library/cipher.c: I missed a bugfix that was made in the crypto branch (remove dead codereturn( 0 );inmbedtls_cipher_write_tag. Please indicate this in the commit message for the merge.tests/scripts/all.sh: I missed removingUSE_CRYPTO_SUBMODULEfromcomponent_test_use_psa_crypto_full_cmake_asan.tests/scripts/run-test-suites.pl: you missed discarding the addition of a crypto submodule thing.tests/suites/helpers.function: you missed some text when merging the two descriptions ofTEST_ASSERT. Also some trivial whitespace differences.This was a complicated merge and I'd like a more detailed commit message that contains sufficiently precise instructions to do the same merge (or at least up to trivial differences such as whitespace).
The post-merge commits up to
772ce79706(I mistakenly posted the review from a tab that didn't have the latest commit) look good and I can't think of something else that would be missing.I made this review under the assumption that https://github.com/ARMmbed/mbedtls/pull/2395 is correct. Since then you've found one bug (https://github.com/ARMmbed/mbedtls/pull/2395/commits/3ea2687053748e701f4d28a644c4c6ea28f3b251) which will need to be fixed here.
@ -91,6 +91,7 @@ static inline psa_algorithm_t mbedtls_psa_translate_cipher_mode(if( taglen == 0 )return( PSA_ALG_CBC_NO_PADDING );/* Intentional fallthrough for taglen != 0 *//* fallthrough */Which compiler requires this specific comment, as opposed to any comment or any comment containing “fallthrough”?
@ -327,0 +335,4 @@{mbedtls_printf( "%s:%i: Input param failed - %s\n",file, line, failure_condition );mbedtls_exit( MBEDTLS_EXIT_FAILURE );Yuck, it's really ugly to have this in every example. But I guess that it's a design problem with
MBEDTLS_CHECK_PARAMSthat it's too late to fix.Stray incursion of the crypto submodule.
@ -83,1 +106,4 @@* It may jump to the \c exit label.** \param TEST The test expression to be tested.*/This is missing some useful text from the crypto branch:
@ -91,6 +91,7 @@ static inline psa_algorithm_t mbedtls_psa_translate_cipher_mode(if( taglen == 0 )return( PSA_ALG_CBC_NO_PADDING );/* Intentional fallthrough for taglen != 0 *//* fallthrough */@ -91,6 +91,7 @@ static inline psa_algorithm_t mbedtls_psa_translate_cipher_mode(if( taglen == 0 )return( PSA_ALG_CBC_NO_PADDING );/* Intentional fallthrough for taglen != 0 *//* fallthrough */Oh. Wait, I'm still confused: reading the documentation, I don't see a setting that recognizes
/* fallthrough */but not/* foo fallthrough bar */— it should be/*-fallthrough*/or/*@fallthrough@*/.@ -91,6 +91,7 @@ static inline psa_algorithm_t mbedtls_psa_translate_cipher_mode(if( taglen == 0 )return( PSA_ALG_CBC_NO_PADDING );/* Intentional fallthrough for taglen != 0 *//* fallthrough */Maybe it's because of the surrounding text.
I've pasted the error into the commit message in https://github.com/Patater/mbed-crypto/tree/update-dev-tls-dev-crypto-merge-6
@ -91,6 +91,7 @@ static inline psa_algorithm_t mbedtls_psa_translate_cipher_mode(if( taglen == 0 )return( PSA_ALG_CBC_NO_PADDING );/* Intentional fallthrough for taglen != 0 *//* fallthrough */Ah, yes, got it. The last bullet point for
-Wimplicit-fallthrough=3allowsfallthroughwith optional surrounding whitespace, together with a few other variants. (And the previous two bullet points allow all-uppercase or titlecase variants.)I'd prefer to change this comment to something like
/*-fallthrough*/to make it clear that it's a compiler-warning directive, as opposed to the comment above which is for humans.Removed
@ -83,1 +106,4 @@* It may jump to the \c exit label.** \param TEST The test expression to be tested.*/OK, re-added. We should consider adding this up to TLS to align if we want this text everywhere.
Rebased to take in latest TLS PR and address code review feedback.
Previous branch at https://github.com/Patater/mbed-crypto/tree/update-dev-tls-dev-crypto-merge-pr-2
Diff against last branch at https://gist.github.com/Patater/5c8847915bd84b96725e6d3b0d322866
The current base, if you want to reproduce my merge, is still
github.com/ARMmbed/mbed-crypto@1fb011f2a6No more rebasing right now. Feel free to review.
The CI passed, except for know-fragile timing selftest on BSD in the Jenkins pr-merge run.
@ -83,1 +106,4 @@* It may jump to the \c exit label.** \param TEST The test expression to be tested.*/@AndrzejKurek when you update the submodule in https://github.com/ARMmbed/mbedtls/pull/2395 could you also add a commit to include this text in our version as well?
I'm approving based on my previous approval and the github-generated force-push diff since the version I previously reviewed.
@ -83,1 +106,4 @@* It may jump to the \c exit label.** \param TEST The test expression to be tested.*/Sure, I wrote that down on my to-do list.
@ -83,1 +106,4 @@* It may jump to the \c exit label.** \param TEST The test expression to be tested.*/This can be done by cherry-picking 8954d0c ("Write documentation for TEST_ASSERT") and resolving the conflicts in the same way I did here, to give credit where credit is due and hopefully help avoid multiple implementation conflicts if we ever merge helpers.function between Mbed TLS and Mbed Crypto again.
@ -91,6 +91,7 @@ static inline psa_algorithm_t mbedtls_psa_translate_cipher_mode(if( taglen == 0 )return( PSA_ALG_CBC_NO_PADDING );/* Intentional fallthrough for taglen != 0 *//* fallthrough */@gilles-peskine-arm Or we could even go with
(I'm not even kidding. (I think.))
@AndrzejKurek We'll need this side-ported to TLS. Perhaps this could be part of 2395? Either way, please let @k-stachowiak know as he's been assigned to fix this on the TLS side.
I am happy with the new merge commit
67ea2c5e6dmade with the updated development-psa PR https://github.com/ARMmbed/mbedtls/pull/2395 and the same mbed-crypto/developmentbase as my previous review. I reviewed the new merge commit by diff with the previous one. I am also happy with the commit message. I am also happy with the additional commits on top of that (rebased from the previous iteration).2 approvals and CI is ok except for a FreeBSD timing test which is a known issue. This is ready for merge.
@ -83,1 +106,4 @@* It may jump to the \c exit label.** \param TEST The test expression to be tested.*/Done.
@ -91,6 +91,7 @@ static inline psa_algorithm_t mbedtls_psa_translate_cipher_mode(if( taglen == 0 )return( PSA_ALG_CBC_NO_PADDING );/* Intentional fallthrough for taglen != 0 *//* fallthrough */Done.