gh-Mbed-TLS/mbedtls-framework
Watch
1
0
Fork
You've already forked mbedtls-framework
 0
Code Issues 39 Pull requests 42 Releases Activity

DI: Reduce the CI load for testing the framework #251

New issue
Open
opened 2025-12-10 03:58:36 -06:00 by gilles-peskine-arm · 0 comments
gilles-peskine-arm commented 2025-12-10 03:58:36 -06:00 (Migrated from github.com)

We currently test the framework by running the complete CI of all consuming branches. There are currently 3 consuming branches, and there will be 5 when we make the next LTS.

The goal of this issue is to reduce the CI load, and only run a subset of the tests on each branch that gives us enough confidence that the framework is still good.

This issue is a first step. Expected outcomes:

  1. Preferably, first reduction: e.g. define a whilelist or blacklist of all.sh components.
  2. Plans for later.
We currently test the framework by running the complete CI of all consuming branches. There are currently 3 consuming branches, and there will be [5 when we make the next LTS](https://github.com/Mbed-TLS/mbedtls-framework/issues/250). The goal of this issue is to reduce the CI load, and only run a subset of the tests on each branch that gives us enough confidence that the framework is still good. This issue is a first step. Expected outcomes: 1. Preferably, first reduction: e.g. define a whilelist or blacklist of `all.sh` components. 2. Plans for later.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
archive/20261015T005621Z-force/pull/322
archive/20260929T231924Z-rescue/rohankl-arm/mldsa-899-framework
archive/20260929T231924Z-rescue/Thomasdjb/pr/spake2p-9342-public-key-import-export
archive/20260929T231924Z-rescue/gilles-peskine-arm/fork-helper-unit-tests-create-framework
archive/20260929T231924Z-rescue/gilles-peskine-arm/iar-1.1.0-check-names
archive/20260929T231924Z-rescue/gilles-peskine-arm/mbedtls_test_stack_usage-create
archive/20260929T231924Z-rescue/sridhar-design/cert_chain
archive/20260929T231924Z-rescue/JulioLoayzaM/add-fuzzing-seeds
dev/gilles-peskine-arm/generate_test_code-unicode-framework
archive/20260929T231924Z-rescue/tpambor/key-wrap
archive/20260929T231924Z-rescue/gilles-peskine-arm/not-not-grep-framework
archive/20260929T231924Z-rescue/davidhorstmann-arm/add-python-package-update-script
archive/20260929T231924Z-rescue/minosgalanakis/tools/fix-auto-generated-files
framework_mergeback
archive/20260929T231924Z-rescue/bjwtaylor/empty-digest-failure
archive/20260929T231924Z-rescue/asmellby/feature/dispatch-library
archive/20260929T231924Z-rescue/sigvartmh/spake2p-test-framework
archive/20260929T231924Z-rescue/bensze01/component-groups
archive/20260929T231924Z-rescue/bensze01/undef
archive/20260929T231924Z-rescue/gilles-peskine-arm/server9-2048
archive/20260929T231924Z-rescue/gilles-peskine-arm/all.sh-call_stack_trace
archive/20260929T231924Z-rescue/Marandil/fix/mbedtls-4314-test-data
archive/20260929T231924Z-rescue/bensze01/ci-testing-development
archive/20260929T231924Z-rescue/schneider-electric/dev/plaj-se/parse-rsassa-pss
archive/20260929T231924Z-rescue/gilles-peskine-arm/pk_helpers-coverity-20260224
archive/20260929T231924Z-rescue/gilles-peskine-arm/python-scripts-vintage-split-proposal
archive/20260929T231924Z-rescue/davidhorstmann-arm/code-size-measurement-script
archive/20260929T231924Z-rescue/gilles-peskine-arm/sha3-shake-builtin-framework
archive/20260929T231924Z-rescue/mpg/fix-quiet
archive/20260929T231924Z-rescue/TakutoYamane/fix-typo
archive/20260929T231924Z-rescue/rgqsch/feature/csr_challenge_password
archive/20260929T231924Z-rescue/gilles-peskine-arm/make_generated_files-improve-for-3.6
archive/20260929T231924Z-rescue/irwir/lean_and_mean
archive/20260929T231924Z-rescue/gilles-peskine-arm/all.sh-generate-only-before-checks-framework
archive/20260929T231924Z-rescue/gilles-peskine-arm/test-data_t-empty_is_empty-framework
archive/20260929T231924Z-rescue/jean-baptisteboric-eaton/pkcs7_test_artifacts
archive/20260929T231924Z-rescue/gilles-peskine-arm/assemble_changelog-utffromtimestamp
dev/bensze01/public-only-commit
archive/20260929T231924Z-rescue/gilles-peskine-arm/config-py-help-directory-percent-framework
archive/20260929T231924Z-rescue/valeriosetti/issue9255-framework
features/tls-defragmentation
archive/20260929T231924Z-rescue/gilles-peskine-arm/outcome-log-make-3.6-framework
md-psa-dispatch-3.6-framework
archive/20260929T231924Z-rescue/benmcollins/add-rsassa-pss-keys
archive/20260929T231924Z-rescue/BrianSipos/119-add-name-constraints
issue-39-follow-up
revert-84-issue-72
dev/paul-elliott-arm/add_iop_key_agreement_exercise_key
dev/gilles-peskine-arm/psa-storage-test-cases-never-supported-positive-framework
issue-72
issue-39
dev/davidhorstmann-arm/port-psa-exercise-key-changes
dev/davidhorstmann-arm/add-some-test-helpers
archive/20260929T231924Z-rescue/paul-elliott-arm/remove_re_sub_count_positional_arg
dev/mpg/move-all-core-dev
issue-51
dev/gilles-peskine-arm/outcome-check-repeated-configurations-framework
dev/gilles-peskine-arm/generate_config_tests-require_psa
dev/gilles-peskine-arm/analyze_outcome-split-framework
dev/gilles-peskine-arm/config-full-booleans-only-framework
archive/20260929T231924Z-rescue/juhaylinen/disable_allow_abbrev
dev/gilles-peskine-arm/ssl-opt-psk-detection-framework
fix-everst-build
dev/minos/minor/code_wrappers_fix_identation
dev/minos/feature/code_wrappers
dev/davidhorstmann-arm/add-test-helpers
dev/davidhorstmann-arm/add-test-data-files
dev/gilles-peskine-arm/test_suite_config-booleans-framework
dev/ronald-cron-arm/fix-generation-of-psa-test-wrappers
dev/ronald-cron-arm/move-psa-headers
dev/external-contrib-copy/mbedtls-pr-9189
dev/davidhorstmann-arm/add-test-generation-files
dev/davidhorstmann-arm/add-mbedtls-framework-module
dev/davidhorstmann-arm/move-test-code
dev/davidhorstmann-arm/move-scripts
dev/gilles-peskine-arm/create-makefiles
mbedtls-4.2.0_tf-psa-crypto-1.2.0
mbedtls-4.1.0_tf-psa-crypto-1.1.0
mbedtls-4.0.0_tf-psa-crypto-1.0.0
Labels
Clear labels   
approved
Design and code approved - may be waiting for CI or backports

  
bug
Something isn't working

  
dependencies
Pull requests that update a dependency file

  
DO-NOT-MERGE

   
enhancement
New feature or request

  
good-first-issue
Good for newcomers

  
help-wanted
This issue is not being actively worked on, but PRs welcome.

  
historical-reviewed
Reviewed & agreed to keep legacy PR/issue

  
historical-reviewing
Currently reviewing (for legacy PR/issues)

  
needs-adoption
stalled PR that someone should pick up and complete

  
needs-ci
Needs to pass CI tests

  
needs-design-approval

   
needs-info
An issue or PR which needs further info from the reporter / author

  
needs-preceding-pr
Requires another PR to be merged first

  
needs-review
Every commit must be reviewed by at least two team members.

  
needs-reviewer
This PR needs someone to pick it up for review

  
needs-work

   
priority-high
High priority - will be reviewed soon

  
priority-low
Low priority - this may not receive review soon

  
priority-medium
Medium priority - this can be reviewed as time permits

  
priority-scheduled
This PR is big - it will require time to be scheduled for review

  
priority-very-high
Highest priority - prioritise this over other review work

  
single-reviewer
This PR qualifies for having only one reviewer

  
size-l
Estimated task size: large (2w+)

  
size-m
Estimated task size: medium (~1w)

  
size-s
Estimated task size: small (~2d)

  
size-xs
Estimated task size: extra small (a few hours at most)

No labels
approved
bug
dependencies
DO-NOT-MERGE
enhancement
good-first-issue
help-wanted
historical-reviewed
historical-reviewing
needs-adoption
needs-ci
needs-design-approval
needs-info
needs-preceding-pr
needs-review
needs-reviewer
needs-work
priority-high
priority-low
priority-medium
priority-scheduled
priority-very-high
single-reviewer
size-l
size-m
size-s
size-xs
Milestone
Clear milestone
No items
No milestone
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
gh-Mbed-TLS/mbedtls-framework#251
No description provided.