b4yuan commented

(Migrated from github.com)

Summary

This pull request introduces a CodeQL workflow to enhance the security analysis of this repository.

What is CodeQL

CodeQL is a static analysis tool that helps identify and mitigate security vulnerabilities. It is primarily intra-function but does provide some support for inter-function analysis. By integrating CodeQL into a GitHub Actions workflow, it can proactively identify and address potential issues before they become security threats.

For more information on CodeQL and how to interpret its results, refer to the GitHub documentation and the CodeQL documentation (https://codeql.github.com/ and https://codeql.github.com/docs/).

What this PR does

We added a new CodeQL workflow file (.github/workflows/codeql.yml) that

Runs on every pull request (functionality to run on every push to main branches is included as a comment for convenience).
Runs daily.
Excludes queries with a high false positive rate or low-severity findings.
Does not display results for git submodules, focusing only on our own codebase.

Validation

To validate the functionality of this workflow, we have run several test scans on the codebase and reviewed the results. The workflow successfully compiles the project, identifies issues, and provides actionable insights while reducing noise by excluding certain queries and third-party code.

Using the workflow results

If this pull request is merged, the CodeQL workflow will be automatically run on every push to the main branch and on every pull request to the main branch. To view the results of these code scans, follow these steps:

Under the repository name, click on the Security tab.
In the left sidebar, click Code scanning alerts.

Is this a good idea?

We are researchers at Purdue University in the USA. We are studying the potential benefits and costs of using CodeQL on open-source repositories of embedded software.

We wrote up a report of our findings so far. The TL;DR is “CodeQL outperforms the other freely-available static analysis tools, with fairly low false positive rates and lots of real defects”. You can read about the report here: https://arxiv.org/abs/2310.00205

Review of engineering hazards

License: see the license at https://github.com/github/codeql-cli-binaries/blob/main/LICENSE.md:

Here's what you may also do with the Software, but only with an Open Source Codebase and subject to the License Restrictions provisions below:

Perform analysis on the Open Source Codebase.

If the Open Source Codebase is hosted and maintained on GitHub.com, generate CodeQL databases for or during automated analysis, CI, or CD.

False positives: We find that around 20% of errors are false positives, but that these FPs are polarized and only a few rules contribute to most FPs. We find that the top rules contributing to FPs are: cpp/uninitialized-local, cpp/missing-check-scanf, cpp/suspicious-pointer-scaling, cpp/unbounded-write, cpp/constant-comparison, and cpp/inconsistent-null-check. Adding a filter to filter out certain rules that contribute to a high FP rate can be done simply in the workflow file.

# Summary This pull request introduces a CodeQL workflow to enhance the security analysis of this repository. # What is CodeQL CodeQL is a static analysis tool that helps identify and mitigate security vulnerabilities. It is primarily intra-function but does provide some support for inter-function analysis. By integrating CodeQL into a GitHub Actions workflow, it can proactively identify and address potential issues before they become security threats. For more information on CodeQL and how to interpret its results, refer to the GitHub documentation and the CodeQL documentation (https://codeql.github.com/ and https://codeql.github.com/docs/). # What this PR does We added a new CodeQL workflow file (.github/workflows/codeql.yml) that - Runs on every pull request (functionality to run on every push to main branches is included as a comment for convenience). - Runs daily. - Excludes queries with a high false positive rate or low-severity findings. - Does not display results for git submodules, focusing only on our own codebase. ## Validation To validate the functionality of this workflow, we have run several test scans on the codebase and reviewed the results. The workflow successfully compiles the project, identifies issues, and provides actionable insights while reducing noise by excluding certain queries and third-party code. ## Using the workflow results If this pull request is merged, the CodeQL workflow will be automatically run on every push to the main branch and on every pull request to the main branch. To view the results of these code scans, follow these steps: 1. Under the repository name, click on the Security tab. 2. In the left sidebar, click Code scanning alerts. # Is this a good idea? We are researchers at Purdue University in the USA. We are studying the potential benefits and costs of using CodeQL on open-source repositories of embedded software. We wrote up a report of our findings so far. The TL;DR is “CodeQL outperforms the other freely-available static analysis tools, with fairly low false positive rates and lots of real defects”. You can read about the report here: https://arxiv.org/abs/2310.00205 # Review of engineering hazards License: see the license at https://github.com/github/codeql-cli-binaries/blob/main/LICENSE.md: ``` Here's what you may also do with the Software, but only with an Open Source Codebase and subject to the License Restrictions provisions below: Perform analysis on the Open Source Codebase. If the Open Source Codebase is hosted and maintained on GitHub.com, generate CodeQL databases for or during automated analysis, CI, or CD. ``` False positives: We find that around 20% of errors are false positives, but that these FPs are polarized and only a few rules contribute to most FPs. We find that the top rules contributing to FPs are: cpp/uninitialized-local, cpp/missing-check-scanf, cpp/suspicious-pointer-scaling, cpp/unbounded-write, cpp/constant-comparison, and cpp/inconsistent-null-check. Adding a filter to filter out certain rules that contribute to a high FP rate can be done simply in the workflow file.

void256 commented

2023-10-28 05:50:40 -05:00

(Migrated from github.com)

Hi researchers at Purdue University in the USA,

I have no authority in this project and it's totally up to rafagafe on what to do with this pull request.

And you certainly have not asked for my opinion 😉 but as a former maintainer of an open source software I couldn't resist.

Is this a good idea?

No, it's not. This pull request is a terrible idea.

This will add the burden of maintaining the integration with CodeQL to the project maintainers (currently that is rafagafe only, I think).

At some point, maybe in the far future (when you all have left university) this integration will break or it needs to be modified. This might not appear to be much work now and maybe you'll say, we can take care of that. And for the time being that is all fine but at some point the maintainer of this project will be left alone with it.

We are studying the potential benefits and costs of using CodeQL on open-source repositories of embedded software.

Why can you not study that in your own fork of the repository? Why does it have to be integrated here?

Why not simply run CodeQL on your own fork, identify the issues it might find and provide pull requests to fix potential security problems? That would be more helpful, I think.

Maybe add to your research the costs of mainting the integration of CodeQL - in your own fork and over a longer period of time - as well? 😄

Much success with your research!

Best regards
void256

Hi researchers at Purdue University in the USA, I have no authority in this project and it's totally up to rafagafe on what to do with this pull request. And you certainly have not asked for my opinion :wink: but as a former maintainer of an open source software I couldn't resist. > Is this a good idea? No, it's not. This pull request is a terrible idea. This will add the burden of maintaining the integration with CodeQL to the project maintainers (currently that is rafagafe only, I think). At some point, maybe in the far future (when you all have left university) this integration will break or it needs to be modified. This might not appear to be much work now and maybe you'll say, we can take care of that. And for the time being that is all fine but at some point the maintainer of this project will be left alone with it. > We are studying the potential benefits and costs of using CodeQL on open-source repositories of embedded software. Why can you not study that in your own fork of the repository? Why does it have to be integrated here? Why not simply run CodeQL on your own fork, identify the issues it might find and provide pull requests to fix potential security problems? That would be more helpful, I think. Maybe add to your research the costs of mainting the integration of CodeQL - in your own fork and over a longer period of time - as well? 😄 Much success with your research! Best regards void256

b4yuan commented

2023-11-29 15:44:47 -06:00

(Migrated from github.com)

@rafagafe Pinging to check in on a possible followup to this PR?

This pull request is broken due to missing fork information.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin 'IVOES/master:IVOES/master'

git switch 'IVOES/master'

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch 'master'

git merge --no-ff 'IVOES/master'

git switch 'IVOES/master'

git rebase 'master'

git switch 'master'

git merge --ff-only 'IVOES/master'

git switch 'IVOES/master'

git rebase 'master'

git switch 'master'

git merge --no-ff 'IVOES/master'

git switch 'master'

git merge --squash 'IVOES/master'

git switch 'master'

git merge --ff-only 'IVOES/master'

git switch 'master'

git merge 'IVOES/master'

git push origin 'master'

Rows
Columns

Add CodeQL Workflow for Code Security Analysis #17

Summary

What is CodeQL

What this PR does

Validation

Using the workflow results

Is this a good idea?

Review of engineering hazards

Checkout

Merge