Please validate rp2040.svd against CMSIS-SVD.xsd

juju2013 commented

2022-10-22 06:28:47 -05:00

(Migrated from github.com)

When passing rp2040.svd to a xml schema validator against CMSIS-SVD, it appears that there's many issue, like mandatory field missing, incorrect order of elements in a sequence etc.

Please validate your document.

When passing rp2040.svd to a [xml schema validator](https://www.utilities-online.info/xsdvalidation) against [CMSIS-SVD](https://github.com/ARM-software/CMSIS/blob/master/CMSIS/Utilities/CMSIS-SVD.xsd), it appears that there's many issue, like mandatory field missing, incorrect order of elements in a sequence etc. Please validate your document.

👍 1

lurch commented

2022-10-24 05:45:10 -05:00

(Migrated from github.com)

there's many issue, like mandatory field missing, incorrect order of elements in a sequence etc.

Yup, a few open-issues regarding SVD already, see https://github.com/raspberrypi/pico-sdk/search?q=svd&state=open&type=issues

Unfortunately fixing up our internal tooling to generate a "better" SVD file isn't at the top of our priority-list at the moment. (We're all pretty busy!)

Please validate your document.

As mentioned here the SVD file did originally pass the SVDConv validation, but I wasn't aware at the time that there were additional things that SVDConv doesn't catch.

> there's many issue, like mandatory field missing, incorrect order of elements in a sequence etc. Yup, a few open-issues regarding SVD already, see https://github.com/raspberrypi/pico-sdk/search?q=svd&state=open&type=issues Unfortunately fixing up our internal tooling to generate a "better" SVD file isn't at the top of our priority-list at the moment. (We're all pretty busy!) > Please validate your document. As mentioned [here](https://github.com/raspberrypi/pico-sdk/issues/352#issuecomment-824052623) the SVD file _did_ originally pass the [SVDConv](https://www.keil.com/pack/doc/CMSIS/SVD/html/svd_SVDConv_pg.html) validation, but I wasn't aware at the time that there were additional things that SVDConv doesn't catch.

JustAnother1 commented

2022-11-09 21:26:07 -06:00

(Migrated from github.com)

There is a validated SVD file for the RP2040 available for download

If there is still an issue with the file you can contact me directly info@chipselect.org

File also attached:
RP2040.svd.zip (compressed as github does not accept *.svd as attachment)

There is a validated SVD file for the RP2040 available for [download](https://chipselect.org/Raspberry%20Pi--RP2040.svd) If there is still an issue with the file you can contact me directly [info@chipselect.org](mailto:info@chipselect.org) File also attached: [RP2040.svd.zip](https://github.com/raspberrypi/pico-sdk/files/9977039/RP2040.svd.zip) (compressed as github does not accept *.svd as attachment)

ghollingworth commented

2022-11-10 04:51:17 -06:00

(Migrated from github.com)

Please note the SVD file is automatically generated from other files, as such the issue is not in the SVD file, but in the generator.

If you can instead provide a list of the 'class' of changes that are required, this may make it easier to achieve.

@JustAnother1 please don't reply to every issue related to SVD pointing at a fixed one, that is not really helping the issue process.

Thanks

Gordon

Please note the SVD file is automatically generated from other files, as such the issue is not in the SVD file, but in the generator. If you can instead provide a list of the 'class' of changes that are required, this may make it easier to achieve. @JustAnother1 please don't reply to every issue related to SVD pointing at a fixed one, that is not really helping the issue process. Thanks Gordon

JustAnother1 commented

2022-11-10 16:29:18 -06:00

(Migrated from github.com)

@ghollingworth If you read the first message in this issue(by @juju2013) you will see a link to an online validation tool, as well as a link to the SVD schema(*.xsd). You can just put the linked xsd and the RP2040 SVD in that web site and you will get all the issues reported . If you want to do it offline then there are several XML validation tools available fro free. For me it was as easy as doing xmllint --noout --schema CMSIS-SVD.xsd RP2040.svd that then gives you the complete list of issues in the SVD file. With that you can start changing the "generator".

Do you have a link to the public repository of the "generator" ? If so I would love to take a look and help fix these issues. As the people as Raspberry Pi are so busy that should be a welcome help, right?

Lurch said they are pretty busy. There is no benefit for anybody if Raspberry Pi staff invests time into fixing there SVD generator if there is already a working open source SVD generator available. They should spend their time in making the products better.

Also fixing the issues is not needed anyway. They can just use the chipselect generated file. Or they can use the generator that chipselect uses to generate the file. That generator is open source. As everything chipselect.

If I get access to those "other files" you mention then I can update the SVD to the changes from those files. I could even give chipselect an interface / API for Raspberry Pi to push in the updated information to then pull out the updated SVD. If you don't trust me you can setup a private chipselect instance. I will help you set that up no problem.

I'm just offering my help here. If you don't like me you can take the open source chipselect stuff from my github repos and use that to fix your issues. you can also use the XML schema validator tool of your choice and look into the chipselect SVD to see how to fix the issues it will report on the official Raspberry Pi file.

Also please excuse me if I messed with your issue process, but I did not only post the chipselect file here to help Raspberry Pi. I also want to help the people that report the issue by suggesting them a solution. A working SVD might not help you or the proprietary generator, but maybe it helps juju2013 with her pysvd project or whatever else was her need for a complaint file.

@ghollingworth If you read the first message in this issue(by @juju2013) you will see a link to an online validation tool, as well as a link to the SVD schema(*.xsd). You can just put the linked xsd and the RP2040 SVD in that web site and you will get all the issues reported . If you want to do it offline then there are several XML validation tools available fro free. For me it was as easy as doing `xmllint --noout --schema CMSIS-SVD.xsd RP2040.svd` that then gives you the complete list of issues in the SVD file. With that you can start changing the "generator". Do you have a link to the public repository of the "generator" ? If so I would love to take a look and help fix these issues. As the people as Raspberry Pi are so busy that should be a welcome help, right? Lurch said they are pretty busy. There is no benefit for anybody if Raspberry Pi staff invests time into fixing there SVD generator if there is already a working open source SVD generator available. They should spend their time in making the products better. Also fixing the issues is not needed anyway. They can just use the chipselect generated file. Or they can use the generator that chipselect uses to generate the file. That generator is open source. As everything chipselect. If I get access to those "other files" you mention then I can update the SVD to the changes from those files. I could even give chipselect an interface / API for Raspberry Pi to push in the updated information to then pull out the updated SVD. If you don't trust me you can setup a private chipselect instance. I will help you set that up no problem. I'm just offering my help here. If you don't like me you can take the open source chipselect stuff from my github repos and use that to fix your issues. you can also use the XML schema validator tool of your choice and look into the chipselect SVD to see how to fix the issues it will report on the official Raspberry Pi file. Also please excuse me if I messed with your issue process, but I did not only post the chipselect file here to help Raspberry Pi. I also want to help the people that report the issue by suggesting them a solution. A working SVD might not help you or the proprietary generator, but maybe it helps [juju2013](https://github.com/juju2013) with her pysvd project or whatever else was her need for a complaint file.

ghollingworth commented

2022-11-11 00:23:14 -06:00

(Migrated from github.com)

Thanks, but the generator is not open source because it's connected very closely to the verilog IP. I'll get someone to look at it in the next couple of weeks.

andygpz11 commented

2022-11-15 08:27:11 -06:00

(Migrated from github.com)

@JustAnother1 Have you tried the .svd file you generated with the arm SVDConv tool?
Although it passes the validator you linked, SVDConv reports a lot of errors and warnings when asked to produce a header.
As Gordon has explained, we will see if we can improve our generator and make an updated RP2040.svd
Clearly we must be sure that the header definitions produced from it are still correct.
Thanks.

@JustAnother1 Have you tried the .svd file you generated with the arm SVDConv tool? Although it passes the validator you linked, SVDConv reports a lot of errors and warnings when asked to produce a header. As Gordon has explained, we will see if we can improve our generator and make an updated RP2040.svd Clearly we must be sure that the header definitions produced from it are still correct. Thanks.

JustAnother1 commented

2022-11-15 09:45:32 -06:00

(Migrated from github.com)

@andygpz11 Initially I did not test it with SVDConv.
Validating against the schema only tests that the XML complies to the rules laid out in the schema. Those rules mostly specify allowed tag names and data ranges.

I now checked with the SVDConv tool. And yes it reports issues. Those are stuff like:

*** INFO M317: Raspberry Pi--RP2040.svd (Line 30) Interrupt <description> not set.

My file is created from the RaspberryPi SVD file. So if there was no description in there, then I also have no description. So this is basically independent of the Convert tables to SVD format but more a information missing kind of issue.

*** INFO M347: Raspberry Pi--RP2040.svd (Line 45) Field 'AINSEL' (width < 6Bit) without any <enumeratedValue> found.

This seems to go into the same direction. I compared it with the data sheet and the SVD has basically the same description for this field. But the SVDConv tool requires that all valid values need to be listed separately (enumeratedValue) This information is just missing.
If someone can point me to where I can get the missing information then I would fix all these issues.

@andygpz11 Initially I did not test it with SVDConv. Validating against the schema only tests that the XML complies to the rules laid out in the schema. Those rules mostly specify allowed tag names and data ranges. I now checked with the SVDConv tool. And yes it reports issues. Those are stuff like: `*** INFO M317: Raspberry Pi--RP2040.svd (Line 30) Interrupt <description> not set.` My file is created from the RaspberryPi SVD file. So if there was no description in there, then I also have no description. So this is basically independent of the Convert tables to SVD format but more a information missing kind of issue. `*** INFO M347: Raspberry Pi--RP2040.svd (Line 45) Field 'AINSEL' (width < 6Bit) without any <enumeratedValue> found.` This seems to go into the same direction. I compared it with the data sheet and the SVD has basically the same description for this field. But the SVDConv tool requires that all valid values need to be listed separately (enumeratedValue) This information is just missing. If someone can point me to where I can get the missing information then I would fix all these issues.

andygpz11 commented

2022-11-15 10:58:29 -06:00

(Migrated from github.com)

Hi @JustAnother1, thank you for your offer of help but as Gordon explained we want to improve our generator.

We expect to just press a button so that documentation is produced from a canonical source.

If we "fix-up" autogenerated files we will have to do this each and every time we edit any of the register descriptions in RTL and want to generate/push a revised RP2040.svd

Sorry but this approach makes no sense; Gordon has already explained why we can't share the tooling.

We do hope to produce a revised RP2040.svd in due course.

May I ask if this is actually causing you a problem, other than you would like to be able to fix it yourself?

Thanks.

Hi @JustAnother1, thank you for your offer of help but as Gordon explained we want to improve our generator. We expect to just press a button so that documentation is produced from a canonical source. If we "fix-up" autogenerated files we will have to do this each and every time we edit any of the register descriptions in RTL and want to generate/push a revised RP2040.svd Sorry but this approach makes no sense; Gordon has already explained why we can't share the tooling. We do hope to produce a revised RP2040.svd in due course. May I ask if this is _actually_ causing you a problem, other than you would like to be able to fix it yourself? Thanks.

JustAnother1 commented

2022-11-15 11:39:36 -06:00

(Migrated from github.com)

I agree that it makes most sense to add the missing documentation to the source files that are then used to generate the documentation. My suggestion was that chipselect could be part of what happens once you press that magic button.

The SVD file is intended to be used for debugging. Tools like SEGGER Ozone use the information from the SVD file to know which addresses belong to Special Function Registers and to decode the information read from such addresses.

If the SVD file is not valid (as in does not comply to the Schema) than all software that tries to use it as debugging help or other uses might run into problems. Like a mp3 player software that should read in a jpeg file.

Please keep in mind that I did not report this issue and that this issue is not the only one SVD related that has been reported.

I just saw an opportunity to help you. I can understand why you want to solve the issue yourself, but I don't know how I can help with that.

I agree that it makes most sense to add the missing documentation to the source files that are then used to generate the documentation. My suggestion was that chipselect could be part of what happens once you press that magic button. The SVD file is intended to be used for debugging. Tools like SEGGER Ozone use the information from the SVD file to know which addresses belong to Special Function Registers and to decode the information read from such addresses. If the SVD file is not valid (as in does not comply to the Schema) than all software that tries to use it as debugging help or other uses might run into problems. Like a mp3 player software that should read in a jpeg file. Please keep in mind that I did not report this issue and that this issue is not the only one SVD related that has been reported. I just saw an opportunity to help you. I can understand why you want to solve the issue yourself, but I don't know how I can help with that.

andygpz11 commented

2022-11-18 06:02:18 -06:00

(Migrated from github.com)

Hi @juju2013, @JustAnother1,

Having done some digging I found that validator linked in https://github.com/raspberrypi/pico-sdk/issues/1076#issue-1419250432 expects correctly ordered attribute elements, some of which are incorrect in the currently issued rp2040.svd file. We have fixed this and I attach a new file.

But we also found some unexpected results from that validator, even with the updated file. It does not accept escaped character sequences such as & and < which we think are 100% correct as used.

If we work around those issues (replace the escape sequences with pure text) the validator has issues with elements using the 'scaledNonNegativeInteger' type, for example 2 is flagged as an error.

Note: I have found both & etc. etc. and identically defined and used in other vendors’ svd files. None of those I have tried pass with this validator for the above reasons.

We did find https://www.liquid-technologies.com/online-xsd-validator
I have been able to successfully test chunks of the revised file (always starting with the top section of course) after cutting out large sections to ensure that the size pasted for test is (well under) the 200k limit imposed by that site.

There may be other sites without a size restriction of course.

The tools we use, xmllint and Arm’s SVDConv, don’t mandate element ordering (hence our non-discovery of this).

HTH.

rp2040-gh-1076.zip

Hi @juju2013, @JustAnother1, Having done some digging I found that validator linked in https://github.com/raspberrypi/pico-sdk/issues/1076#issue-1419250432 expects correctly ordered attribute elements, some of which are incorrect in the currently issued rp2040.svd file. We have fixed this and I attach a new file. But we also found some unexpected results from that validator, even with the updated file. It does not accept escaped character sequences such as & and < which we think are 100% correct as used. If we work around those issues (replace the escape sequences with pure text) the validator has issues with elements using the 'scaledNonNegativeInteger' type, for example <nvicPrioBits>2</nvicPrioBits> is flagged as an error. Note: I have found both & etc. etc. and <nvicPrioBits> identically defined and used in other vendors’ svd files. None of those I have tried pass with this validator for the above reasons. We did find https://www.liquid-technologies.com/online-xsd-validator I have been able to successfully test chunks of the revised file (always starting with the top section of course) after cutting out large sections to ensure that the size pasted for test is (well under) the 200k limit imposed by that site. There may be other sites without a size restriction of course. The tools we use, xmllint and Arm’s SVDConv, don’t mandate element ordering (hence our non-discovery of this). HTH. [rp2040-gh-1076.zip](https://github.com/raspberrypi/pico-sdk/files/10041041/rp2040-gh-1076.zip)

JustAnother1 commented

2022-11-18 06:59:10 -06:00

(Migrated from github.com)

good news:
$ xmllint --noout --schema CMSIS-SVD.xsd rp2040-gh-1076.svd rp2040-gh-1076.svd validates

good news: `$ xmllint --noout --schema CMSIS-SVD.xsd rp2040-gh-1076.svd rp2040-gh-1076.svd validates`

JustAnother1 commented

2022-11-18 07:02:50 -06:00

(Migrated from github.com)

On the other hand ARMs SVDConv still has lot to say,..

andygpz11 commented

2022-11-18 07:20:33 -06:00

(Migrated from github.com)

There is one warning, relating to the case of the filename. We use -x M223 to remove that from the output
We do not think the INFO messages are problematic and they can also be supressed of course.

There is one warning, relating to the case of the filename. We use `-x M223` to remove that from the output We do not think the INFO messages are problematic and they can also be supressed of course.

kilograham commented

2023-05-26 10:44:25 -05:00

(Migrated from github.com)

can this be closed?

andygpz11 commented

2023-05-26 11:48:49 -05:00

(Migrated from github.com)

I think not quite. I want to check the flow to ensure a "fresh" SDK contains an SVD that's as "good" as the one linked here.
I will do this in the next few days.

I think not quite. I want to check the flow to ensure a "fresh" SDK contains an SVD that's as "good" as the one linked here. I will do this in the next few days.

kilograham commented

2023-06-09 09:39:39 -05:00

(Migrated from github.com)

updated svd merged into develop

Rows
Columns

Please validate rp2040.svd against CMSIS-SVD.xsd #1076