Problem adding sequence points to a method with portable PDB #840

Open
opened 2022-02-22 07:25:02 -06:00 by Zastai · 2 comments
Zastai commented 2022-02-22 07:25:02 -06:00 (Migrated from github.com)

Whenever I attempt to add a sequence point to a method with a portable PDB involved, the resulting PDB has a broken SequencePoints blob, resulting in

System.BadImageFormatException: Invalid compressed integer.
   at System.Reflection.Throw.InvalidCompressedInteger()
   at System.Reflection.Metadata.SequencePointCollection.Enumerator.MoveNext()

during processing, like when when hovering over the "SequencePoints" value when looking at the MethodDebugInformation table in ILSpy, or when running pdb2xml.

Looking at a before/after, for a case where I added only a single "hidden" (lines set to 0xfeefee) sequence point, the original blob contained 376 bytes:

01-00-00-19-80-D8-07-05-00-1D-10-00-05-02-24-06
00-51-00-5D-18-00-25-00-4B-02-00-20-00-35-04-00
1F-00-3A-0A-00-16-00-00-81-2F-00-36-04-08-1F-00
34-02-00-1A-00-24-02-00-11-00-34-02-00-1A-00-24
02-00-11-00-06-02-00-01-00-36-04-00-1F-00-34-02
00-1A-00-24-02-00-11-00-34-02-00-1A-00-24-02-00
11-00-06-02-00-01-00-36-04-00-1F-00-34-02-00-1A
00-24-02-00-11-00-34-02-00-1A-00-24-02-00-11-00
06-02-00-01-00-36-04-00-1F-00-34-02-00-1A-00-24
02-00-11-00-34-02-00-1A-00-24-02-00-11-00-06-02
00-01-00-36-04-00-1F-00-34-02-00-1A-00-24-02-00
11-00-34-02-00-1A-00-24-02-00-11-00-34-02-00-1A
00-24-02-00-11-00-34-02-00-1A-00-24-02-00-11-00
34-02-00-1A-00-24-02-00-11-00-06-02-00-01-00-36
04-00-1F-00-34-02-00-1A-00-24-02-00-11-00-34-02
00-1A-00-3E-02-00-1F-00-24-02-00-11-00-06-02-00
01-00-36-04-00-1F-00-34-02-00-1A-00-24-02-00-11
00-34-02-00-1A-00-24-02-00-11-00-06-02-00-01-00
36-04-00-1F-00-34-02-00-1A-00-24-02-00-11-00-34
02-00-1A-00-24-02-00-11-00-06-02-00-01-00-36-04
00-1F-00-34-02-00-1A-00-24-02-00-11-00-34-02-00
1A-00-24-02-00-11-00-06-02-00-01-00-36-04-00-1F
00-34-02-00-1A-00-24-02-00-11-00-34-02-00-1A-00
24-02-00-11-00-01-0E-75

and the new, broken one contains 386 bytes:

01-00-00-19-80-D8-07-22-00-1D-10-00-06-02-24-06
00-51-00-5D-18-00-25-00-4B-02-00-20-00-35-04-00
1F-00-3A-0A-00-16-00-00-81-6A-00-36-04-08-1F-00
34-02-00-1A-00-24-02-00-11-00-34-02-00-1A-00-24
02-00-11-00-06-02-00-05-00-36-04-00-1F-00-34-02
00-1A-00-24-02-00-11-00-34-02-00-1A-00-24-02-00
11-00-06-02-00-05-00-36-04-00-1F-00-34-02-00-1A
00-24-02-00-11-00-34-02-00-1A-00-24-02-00-11-00
06-02-00-05-00-36-04-00-1F-00-34-02-00-1A-00-24
02-00-11-00-34-02-00-1A-00-24-02-00-11-00-06-02
00-05-00-36-04-00-1F-00-34-02-00-1A-00-24-02-00
11-00-34-02-00-1A-00-24-02-00-11-00-34-02-00-1A
00-24-02-00-11-00-34-02-00-1A-00-24-02-00-11-00
34-02-00-1A-00-24-02-00-11-00-06-02-00-05-00-36
04-00-1F-00-34-02-00-1A-00-24-02-00-11-00-34-02
00-1A-00-3E-02-00-1F-00-24-02-00-11-00-06-02-00
05-00-36-04-00-1F-00-34-02-00-1A-00-24-02-00-11
00-34-02-00-1A-00-24-02-00-11-00-06-02-00-05-00
36-04-00-1F-00-34-02-00-1A-00-24-02-00-11-00-34
02-00-1A-00-24-02-00-11-00-06-02-00-05-00-36-04
00-1F-00-34-02-00-1A-00-24-02-00-11-00-34-02-00
1A-00-24-02-00-11-00-06-02-00-05-00-36-04-00-1F
00-34-02-00-1A-00-24-02-00-11-00-34-02-00-1A-00
24-02-00-11-00-01-0E-75-FF-FF-F8-49-00-00-87-B9
00-00

My gaze is drawn by that FFFFF8 sequence near the end. The method in question includes a "hidden" sequence point (string hashing code for a string-based switch), so it can't be the 0xfeefee causing an issue there. My suspicion is that either the PDB writing code does not handle negative values correctly for a "compressed integer", or other processing results in a negative value when it shouldn't.

Looking at the code that reads signed values back, the sign bit gets moved to the end. So if that 0xfffff849 is intended to be signed value -1975, I would expect to see something like 0x4f6f instead, if I read the code correctly (bit 0x40 in first byte to indicate: 2 bytes to read in total, then 0xf6f which is 1975 shifted left + the sign bit).

I'll try to make a minimal repro case, but hopefully this may already point in the right direction.

Whenever I attempt to add a sequence point to a method with a portable PDB involved, the resulting PDB has a broken SequencePoints blob, resulting in ``` System.BadImageFormatException: Invalid compressed integer. at System.Reflection.Throw.InvalidCompressedInteger() at System.Reflection.Metadata.SequencePointCollection.Enumerator.MoveNext() ``` during processing, like when when hovering over the "SequencePoints" value when looking at the MethodDebugInformation table in ILSpy, or when running [pdb2xml](https://github.com/dotnet/symreader-converter). Looking at a before/after, for a case where I added only a single "hidden" (lines set to 0xfeefee) sequence point, the original blob contained 376 bytes: ``` 01-00-00-19-80-D8-07-05-00-1D-10-00-05-02-24-06 00-51-00-5D-18-00-25-00-4B-02-00-20-00-35-04-00 1F-00-3A-0A-00-16-00-00-81-2F-00-36-04-08-1F-00 34-02-00-1A-00-24-02-00-11-00-34-02-00-1A-00-24 02-00-11-00-06-02-00-01-00-36-04-00-1F-00-34-02 00-1A-00-24-02-00-11-00-34-02-00-1A-00-24-02-00 11-00-06-02-00-01-00-36-04-00-1F-00-34-02-00-1A 00-24-02-00-11-00-34-02-00-1A-00-24-02-00-11-00 06-02-00-01-00-36-04-00-1F-00-34-02-00-1A-00-24 02-00-11-00-34-02-00-1A-00-24-02-00-11-00-06-02 00-01-00-36-04-00-1F-00-34-02-00-1A-00-24-02-00 11-00-34-02-00-1A-00-24-02-00-11-00-34-02-00-1A 00-24-02-00-11-00-34-02-00-1A-00-24-02-00-11-00 34-02-00-1A-00-24-02-00-11-00-06-02-00-01-00-36 04-00-1F-00-34-02-00-1A-00-24-02-00-11-00-34-02 00-1A-00-3E-02-00-1F-00-24-02-00-11-00-06-02-00 01-00-36-04-00-1F-00-34-02-00-1A-00-24-02-00-11 00-34-02-00-1A-00-24-02-00-11-00-06-02-00-01-00 36-04-00-1F-00-34-02-00-1A-00-24-02-00-11-00-34 02-00-1A-00-24-02-00-11-00-06-02-00-01-00-36-04 00-1F-00-34-02-00-1A-00-24-02-00-11-00-34-02-00 1A-00-24-02-00-11-00-06-02-00-01-00-36-04-00-1F 00-34-02-00-1A-00-24-02-00-11-00-34-02-00-1A-00 24-02-00-11-00-01-0E-75 ``` and the new, broken one contains 386 bytes: ``` 01-00-00-19-80-D8-07-22-00-1D-10-00-06-02-24-06 00-51-00-5D-18-00-25-00-4B-02-00-20-00-35-04-00 1F-00-3A-0A-00-16-00-00-81-6A-00-36-04-08-1F-00 34-02-00-1A-00-24-02-00-11-00-34-02-00-1A-00-24 02-00-11-00-06-02-00-05-00-36-04-00-1F-00-34-02 00-1A-00-24-02-00-11-00-34-02-00-1A-00-24-02-00 11-00-06-02-00-05-00-36-04-00-1F-00-34-02-00-1A 00-24-02-00-11-00-34-02-00-1A-00-24-02-00-11-00 06-02-00-05-00-36-04-00-1F-00-34-02-00-1A-00-24 02-00-11-00-34-02-00-1A-00-24-02-00-11-00-06-02 00-05-00-36-04-00-1F-00-34-02-00-1A-00-24-02-00 11-00-34-02-00-1A-00-24-02-00-11-00-34-02-00-1A 00-24-02-00-11-00-34-02-00-1A-00-24-02-00-11-00 34-02-00-1A-00-24-02-00-11-00-06-02-00-05-00-36 04-00-1F-00-34-02-00-1A-00-24-02-00-11-00-34-02 00-1A-00-3E-02-00-1F-00-24-02-00-11-00-06-02-00 05-00-36-04-00-1F-00-34-02-00-1A-00-24-02-00-11 00-34-02-00-1A-00-24-02-00-11-00-06-02-00-05-00 36-04-00-1F-00-34-02-00-1A-00-24-02-00-11-00-34 02-00-1A-00-24-02-00-11-00-06-02-00-05-00-36-04 00-1F-00-34-02-00-1A-00-24-02-00-11-00-34-02-00 1A-00-24-02-00-11-00-06-02-00-05-00-36-04-00-1F 00-34-02-00-1A-00-24-02-00-11-00-34-02-00-1A-00 24-02-00-11-00-01-0E-75-FF-FF-F8-49-00-00-87-B9 00-00 ``` My gaze is drawn by that FFFFF8 sequence near the end. The method in question includes a "hidden" sequence point (string hashing code for a string-based `switch`), so it can't be the 0xfeefee causing an issue there. My suspicion is that either the PDB writing code does not handle negative values correctly for a "compressed integer", or other processing results in a negative value when it shouldn't. Looking at [the code that reads signed values back](https://github.com/dotnet/runtime/blob/976428fa4778faa3257b9cd7d6d3ea76a43410a0/src/libraries/System.Reflection.Metadata/src/System/Reflection/Metadata/BlobReader.cs#L490), the sign bit gets moved to the end. So if that 0xfffff849 is intended to be signed value -1975, I would expect to see something like 0x4f6f instead, if I read [the code](https://github.com/dotnet/runtime/blob/976428fa4778faa3257b9cd7d6d3ea76a43410a0/src/libraries/System.Reflection.Metadata/src/System/Reflection/Internal/Utilities/MemoryBlock.cs#L146) correctly (bit 0x40 in first byte to indicate: 2 bytes to read in total, then 0xf6f which is 1975 shifted left + the sign bit). I'll try to make a minimal repro case, but hopefully this may already point in the right direction.
Zastai commented 2022-06-08 10:41:53 -05:00 (Migrated from github.com)

Note that while this issue prevents debugging in Visual Studio (because the entire PDB reading aborts), Rider is less affected (it seems to just silently discard the "bad" sequence points).

Note that while this issue prevents debugging in Visual Studio (because the entire PDB reading aborts), Rider is less affected (it seems to just silently discard the "bad" sequence points).
Zastai commented 2022-06-10 17:44:14 -05:00 (Migrated from github.com)

Looks like the problem is not so much the hidden sequence points - the sequence point records are emitted correctly.

However, the problem is that SignatureWriter.WriteSequencePoints assumes that MethodDebugInformation.SequencePoints is sorted by offset. However, that does not seem to be guaranteed. I'm not even certain that it's guaranteed that it would have duplicate offsets in it.

I'll prepare a PR.

Looks like the problem is not so much the hidden sequence points - the sequence point records are emitted correctly. However, the problem is that `SignatureWriter.WriteSequencePoints` assumes that `MethodDebugInformation.SequencePoints` is sorted by offset. However, that does not seem to be guaranteed. I'm not even certain that it's guaranteed that it would have duplicate offsets in it. I'll prepare a PR.
Sign in to join this conversation.
No milestone
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
gh-jbevain/cecil#840
No description provided.