Fix rounding error when getting VisualLine #425

Open
trijnsburger wants to merge 1 commit from trijnsburger/Fix-rounding-error-when-getting-VisualLine into master
trijnsburger commented 2024-05-27 09:37:31 -05:00 (Migrated from github.com)

A rounding error occasionally occurs here (found when using RectangleSelect), causing this method to unexpectedly return null, which in turn causes RectangleSelection.ReplaceSelectionWithText() to incorrectly set textArea.Caret.Position to (Line = 1, Column = 1).

Note that the epsilon value of 1e-12 is an estimation based on the accuracy of the visualTop value, which has 13 decimals when its integer value is in the thousands (4 digits).

A rounding error occasionally occurs here (found when using RectangleSelect), causing this method to unexpectedly return null, which in turn causes RectangleSelection.ReplaceSelectionWithText() to incorrectly set textArea.Caret.Position to (Line = 1, Column = 1). Note that the epsilon value of 1e-12 is an estimation based on the accuracy of the visualTop value, which has 13 decimals when its integer value is in the thousands (4 digits).
siegfriedpammer commented 2025-04-12 13:40:50 -05:00 (Migrated from github.com)

Can you please provide reproduction steps (using ICSharpCode.AvalonEdit.Sample.exe) for this problem? The GetVisualLineFromVisualTop method is used in a lot of contexts and I would prefer to not introduce any weird behavior due to magic constants.

I would expect the "lower than" relation to be not affected by epsilon errors.

Thanks!

Can you please provide reproduction steps (using ICSharpCode.AvalonEdit.Sample.exe) for this problem? The `GetVisualLineFromVisualTop` method is used in a lot of contexts and I would prefer to not introduce any weird behavior due to magic constants. I would expect the "lower than" relation to be not affected by epsilon errors. Thanks!
This pull request is broken due to missing fork information.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin 'trijnsburger/Fix-rounding-error-when-getting-VisualLine:trijnsburger/Fix-rounding-error-when-getting-VisualLine'
git switch 'trijnsburger/Fix-rounding-error-when-getting-VisualLine'

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch 'master'
git merge --no-ff 'trijnsburger/Fix-rounding-error-when-getting-VisualLine'
git switch 'trijnsburger/Fix-rounding-error-when-getting-VisualLine'
git rebase 'master'
git switch 'master'
git merge --ff-only 'trijnsburger/Fix-rounding-error-when-getting-VisualLine'
git switch 'trijnsburger/Fix-rounding-error-when-getting-VisualLine'
git rebase 'master'
git switch 'master'
git merge --no-ff 'trijnsburger/Fix-rounding-error-when-getting-VisualLine'
git switch 'master'
git merge --squash 'trijnsburger/Fix-rounding-error-when-getting-VisualLine'
git switch 'master'
git merge --ff-only 'trijnsburger/Fix-rounding-error-when-getting-VisualLine'
git switch 'master'
git merge 'trijnsburger/Fix-rounding-error-when-getting-VisualLine'
git push origin 'master'
Sign in to join this conversation.
No description provided.