Open files larger than available RAM? #382

New issue

Open

opened 2022-12-18 14:54:33 -06:00 by sopgenorth · 3 comments

sopgenorth commented

2022-12-18 14:54:33 -06:00

(Migrated from github.com)

Is it possible to use AvalonEdit with files that are larger than the available RAM? For example, a setup where only a buffered portion of a file is kept in memory, and updated as the user scrolls.

Are there any examples or pointers available on how to achieve this kind of setup?

Is it possible to use AvalonEdit with files that are larger than the available RAM? For example, a setup where only a buffered portion of a file is kept in memory, and updated as the user scrolls. Are there any examples or pointers available on how to achieve this kind of setup?

damian-666 commented

2022-12-18 15:47:39 -06:00

(Migrated from github.com)

Don' t know if this will help, but see https://www.codeguru.com/csharp/memory-mapped-files-c-sharp/ Do you mean using this sort of thing? Im curious how easy it would be to just integrate to the existing editors input , but with the disparate immutable partial clones and such for threading and parsing performance, i just saw discussed.. mabye not. also useful examples of usecases are in image processing for large resolution files say Archiectural plots which would require 24Gig at 600 dpi 1 bit. They take a tiling or an pipeline approach and im not sure its trival to just change the stream from a file to a memory map file. Other example is here https://docs.julialang.org/en/v1/stdlib/Mmap/

sopgenorth commented

2022-12-18 16:10:01 -06:00

(Migrated from github.com)

Yeah, I think some sort of memory mapped file could work. Might also be able to use FileStream.ReadBytes with the available offset to do a similar thing.

My thoughts were that the strategy used would want to buffer some number of bytes above and below what's currently viewable.

The "interface" for Rope (https://github.com/icsharpcode/AvalonEdit/blob/master/ICSharpCode.AvalonEdit/Utils/Rope.cs) used by the TextDocument seems like it's not too far off from being able to accomodate something like this, as long as the users of it don't just always jump to getting the full text at once...

Yeah, I think some sort of memory mapped file could work. Might also be able to use FileStream.ReadBytes with the available offset to do a similar thing. My thoughts were that the strategy used would want to buffer some number of bytes above and below what's currently viewable. The "interface" for Rope (https://github.com/icsharpcode/AvalonEdit/blob/master/ICSharpCode.AvalonEdit/Utils/Rope.cs) used by the TextDocument seems like it's not too far off from being able to accomodate something like this, as long as the users of it don't just always jump to getting the full text at once...

sopgenorth commented

2022-12-18 16:28:34 -06:00

(Migrated from github.com)

There's a plugin for Notepad++ that adds a similar kind of large file support too: https://github.com/superolmo/BigFiles

No milestone

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

gh-icsharpcode/AvalonEdit#382

No description provided.

Rows
Columns