Editor for large text files (>1 GIG)

bnoam · **Editor for large text files (>1 GIG) -** 01-20-2010, 01:52 AM

Hello,
It is known that eclipse default text editor lack of support to read huge files.
It reads the entire file into a string buffer and then creates a document out of it.

My question is if there is a text editor in Eclipse that support paging of file? meaning -
a user opens a huge file, but only the part that he currently sees is loaded into the memory, and he doesn't feel that only part of the file is loaded.

If not, I would like some tips for where to get started in writing such editor my self. I have good knowledge of PDE and wrote a few plugins, but this problem seems to be more complicated.
Moreover, the files I want to be able to open are not in the local workspace, but outside of it, so I am using IFileStore as a resource, FileStoreEditorInput as an IEditorInput and TextFileDocumentProvider as provider

Thanks a lot !

ggleblanc · 01-25-2010, 09:14 AM

Quote:

Originally Posted by bnoam

Hello,
It is known that [the] eclipse default text editor lack of support to read huge files. It reads the entire file into a string buffer and then creates a document out of it.

It's actually worse. I did some tests on Eclipse 3.2, and depending on what you were doing in the text editor, there were as many as 5 copies of the edited file in memory.

I don't think that the memory usage has improved that much through Eclipse 3.5.

Quote:

Originally Posted by bnoam

My question is if there is a text editor in Eclipse that support paging of file? meaning - a user opens a huge file, but only the part that he currently sees is loaded into the memory, and he doesn't feel that only part of the file is loaded.

If not, I would like some tips for where to get started in writing such editor my self. I have good knowledge of PDE and wrote a few plugins, but this problem seems to be more complicated. Moreover, the files I want to be able to open are not in the local workspace, but outside of it, so I am using IFileStore as a resource, FileStoreEditorInput as an IEditorInput and TextFileDocumentProvider as provider

Thanks a lot !

I think SlickEdit has an Eclipse editor that can better handle large files.

As far as writing your own, good luck.

The classes that are derived from IDocument are so interrelated, that you're going to have to build your editor from the ground up.

I've been working on and off creating an ISPF-style Eclipse editor, and it's been a massive undertaking.

bnoam · 01-25-2010, 11:40 AM

I know its a hard task and that I'll probably won't be able to make the dynamic page loading totally transparent to the user.
But I am working on something that will at least enable to open huge files, without crashing the platform, and enabling to scroll using button etc.

I would like to hear an official Eclipse developer about that issue, whether its going to be solved in the future?

Thanks.

Noam

jwenting · 01-26-2010, 07:56 AM

I doubt it will ever be "solved" as no solution is needed to serve the product's intended purpose. Eclipse is a code editor at heart, code files that large are extremely uncommon and utterly useless anyway.
Edit them if you have to using VI or something similar, cut them up into more manageable units, and use those instead.

johnttrick · 01-31-2010, 06:58 AM

I am a fan of Slickedit - I have been asked to work on our SlickEdit version of the editor. We have written many supporting programs written to handle various tasks. i find it apt for your needs though i am still unsure what you are at.

bnoam · 01-31-2010, 08:28 AM

Hi,

Actually I need it to be in the Eclipse platform, in order to work with other plugins we have them.
I managed to find a good way to it in the meanwhile, but it really need a lot of hacks. for example, suppose I do load part of the file into the editor, now CTRL+F will search for words only in this segment. I need to extend take control over it and change it to my search, that searches in the file.

Noam

bnoam · 01-31-2010, 08:31 AM

Hi,

Actually I need it to be in the Eclipse platform, in order to work with other plugins we have them.
I managed to find a good way to it in the meanwhile, but it really need a lot of hacks. for example, suppose I do load part of the file into the editor, now CTRL+F will search for words only in this segment. I need to extend take control over it and change it to my search, that searches in the file.

Noam