Sublime Forum

Extreme slow at opening large textfiles (SQL)

#1

Working with extreme large SQL import tables, sublime is impossible to use to open those files.

They only have to be more than 100Kb big then ST is very slow. I attached a 3 mb file - enjoy opening with ST.

SQL TEST FILE

1 Like

#2

It is a file containing only one line. Sublime Text doesn’t deal with extremely long lines very well, consider putting each record to insert on a new line

0 Likes

#3

So if there’s no ‘\n’ it’s just to much for ST :smiley: , well ok. Do you know why it needs a linebreak ?

0 Likes

#4

no idea, I’d quite like to know the technical reason for it too :slightly_smiling:

it’s not related to syntax highlighting, because ST doesn’t attempt to lex such long lines and it even happens in Plain Text syntax. Maybe to do with needing such a super-wide graphics buffer?

0 Likes

#5

Win 10 64bit, 8 GB RAM, SSD, I7 3rd Gen Dual Core + HT: Both Scite and Sublime text took about 3 Seconds to load your sql file.
But ST is somewhat slow when navigation through the line. That may be caused by the plugin BracketHighlighter, which has to catch up with a lot of round brackets opening and closing.

On the other hand: if anyone hands me such a script and tells me “That’s my import statement” i would start looking for the camera. Surely it is possible to dump a whole table or database into sql statements, but that’s ment for backup/restore or transport reasons - not for data manipulation.

0 Likes

#6

it’s the whole of wikipedia i’m importing. The SQL is generated from an dedicated app i wrote to filter a SQL dump from wiki, it’s not for manipulating - as you said it would be crazy :smiley: . The INSERT statement supports 1000 values at a time, doing that minimizes file size, since wikipedia database is enormous it’s important for me to limit the number of INSERT strings :D. I got 4.2 GB og such import files … so the 3 mb files is a infinitesimal small part of it :smiley:

It takes 2-4 min on my macbook, ssd, i7 2.5 Ghz, 8 GB ram to open in ST.

0 Likes