Sublime Forum

Graphics extensions possible?

#1

Hey! is it currently possible to write a plugin that can render graphics to a tab/dockable window? I’m thinking it shouldn’t be too hard to pass us an OpenGL context to use in a plugin, especially since OpenGL is supported on all the platforms Sublime Text 3 supports.

I’ve seen people do something similar with Visual Studio Code, but iirc it was done with WebGL. I’m primarily interested in just a basic OpenGL context/surface for plugging some existing software in to the editor for visualization and other purposes…

Thanks!

1 Like
#2

Python is a powerful language - you can use Python to open GUI Windows ( Color Picker does it and some others too ) so it should be possible to do what you’re thinking of…

I’m planning on experimenting with it too so if you end up making one before me - please respond :slight_smile:

I am also going to test a few things in Sublime text to see if I can overwrite null-ares within ST3 ( ie if you create a new column by resizing the main column smaller than the full window size, you create a gap of all-black … normally null spaces you end up seeing trails from each frame but in ST3 you don’t … I want to see if I can render an advanced panel in that area using hooks then I don’t need to worry about docking, etc… although I will want that feature to exist…

As for actual docking - it won’t be within sublime text ( unless you simply have it set to always on top, and have it render in one of those null zones - you can save the data to a runtime config file and have your app read it and monitor for changes and have it render in that location simply by grabbing the sublime text window reference, grabbing the size, location on the digital plane / desktop, and getting the size of the borders, etc ( os specific so you’ll need to have each os module load specifically with each os to calculate the border sizes and everything else ) to figure out exactly where to place it so it is seamless )…

0 Likes
#3

No, this is not currently possible; there is no API hook that would give you a context. In fact I’m pretty sure that in fact Sublime only uses OpenGL to render on MacOS, where it’s an option you can turn off, and is the root reason for performance problems in recent builds on that OS. The core renders using Skia and only uses OpenGL on MacOS to render the composited tiles to the screen.

The best you’re probably going to be able to do is open an external application and talk to it (like the example mentioned above to open a color picker); that’s less rendering custom controls and more just asking a program to do something for you and then seeing what the result is,

Presumably this is possible in VSC because it’s an Electron and provides you more control over what’s displayed in a file view than Sublime does. In comparison, Sublime is written in C++ using it’s own custom GUI engine.

0 Likes
#4

Yeah I figured it wouldn’t be that easy. Anyways, I just found out that ST3 has an image viewer built in, and it automatically reloads the file when it detects changes. So I rigged up some basic code in my existing OpenGL app to write the contents of the framebuffer to a bitmap file every frame, and then loaded it up in ST3 to see if I could get something even remotely close to realtime rendering.

…unsurprisingly that didn’t work :unamused: The fastest I was able to get was like 2-3 frames per second in Sublime. It seems like the bottleneck is in Sublime, because my program was writing it faster than Sublime was updating. It might also have to do with whatever method they’re using to detect changes in the source file and any overhead associated with that. I even tried writing to a RAM disk using ImDisk on Windows, but that didn’t make any difference.

So realtime rendering is certainly off the table, but if there’s a way to calculate the mouse location and state within the image viewer in Sublime, this method could very easily be used to embed a UI with custom controls, sliders, buttons, etc. It wouldn’t be very responsive, but it may still be useful in some cases.

But unfortunately it’s not what I was looking for. Hopefully the Sublime developers throw us some more ways to control the display. I know it uses Skia, but that doesn’t mean it isn’t possible to have an OpenGL viewport somewhere in there. They could also at least expose some basic drawing routines on top of Skia to the Python API. I’m new to Sublime though, so I don’t know how active the development is and whether I should expect that to ever actually happen.

0 Likes
#5

I believe the right way forward in this direction is to utilize minihtml in more impactful ways.

0 Likes
#6

In regards to image reloading, we rely on filesystem notifications to know when a file is changed, and that triggers a rescan of the file/folder in question. I would bet the delay is a combination of various implementation details of all of that. However, the image viewing feature wasn’t built to try and allow for custom rendering, so I’m not surprised it doesn’t work for that.

As @OdatNurd has pointed out, we don’t use OpenGL on Windows or Linux. This is based on experience than Jon has had working on games and with Sublime Text 1 being based on OpenGL. There were just far too many issues related to OpenGL drivers and hardware. Even to this day browsers have extensive blacklists for GPU acceleration on Linux.

In general I doubt we’ll be exposing a raw drawing API any time soon for Sublime Text. The more that is exposed, the more general purpose the entire code base has to be, and then that makes it harder to keep things fast since they can’t be as purpose built. And then there is the maintenance burden. Every time something is exposed, it locks us into supporting that at least for the remainder of the major version. @rwols make a pretty good point that minihtml is the closest thing we have to arbitrary text/graphics rendering, however that is currently limited to popups and phantoms, and probably wouldn’t work well for rapidly changing content.

I think we are far more focused on trying to be a fast text editor that looks good, can be taught to work with new languages pretty easily, and has a bunch of heuristic-based helpers that save busy-work when coding. There is no way we’ll be competing with language-specific IDEs, or general purpose browsers with text editor controls built on top of them. But since we have lots of purpose-built components, we can generally make things perform well, optimize things where necessary and provide a pretty decent cross-platform experience.

3 Likes
#7

It sounds like you would be better off just rendering stuff in your own OS window using whatever technique you like possibly in a platform-specific way. Users can just arrange their windows side-by-side or you could set your window to be on-top.

0 Likes