Why not choose some RPC protocol, document the interface and let devs write in whatever they want?
I actually considered this briefly. I even wrote a quick proof of concept: a package that spawns a Python 3.7 process and gives it access to
sublime_api over pipes. This has some potential, but also some drawbacks.
I’m running the Mac OS. On this machine, Sublime is communicating with
plugin_host via POSIX pipes and semaphores. I would hazard a guess that it is converting internal data to Python types in Sublime and passing them to the runtime via a standard Python serialization format. I also presume that the communication channel is platform-specific, and that the Windows implementation is different.
This works fine for a single runtime tightly controlled by Sublime HQ, but not necessarily for user-supplied runtimes. For one thing, if non-Python runtimes are supported, then the API must use language-agnostic serialization, such as JSON. This incurs overhead on both sides of the channel. For another thing, a pluggable implementation would most likely use a different channel to support multiple plugin runtimes and so that a single runtime implementation could work cross-platform. The obvious choice is a TCP socket. I haven’t run the numbers, but I would expect this to also incur significant overhead versus the current pipe implementation.
Sublime is useless without plugins; plenty of core functionality is implemented in the Default package. So Sublime HQ would still need to provide and maintain a built-in Python runtime. However, to support user-provided runtimes they would also need to maintain and document a “runtime API”. Currently, the
sublime_api package is an implementation detail that Sublime HQ can change at any time. If other runtimes were to rely on it, then Sublime HQ would need to formally stabilize the API, write complete documentation, and maintain the API forever as new features are introduced.
And while user-supplied runtimes would be very convenient for developers, they could pose a substantial risk to the package ecosystem. Right now, every Sublime plugin uses the same runtime, which comes preinstalled with Sublime and is maintained by Sublime HQ. If user-provided runtimes were widely adopted, then a large number of packages would depend not only on the package author but on the runtime author. Because the 3.3 runtime is bundled with Sublime, Sublime HQ is obliged to actively maintain it (even e.g. backporting security fixes from future Python versions). Maybe the custom runtimes would be implemented, maintained and documented to the same standard as the core runtime, or maybe not. A poorly-maintained runtime could break or orphan many, many packages.
Don’t get me wrong, it would be really neat to have multiple pluggable runtimes. But I’m not sure that it would be useful enough to justify the complexity and performance costs or the risk to the ecosystem.
On the other hand, thinking through the problem of multiple runtimes does give me hope that it might not be as difficult as I’d feared to get first-party 3.3 and 3.7 runtimes working side-by-side. Of course, the hard part there is designing the developer-facing boundary that determines what runtime to use for a given plugin.