Sublime Forum

Package Control Outage

#23

Package Control in 2018 cost around $1,000 to run the server. Opting into Linode’s backup plan will raise the cost to probably around $1,230 for 2019. I will likely start going over the 8TB limit for bandwidth in the next month or so, so a realistic projection of costs for 2019 will be $1,300.

Currently there is no plan to transfer the project to Sublime HQ. If I get to the point where I can realistically no longer maintain the server, there would be discussion about community involvement vs Sublime HQ. Ultimately that would probably require some changes due to privacy concerns. Currently I am the only individual who has access to the server and the information.

All that to say, I currently basically maintain the server and do some light work on the code base. @FichteFoll and @ThomSmith do a ton of day-to-day work on the primary channel, reviewing packages and giving users feedback.

All parts of Package Control, the website and default channel are open source, but that doesn’t really affect the operational costs of keeping the default channel server running.

4 Likes
#24

Thanks so much for the reply. Like to make some time building a mirror site/resource. That effort would be a type of donation I could get behind now that I see there is a need.

0 Likes
#25

Is this situation resolved now? I’m still having trouble with package control, so am wondering if it is related to this or a separate issue? Thanks for all of your work getting it sorted :slight_smile:

0 Likes
#26

No, the site is not back yet. I’m most of way through recovering data from the database. Hopefully it will be functional later today.

I’m open to discussing people’s ideas, but this could end up being a rather involved change to the hosting configuration. Moving to multiple servers and replication is going to take time and testing, and we’d need double the server resources. We could opt for a simpler mirror of the channel data, but then that leaves the service still degraded (usage submission and website would be offline) and still requiring disaster recovery involving database recovery/restoration.

My plan for now is to throw a little more money at the situation rather than time. If I add block storage to the VM, I can return to doing full daily dumps and compressing those and sending them to my NAS. I can also pay to enable Linode’s backups that will give me a couple of snapshots of the server at preset intervals.

0 Likes
#27

what`s now?

0 Likes
#28

[quote=“LokisHat, post:13, topic:41274”]
I don’t know - I’m throwing ideas out there because I feel I’ve had more service out of my license than seems fair and I think you deserve more money for the app.[/quote]

See the Sustainable business model? thread, for users firmly in agreement with you. My post in that thread points out that if ST rises in cost with USD inflation (since we pay in USD) “it will take until 2023 before a Sublime Text license hits $100 - which I would still consider a bargain at twice the price.”

0 Likes
#29

The problem this outage demonstrates is the dependency on the package server for almost everyone using Sublime with extensions. It suggests that instructions for installing packages from the file system level and/or instructions for producing private repository listings is going to be essential for having a robust systems that does not have such a single point of failure. The reliability of the package server up to this point has produced a false sense of security. I would certainly be happy to support making the existing system more robust in any case.

0 Likes
#30

I’ve brought the uwsgi workers down to prevent data corruption as I work on restoring the repaired database to a new machine. You’ll likely see 502 errors now instead of 500 and 404.

1 Like
Package control dont work
#31

Quick tip for anyone who needs to install something now - Google has packagecontrol.io cached, so at least for the plugin I needed, I just clicked the little disclosure triangle in the google results, selected “cached” and got this:

https://webcache.googleusercontent.com/search?q=cache:FgGP3l_yPTgJ:https://packagecontrol.io/packages/Elixir+&cd=1&hl=en&ct=clnk&gl=us&client=safari

which, in this case at least, included installation instructions without using package control.

@wbond thanks for your great work on Package Control! Good luck with the repairs!!

0 Likes
#32

We should be back online now. DNS just switched over. If you are still seeing issues, make sure packagecontrol.io resolves to 74.207.232.232.

6 Likes
#33

Thank you for your job!

0 Likes
#34

Package Control is now accessible, but there are still problems with some packages. On ST start-up, I have messages in the console for a number of packages:

Package Control: The package “PackageName” is either not available on this platform or for this version of Sublime Text

(I use ST3, and these packages worked normally before). For example, I have this for packages PlainNotes, LaTeXTools, WolframLanguage. They are then automatically put into ignored packages. If I delete them from ignored and restart Sublime Text, the situation repeats. Besides, I can neither remove these packages nor reinstall them - they are simply not in the list when I go ctrl+shift+p and choose to install or remove a package.

0 Likes
#35

How can I check that without resorting to external network monitoring tools? :slight_smile:

UPD: that was a silly question :slight_smile:

It does resolve correctly. But I still experience problems with some of the packages.

0 Likes
#36

@wbond: Thank you for your (really professional!) information policy in this forum and of course for recovering PC so fast!

I am glad that a I decided to give at least a little donation a few weeks ago. Your work is very important for the Sublime eco system - I guess now many more people know that compared to a week ago :wink:

So, all in all: Thanks to you and all the volunteers doing a great job for the community!

0 Likes
#37

Ok, so after few hours, those problems with certain packages are gone. Although Package Control showed some error messages (see below), the packages are nonetheless installed.

Package Control: Error submitting usage information. HTTP error 500 downloading https://packagecontrol.io/submit?version=0.3.0&package_control_version=3.3.0&package=PlainNotes&sublime_platform=linux&operation=install&sublime_version=3176.
Package Control: Installed missing package PlainNotes

Thanks again!

2 Likes
#38

Thanks for the heads up - I had made a mental note of resetting a sequence in postresql, but then forgot. Should be fixed now.

0 Likes
#39

Thank you :sweat_smile:

0 Likes
#40

I am terribly sorry about that it seems that package control still does not work in China.
Maybe packagecontrol.io does not resolve to 74.207.232.232.

0 Likes
#41

I have ping package control domain name, the resolve IP is 74.207.232.232, but I still can’t access the website. I seems that the IP is blocked by Chinese Great Firewall,

0 Likes
#42

Was the old site accessible through the firewall?

0 Likes