Vizhub not saving code

Bugs
#1

I’ve been using Vizhub this past week.

Today the website is not saving any of my code. Sometimes I get a “trying to reconnect” message, but not always and even if I get one, my connection still works on other websites.

When I checked the javascript console I get the following error code:

Uncaught TypeError: f is not a function
    H index.js:126
    e CodeMirrorReactBinding.js:223
    ve codemirror.js:568
    Oi codemirror.js:3912
    Si codemirror.js:3813
    Ci codemirror.js:3796
    Ln codemirror.js:2050
    Ci codemirror.js:3793
    Ei codemirror.js:3922
    poll codemirror.js:9535
    init codemirror.js:9320

I tried using chrome and firefox

There is also another error that firefox cant establish a connection to wss://vizhub.com/ (Bad Gateway 502)
As mentioned above, other websites work fine

Thanks for reading

#2

I am having the same issue as well, so its not just an internet provider thing.

#3

I have received this report from several people and am investigating the issue. It seems related to the WebSocket connection, which is used to save changes as they are made. Will report here when a resolution is reached. I can reproduce the issue.

I notice that this bug impacts all changes, not just to the files, as all the changes go over the same WebSocket connection, which is where the issue seems to lie. This includes changing any of the settings of the viz, such as public/private or viz height.

#4

It’s back up and running now, saving changes. Sorry about the outage!

The resolution was to restore a backup of the server. The root cause remains unknown, but I hope to track it down and understand in detail what happened and why. It’s a mystery, as nothing on the server had changed recently. This error just started happening randomly. Perhaps something to do with running out of disk space or memory? No idea.

#5

Thanks so much, Curran. Was a bit frustrating. Appreciate your help!

#6

My pleasure! Yeah I totally freaked out myself when I noticed it breaking in this way. Hoping it never happens again and the product can remain stable!

Maybe this is too much detail, but after some investigation, I think I got to the bottom of what happened. I believe the AWS instance that was housing the Redis server went down (became unreachable) in a freak AWS outage. Redis handles horizontal scalability on the server side for the real-time features of VizHub. So I think what happened is that the Redis outage ended up surfacing itself as a WebSocket error, because I believe interfacing with Redis is part of the WebSocket setup each time a client loads a page.

So, lots to learn from this. I would like to first of all set it up so that a Redis reachability error presents itself clearly as such. Secondly I would like to make the whole VizHub server infrastructure more resilient by introducing clusters for each service (not having a given service running on just one virtual machine, which is how it’s set up now).

Anyway, thanks for reporting the issue, and I’m happy that you’re enjoying the platform!