Stav: -> closed Popis: Pár slov k výpadku Na našem blogu jsme uvedli více informací ke čtvrtečnímu výpadku napájení v Praze. Blog obsahuje také vyjádření, které jsme obdrželi od MasterDC: https://blog.vpsfree.cz/post-mortem-par-slov-k-vypadku/ Nahlásil: Jakub Skokan ENGLISH: State: -> closed Summary: A few words regarding the outage We have published a blog post with more information about the power outage on Thursday. It also contains a communication we've received from MasterDC. https://blog.vpsfree.cz/post-mortem-par-slov-k-vypadku/ Since the blog post is in Czech, I include a short translation in English. The outage was connected to power loss in Prague city, not just the datacenter. While the DC has power backups, they failed and the first power line went down. At this time, we've continued to run on the second power line. As the DC was reconnecting selected devices to the second power line, a short circuit has occurred and the second power line went down as well. Everything we have in Prague was thus shut down, including our email support and tools that we otherwise use to communicate with our members. We've immediately set out to the DC to sort out the issues on the spot. The outage was further complicated by slowly booting switches. Most of our nodes are booted from PXE and since the nodes were up faster than the network, they failed to boot. This led to an additional delay, as we've had to reset them again. We've also lost one 10G switch during the outage. We're working on making the PXE server available faster to avoid the boot issues in the future. We appreciate MasterDC's response, as they were open about the situation. Reported by: Jakub Skokan -----BEGIN BASE64 ENCODED PARSEABLE JSON----- eyJpZCI6MjM3MywiY2hhbmdlcyI6eyJzdGF0ZSI6eyJmcm9tIjpudWxsLCJ0 byI6ImNsb3NlZCJ9fSwidHJhbnNsYXRpb25zIjp7ImVuIjp7InN1bW1hcnki OiJBIGZldyB3b3JkcyByZWdhcmRpbmcgdGhlIG91dGFnZSIsImRlc2NyaXB0 aW9uIjoiV2UgaGF2ZSBwdWJsaXNoZWQgYSBibG9nIHBvc3Qgd2l0aCBtb3Jl IGluZm9ybWF0aW9uIGFib3V0IHRoZSBwb3dlciBvdXRhZ2Vcclxub24gVGh1 cnNkYXkuIEl0IGFsc28gY29udGFpbnMgYSBjb21tdW5pY2F0aW9uIHdlJ3Zl IHJlY2VpdmVkIGZyb20gTWFzdGVyREMuXHJcblxyXG5odHRwczovL2Jsb2cu dnBzZnJlZS5jei9wb3N0LW1vcnRlbS1wYXItc2xvdi1rLXZ5cGFka3UvXHJc blxyXG5TaW5jZSB0aGUgYmxvZyBwb3N0IGlzIGluIEN6ZWNoLCBJIGluY2x1 ZGUgYSBzaG9ydCB0cmFuc2xhdGlvbiBpbiBFbmdsaXNoLlxyXG5cclxuVGhl IG91dGFnZSB3YXMgY29ubmVjdGVkIHRvIHBvd2VyIGxvc3MgaW4gUHJhZ3Vl IGNpdHksIG5vdCBqdXN0IHRoZSBkYXRhY2VudGVyLlxyXG5XaGlsZSB0aGUg REMgaGFzIHBvd2VyIGJhY2t1cHMsIHRoZXkgZmFpbGVkIGFuZCB0aGUgZmly c3QgcG93ZXIgbGluZSB3ZW50IGRvd24uXHJcbkF0IHRoaXMgdGltZSwgd2Un dmUgY29udGludWVkIHRvIHJ1biBvbiB0aGUgc2Vjb25kIHBvd2VyIGxpbmUu IEFzIHRoZSBEQyB3YXNcclxucmVjb25uZWN0aW5nIHNlbGVjdGVkIGRldmlj ZXMgdG8gdGhlIHNlY29uZCBwb3dlciBsaW5lLCBhIHNob3J0IGNpcmN1aXQg aGFzXHJcbm9jY3VycmVkIGFuZCB0aGUgc2Vjb25kIHBvd2VyIGxpbmUgd2Vu dCBkb3duIGFzIHdlbGwuXHJcblxyXG5FdmVyeXRoaW5nIHdlIGhhdmUgaW4g UHJhZ3VlIHdhcyB0aHVzIHNodXQgZG93biwgaW5jbHVkaW5nIG91ciBlbWFp bCBzdXBwb3J0XHJcbmFuZCB0b29scyB0aGF0IHdlIG90aGVyd2lzZSB1c2Ug dG8gY29tbXVuaWNhdGUgd2l0aCBvdXIgbWVtYmVycy4gV2UndmUgaW1tZWRp YXRlbHlcclxuc2V0IG91dCB0byB0aGUgREMgdG8gc29ydCBvdXQgdGhlIGlz c3VlcyBvbiB0aGUgc3BvdC4gVGhlIG91dGFnZSB3YXMgZnVydGhlclxyXG5j b21wbGljYXRlZCBieSBzbG93bHkgYm9vdGluZyBzd2l0Y2hlcy4gTW9zdCBv ZiBvdXIgbm9kZXMgYXJlIGJvb3RlZCBmcm9tIFBYRVxyXG5hbmQgc2luY2Ug dGhlIG5vZGVzIHdlcmUgdXAgZmFzdGVyIHRoYW4gdGhlIG5ldHdvcmssIHRo ZXkgZmFpbGVkIHRvIGJvb3QuIFRoaXNcclxubGVkIHRvIGFuIGFkZGl0aW9u YWwgZGVsYXksIGFzIHdlJ3ZlIGhhZCB0byByZXNldCB0aGVtIGFnYWluLiBX ZSd2ZSBhbHNvIGxvc3Rcclxub25lIDEwRyBzd2l0Y2ggZHVyaW5nIHRoZSBv dXRhZ2UuXHJcblxyXG5XZSdyZSB3b3JraW5nIG9uIG1ha2luZyB0aGUgUFhF IHNlcnZlciBhdmFpbGFibGUgZmFzdGVyIHRvIGF2b2lkIHRoZSBib290IGlz c3Vlc1xyXG5pbiB0aGUgZnV0dXJlLiBXZSBhcHByZWNpYXRlIE1hc3RlckRD J3MgcmVzcG9uc2UsIGFzIHRoZXkgd2VyZSBvcGVuIGFib3V0IHRoZVxyXG5z aXR1YXRpb24uXHJcbiJ9LCJjcyI6eyJzdW1tYXJ5IjoiUMOhciBzbG92IGsg dsO9cGFka3UiLCJkZXNjcmlwdGlvbiI6Ik5hIG5hxaFlbSBibG9ndSBqc21l IHV2ZWRsaSB2w61jZSBpbmZvcm1hY8OtIGtlIMSNdHZydGXEjW7DrW11IHbD vXBhZGt1IG5hcMOhamVuw61cclxudiBQcmF6ZS4gQmxvZyBvYnNhaHVqZSB0 YWvDqSB2eWrDoWTFmWVuw60sIGt0ZXLDqSBqc21lIG9iZHLFvmVsaSBvZCBN YXN0ZXJEQzpcclxuXHJcbmh0dHBzOi8vYmxvZy52cHNmcmVlLmN6L3Bvc3Qt bW9ydGVtLXBhci1zbG92LWstdnlwYWRrdS9cclxuIn19fQ== -----END BASE64 ENCODED PARSEABLE JSON-----