Datum a čas: 2023-12-19 23:24 CET
Očekavaná délka: 60 minut
Oznámení se týká serverů: node23.prg
Typ výpadku: vps_reset
Důvod: System reset
Výpadek řeší: Pavel Šnajdr, Jakub Skokan
Pravdepodobne interakce mezi bugy ve firmwarech M.2 SSD WD SN850 a AMD/Dellu. Ten M.2 disk neni vyuzity a muzeme ho v blizke budoucnosti vymontovat.
ENGLISH:
Date and time: 2023-12-19 23:24 CET
Expected duration: 60 minutes
Affected systems: node23.prg
Outage type: vps_reset
Reason: System reset
Handled by: Pavel Šnajdr, Jakub Skokan
Probably interaction between firmware bugs of M.2 SSD WD SN850 and AMD/Dell. The M.2 drive is unused and can be removed from the system in near future.
-----BEGIN BASE64 ENCODED PARSEABLE JSON-----
eyJpZCI6MTA2MCwicGxhbm5lZCI6ZmFsc2UsImJlZ2luc19hdCI6IjIwMjMt
MTItMTlUMjM6MjQ6MDArMDE6MDAiLCJkdXJhdGlvbiI6NjAsInR5cGUiOiJ2
cHNfcmVzZXQiLCJlbnRpdGllcyI6W3sibmFtZSI6Ik5vZGUiLCJpZCI6MTI0
LCJsYWJlbCI6Im5vZGUyMy5wcmcifV0sImhhbmRsZXJzIjpbIlBhdmVsIMWg
bmFqZHIiLCJKYWt1YiBTa29rYW4iXSwidHJhbnNsYXRpb25zIjp7ImVuIjp7
InN1bW1hcnkiOiJTeXN0ZW0gcmVzZXQiLCJkZXNjcmlwdGlvbiI6IlByb2Jh
Ymx5IGludGVyYWN0aW9uIGJldHdlZW4gZmlybXdhcmUgYnVncyBvZiBNLjIg
U1NEIFdEIFNOODUwIGFuZCBBTUQvRGVsbC4gVGhlIE0uMiBkcml2ZSBpcyB1
bnVzZWQgYW5kIGNhbiBiZSByZW1vdmVkIGZyb20gdGhlIHN5c3RlbSBpbiBu
ZWFyIGZ1dHVyZS4ifSwiY3MiOnsic3VtbWFyeSI6IlN5c3RlbSByZXNldCIs
ImRlc2NyaXB0aW9uIjoiUHJhdmRlcG9kb2JuZSBpbnRlcmFrY2UgbWV6aSBi
dWd5IHZlIGZpcm13YXJlY2ggTS4yIFNTRCBXRCBTTjg1MCBhIEFNRC9EZWxs
dS4gVGVuIE0uMiBkaXNrIG5lbmkgdnl1eml0eSBhIG11emVtZSBobyB2IGJs
aXprZSBidWRvdWNub3N0aSB2eW1vbnRvdmF0LiJ9fX0=
-----END BASE64 ENCODED PARSEABLE JSON-----
Datum a čas: 2023-12-17 14:00 CET
Očekavaná délka: 60 minut
Oznámení se týká serverů: Brno
Typ výpadku: network
Důvod: Vypadky site kvuli DDoS
Výpadek řeší: Pavel Šnajdr, Jakub Skokan
Jeden ze clenu byl zasazen prichozim DDoSem, ktery vygradoval postupne pres 40 Gbit. Snazili jsme se ho zablokovat filtrovanim na strane naseho ISP (MasterDC), ale museli jsme odstavit celou cilovou IP adresu, abychom se vyhnuli packet lossu, jak se ten DDoS zvetsoval pridavanim novych zdrojovych adres.
ENGLISH:
Date and time: 2023-12-17 14:00 CET
Expected duration: 60 minutes
Affected systems: Brno
Outage type: network
Reason: Network issues due to DDoS
Handled by: Pavel Šnajdr, Jakub Skokan
One of our members experienced incoming DDoS, which escalated to over 40 Gbps. We've tried to block it through filtering at our ISP level (MasterDC), but had to block the whole IP to mitigate ongoing packet loss as the DDoS kept increasing by adding new sources.
-----BEGIN BASE64 ENCODED PARSEABLE JSON-----
eyJpZCI6MTA1OCwicGxhbm5lZCI6ZmFsc2UsImJlZ2luc19hdCI6IjIwMjMt
MTItMTdUMTQ6MDA6MDArMDE6MDAiLCJkdXJhdGlvbiI6NjAsInR5cGUiOiJu
ZXR3b3JrIiwiZW50aXRpZXMiOlt7Im5hbWUiOiJMb2NhdGlvbiIsImlkIjo0
LCJsYWJlbCI6IkJybm8ifV0sImhhbmRsZXJzIjpbIlBhdmVsIMWgbmFqZHIi
LCJKYWt1YiBTa29rYW4iXSwidHJhbnNsYXRpb25zIjp7ImVuIjp7InN1bW1h
cnkiOiJOZXR3b3JrIGlzc3VlcyBkdWUgdG8gRERvUyIsImRlc2NyaXB0aW9u
IjoiT25lIG9mIG91ciBtZW1iZXJzIGV4cGVyaWVuY2VkIGluY29taW5nIERE
b1MsIHdoaWNoIGVzY2FsYXRlZCB0byBvdmVyIDQwIEdicHMuIFdlJ3ZlIHRy
aWVkIHRvIGJsb2NrIGl0IHRocm91Z2ggZmlsdGVyaW5nIGF0IG91ciBJU1Ag
bGV2ZWwgKE1hc3RlckRDKSwgYnV0IGhhZCB0byBibG9jayB0aGUgd2hvbGUg
SVAgdG8gbWl0aWdhdGUgb25nb2luZyBwYWNrZXQgbG9zcyBhcyB0aGUgRERv
UyBrZXB0IGluY3JlYXNpbmcgYnkgYWRkaW5nIG5ldyBzb3VyY2VzLiJ9LCJj
cyI6eyJzdW1tYXJ5IjoiVnlwYWRreSBzaXRlIGt2dWxpIEREb1MiLCJkZXNj
cmlwdGlvbiI6IkplZGVuIHplIGNsZW51IGJ5bCB6YXNhemVuIHByaWNob3pp
bSBERG9TZW0sIGt0ZXJ5IHZ5Z3JhZG92YWwgcG9zdHVwbmUgcHJlcyA0MCBH
Yml0LiBTbmF6aWxpIGpzbWUgc2UgaG8gemFibG9rb3ZhdCBmaWx0cm92YW5p
bSBuYSBzdHJhbmUgbmFzZWhvIElTUCAoTWFzdGVyREMpLCBhbGUgbXVzZWxp
IGpzbWUgb2RzdGF2aXQgY2Vsb3UgY2lsb3ZvdSBJUCBhZHJlc3UsIGFieWNo
b20gc2UgdnlobnVsaSBwYWNrZXQgbG9zc3UsIGphayBzZSB0ZW4gRERvUyB6
dmV0c292YWwgcHJpZGF2YW5pbSBub3Z5Y2ggemRyb2pvdnljaCBhZHJlcy4i
fX19
-----END BASE64 ENCODED PARSEABLE JSON-----
Datum a čas: 2023-12-03 01:15 CET
Očekavaná délka: 40 minut
Oznámení se týká serverů: node22.prg
Typ odstávky: vps_restart
Důvod: Upgrade jadra
Odstávku řeší: Pavel Šnajdr, Jakub Skokan
=> Linux 6.6
ENGLISH:
Date and time: 2023-12-03 01:15 CET
Expected duration: 40 minutes
Affected systems: node22.prg
Maintenance type: vps_restart
Reason: Kernel upgrade
Handled by: Pavel Šnajdr, Jakub Skokan
=> Linux 6.6
-----BEGIN BASE64 ENCODED PARSEABLE JSON-----
eyJpZCI6MTA1NCwicGxhbm5lZCI6dHJ1ZSwiYmVnaW5zX2F0IjoiMjAyMy0x
Mi0wM1QwMToxNTowMCswMTowMCIsImR1cmF0aW9uIjo0MCwidHlwZSI6InZw
c19yZXN0YXJ0IiwiZW50aXRpZXMiOlt7Im5hbWUiOiJOb2RlIiwiaWQiOjEy
MywibGFiZWwiOiJub2RlMjIucHJnIn1dLCJoYW5kbGVycyI6WyJQYXZlbCDF
oG5hamRyIiwiSmFrdWIgU2tva2FuIl0sInRyYW5zbGF0aW9ucyI6eyJlbiI6
eyJzdW1tYXJ5IjoiS2VybmVsIHVwZ3JhZGUiLCJkZXNjcmlwdGlvbiI6Ij1c
dTAwM2UgTGludXggNi42In0sImNzIjp7InN1bW1hcnkiOiJVcGdyYWRlIGph
ZHJhIiwiZGVzY3JpcHRpb24iOiI9XHUwMDNlIExpbnV4IDYuNiJ9fX0=
-----END BASE64 ENCODED PARSEABLE JSON-----
