[ML][MS][GLADOS] Disk Failure
One of our Virtual Machines, GLADOS-2, had a disk failure.
Recovery was started and is complete. Data loss should have been mitigated, but if you notice a major data loss, by all means report back and staff will fix it.
Affected Game Servers: Moubootaur Legends, Mana Spheres
Affected Services: GitLab CI
Affected Miscellaneous: Liviobot, ManaVerse Downloads*, Game Update Download**
*: No data loss expected: The affected VM is where ManaVerse binaries are. Report if corruption is identified.
**: No data loss expected: The affected VM is where the game updates for every server are (the ones when you start playing). Report if corruption is identified.
Identification: 05:00 UTC
Resolution: 06:00 UTC
Cause: Docker
Type: Disk Failure
Solution: VPS Panel recovery
Rolled Backup Date: No backup rolled back. Snapshot from 04:30 preserved.
Additional notes: Char-Server and Map-Server crash might have caused strange data loss depending on MySQL unavailability during the incident. Contact staff if in need of special/additional recovery.
Further actions taken: Unused Docker volumes purged, EXP and DROP rate ×2 during the weekend (affected servers only).