PSA: Switching Singularity DB cluster out for a test


(CCP Goliath) #1

Hi Sisi fans,

CCP DeNormalized and I will be switching out the DB cluster that Singularity uses today. This means that your build jobs, structures, POSes etc. set up since the last time we mirrored will not appear for the next day or two. This is a temporary test, the cluster will be switched back to the old one once concluded. Due to the temporary nature of the change, we advise you to not set up anything complicated, or that will take longer than a day to bear fruit.

Following this test, assuming it is successful, we may be mirroring the server again in the next week or two, to allow for a proper transition over to the new cluster. More info provided as we have it.

(CCP DeNormalized) #2

My first post on the new forums!! \o/

So this is a new cluster, using SQL 2016 with AlwaysOn - I’ll be testing some random failovers between servers to see how it acts/etc… I’ll be in-game on Singularity to announce and monitor.

Thanks everyone!

CCP DeNormalized

(Raudur Froskur) #3

Care to share tech specs? Need to scratch my nerd-itch… :slight_smile:

(CCP DeNormalized) #4

it’s actually the old TQ DB hardware:

2x IBM x3690 X5 (32 cores, 2.27 GHz) w/ 512 GB ram each

(Raudur Froskur) #5


(CCP DeNormalized) #6

The storage is similar to the old TQDB stuff, but not quite as isolated. This particular SAN is used for all test servers and some other stuff so while it does have some SSD (in flash tier) , it’s primarily SAS based.

(CCP DeNormalized) #7

So this is delayed until tomorrow. Still prepping the new cluster and ensuring we have an easy way to migrate back to the old one once our tests are done.

Will update here and in-game on Singularity when we’re ready to make the switch

(StonerPhReaK) #8

I will take this time to wish you well in your tests. And to gift you some likes.

(CCP DeNormalized) #9

SISI is down atm while we do some startup tests

(CCP DeNormalized) #10

testing completed for now and we’re opening SISI again now (back on the old cluster)

(Raudur Froskur) #11

This did not interfere with the deployment of my Astrahus. Much appreciated.

(CCP DeNormalized) #12

nice!!! glad to hear :slight_smile: We’ll be doing more tests like this in the coming days so stay tuned I may yet ruin something :slight_smile:

What's Up With SiSi?
(Clandestiny) #13

I hate to assume…so, is this the reason SiSi has been down for 9½ hours?

:helmet_with_cross: or :face_with_symbols_over_mouth: ?

(CCP DeNormalized) #14

I was crashing things this weekend, but it wasn’t SISI!

Sore ribs and leg from a could-of-been-much-worse dirt bike crash :slight_smile:

(Clandestiny) #15

Glad you lived to tell the tale. Many don’t :cry: