Realex

Giblet

Anyone else suffering because of their outage :pac:

Boards.ie: Paddy

Yep. No credit card payments on boardsdeals this morning. Thankfully we accept PayPal payments too so we're not totally dead in the water.

Giblet

We turned off preauth for €1 shadow charges, but that only affects a few clients, rest are boned completely. They lost a SAN...

TeRmInAlCrAzY

Useful ...

http://www.realexpayments.com/service-status-update

Also

https://twitter.com/#!/realexpayments

Creamy Goodness

doesn't instill confidence if their backups fail...

Webmonkey

Yeah tha'ts pretty bad. I'm guessing Register365.ie's payments go via this also as I can't pay for a domain that is about to be suspended due to late payment

mccarte2

I guess this is why Aer Lingus are having payment issues today :mad:

JDxtra

Why on earth were they replacing a failed SAN component at the start of a business day?? Madness.

SANs are built with resilient hardware. Surely the work could have waited - even until tomorrow.

Creamy Goodness

mccarte2 wrote: »

I guess this is why Aer Lingus are having payment issues today :mad:

also meteor, pretty much the whole of the irish internet...

JDxtra

From their support page...

We have plans in place that will offer an alternative solution within 40 minutes.

It's called PayPal.

testicle

Creamy Goodness wrote: »

doesn't instill confidence if their backups fail...

Backup Domain Controller != Backups

Saying that, it appears to be a very poorly designed infrastructure.

JDxtra

Stupid mistakes (the SAN change) followed by failures where recovery options either don't exist or weren't tested properly.

Now the VMs can't see the production SAN. Oh dear guys.

They're fecked now. Payment systems NEEDS to be bulletproof. If I was a big customer, I'd be putting them under serious pressure for these errors.

Giblet

Trust me, they are being put under pressure Who's betting the engineer replaced the working controller, not the faulty one

jwilco

It's back

Raekwon

Is this the same Realex Payments in Monkstown that seem to be constantly hiring for the same positions yet seem impossible to even get an interview with?

Very interesting

Creamy Goodness

They have recently moved offices to somewhere on the docks i think

Raekwon

I thought that their sister company Carapay was being setup on Sir John Rogerson's Quay and Realex were staying in Monkstown?

JDxtra

Giblet wrote: »

Who's betting the engineer replaced the working controller, not the faulty one

Exactly!

Giblet

In fairness to Realex, besides the awful mistake, it was a tough one to recover from too.

MagicRon

Giblet wrote: »

In fairness to Realex, besides the awful mistake, it was a tough one to recover from too.

I think many companies have today realised the need to have a second payment provider available to them to fall clients back on to when their main provider has problems or goes offline.

Too many people were depending on just Realex, and five hours offline today has caused many thousands of losses for many of Ireland's biggest companies. And worse is that it wasn't just Ireland affected.

amen

fairness to Realex, besides the awful mistake, it was a tough one to recover from too

Not really, although this can happen to anyone (and we have all fingers crossed here) if you have the proper infrastructure in place and its tested regularly i.e. switch back and forth between your primary and secondary site on a regular basis using each as the primary/secondary for several periods during the year, not changing controllers on a Live SAN, constantly fault testing your app/infrastructure and looking for points failure you can minimise this occurring.

Giblet

I'm talking about recovery, not prevention, as in "**** IT'S HAPPENED"