Email down again!

rob316

4 hours today my business is without email, which I vitally need.

Whats happening, will it be restored by the end of the day.

mneylon

Our technical team are working on it with the vendor. 
Updates are being posted here: http://www.blacknightstatus.com/2014/12/shared-hosting-email-performance.html

I'm in the same boat, got onto Support and they told me I shouldn't be using shared hosting for business email, using their fault to try and sell. Read on status update at 1.00 -
Update 13:00: We are rebooting nodes to allow for some emergency maintenance that was required, we will update this status post once that is completed and services are back online.

Surely emergency maintenance should be done at off-peak times over the weekend, not early on Monday at busiest time!!!

Was onto support, get told no ETA to be honest as if it's nothing to get too bothered about, think Blacknight needs a credible Irish competitor to get them motivated again.

All day without email for business is not good enough!

mneylon

Deleted User wrote: »

I'm in the same boat, got onto Support and they told me I shouldn't be using shared hosting for business email, using their fault to try and sell. Read on status update at 1.00 -
Update 13:00: We are rebooting nodes to allow for some emergency maintenance that was required, we will update this status post once that is completed and services are back online.

Surely emergency maintenance should be done at off-peak times over the weekend, not early on Monday at busiest time!!!

Was onto support, get told no ETA to be honest as if it's nothing to get too bothered about, think Blacknight needs a credible Irish competitor to get them motivated again.

All day without email for business is not good enough!

Our CTO is going to be posting more details explaining what exactly happened. 
"Maintenance" might not have been the best word to use, as that suggests something planned, which this wasn't. 
Scheduled maintenance happens outside Irish office hours, "emergency maintenance" is when we have to do something immediately eg. if a system fails

sysprogrammer

Blacknight didn't post updates though on your Blacknightstatus.com , I know because we were checking every few mins. You just left the customers in the dark.
 
I phoned support and they said to check blacknightstatus.com. So nice roundabout you have there.
 
I logged a support ticket and this has not even been responded to (lost in internal email ??)

I don't care so much as what happened , more the fact that nothing was said to paying customers.

I believe we all understand that stuff breaks, but not being told a single thing is not the best approach.

or maybe just change the name to BlackOut.ie.

/end rant
 

mneylon

sysprogrammer wrote: »

Blacknight didn't post updates though on your Blacknightstatus.com , I know because we were checking every few mins. You just left the customers in the dark.
 
I phoned support and they said to check blacknightstatus.com. So nice roundabout you have there.
 
I logged a support ticket and this has not even been responded to (lost in internal email ??)

I don't care so much as what happened , more the fact that nothing was said to paying customers.

I believe we all understand that stuff breaks, but not being told a single thing is not the best approach.

or maybe just change the name to BlackOut.ie.

/end rant
 

I'm sorry you feel this way, but our technical team did post updates as can be seen here:
http://www.blacknightstatus.com/2014/12/shared-hosting-email-performance.html

If you have the ticket ID I'd be happy to look into what happened to it, as normally we reply to ALL support tickets

Thanks

Michele

sysprogrammer

Michele
I call Ninja edit. the updates were not posted on the day we were checking.
 
If the BK automated email ticket system had returned an ID I would let you know. but it didn't which is why I thought your support tickets are being subject to the same issue as customers.(mail routed externally?)
 
The staff answering the phones were also unable to give any info. Or did not feel okay saying "we don't know yet"
 
We do know each other Michele and I'm not going to harp on about this, but since I have this direct line to the Big Cheese can you perhaps have a look into this particular issue please ?
 
Blacknight have an excellent uptime record , on virtual and shared hosting , but this little issue was particularly frustrating by the lack of info supplied to the staff answering the phones.
 
thanks for your time,
 
Sys Prog

mneylon

sysprogrammer wrote: »

Michele
I call Ninja edit. the updates were not posted on the day we were checking.
 
If the BK automated email ticket system had returned an ID I would let you know. but it didn't which is why I thought your support tickets are being subject to the same issue as customers.(mail routed externally?)
 
The staff answering the phones were also unable to give any info. Or did not feel okay saying "we don't know yet"
 
We do know each other Michele and I'm not going to harp on about this, but since I have this direct line to the Big Cheese can you perhaps have a look into this particular issue please ?
 
Blacknight have an excellent uptime record , on virtual and shared hosting , but this little issue was particularly frustrating by the lack of info supplied to the staff answering the phones.
 
thanks for your time,
 
Sys Prog

The updates were posted the day at issue at the times noted in the post. Please do not call me or my staff liars.

As for the support ticket system - it's an external system and its email was not impacted by any issues involving email. I can't check your ticket since you haven't give me any information to check it against eg. a real name an email address or something else

With respect to our staff's responses on the phones - I'll have to discuss with their manager

Regards

Michele

sysprogrammer

Blacknight wrote: »

sysprogrammer wrote: »

Michele
I call Ninja edit. the updates were not posted on the day we were checking.
 
If the BK automated email ticket system had returned an ID I would let you know. but it didn't which is why I thought your support tickets are being subject to the same issue as customers.(mail routed externally?)
 
The staff answering the phones were also unable to give any info. Or did not feel okay saying "we don't know yet"
 
We do know each other Michele and I'm not going to harp on about this, but since I have this direct line to the Big Cheese can you perhaps have a look into this particular issue please ?
 
Blacknight have an excellent uptime record , on virtual and shared hosting , but this little issue was particularly frustrating by the lack of info supplied to the staff answering the phones.
 
thanks for your time,
 
Sys Prog

The updates were posted the day at issue at the times noted in the post. Please do not call me or my staff liars.

As for the support ticket system - it's an external system and its email was not impacted by any issues involving email. I can't check your ticket since you haven't give me any information to check it against eg. a real name an email address or something else

With respect to our staff's responses on the phones - I'll have to discuss with their manager

Regards

Michele

Don't infer I am a liar either, I know I was checking blacknightstatus.com throughout the day.

A support ticket was submitted, of this I am sure.  Send me a pm and I'll send you my details. I have no ulterior motive other than lost email.

If you record calls for quality and training I'm sure you'll hear what I report. please do get the managers attention and do something about it.

at no stage have you or your staff have held your hands up and said sorry we didn't provide the service you paid for , rather I am getting this run around , just like the day on the phone.

I'm sure this outage falls within the SLA but it still leaves a bad taste.

Sysprog

Hi Sysprog,

I am the support manager at Blacknight, I am also the Engineer who created the entry on the Blacknight status page http://www.blacknightstatus.com/2014/12/shared-hosting-email-performance.html I can assure you there was no ninja edits and the updates were posted at the time, I myself posted some of them.

The CTO has posted an interim RF today at http://www.blacknightstatus.com/2014/12/shared-hosting-email-interim-reason-for-outage-notification.html

The support team were updated with any information that became available from the Engineers and Vendors. As Blacknight and Vendor Engineers worked to find the cause the of issue and resolve it there was no ETA available, as you can see from the RFO there was a long period where we waited on the volume to import with no information as to what was causing the issue and thus no ETA was available.

The support team had a large volume of calls and tickets during the outage and they worked to get through that volume of calls and tickets as quickly as possible to ensure customers were not left on hold or awaiting a ticket response for a long period of time, they provided all the information that was available to them which included stating we had no ETA.

If you PM me details on the ticket you raised I will investigate and come back to you, our internal email and ticketing system were not affected by the outage, I should also point out that our Business email platform was also not affected.

I can only apologise for the outage and assure you that we have invested heavily in our shared mail platform this year and have plans to do even more upgrades next year.

sysprogrammer

Hi Alan,
 
Thanks for your reply. I have read the report on the outage, it does read like unfortunate circumstance that all thread handles can expend without any monitoring software causing alerts...
 
The bkstatus.com webpage is overwritten with Paul lw Kellys note so , yeh, I'm just going to go back to work now.
 
Great news that the business email platform wasn't affected, neither was letshost dot ie,  but that wasn't my comment about business email earlier. I was waiting on a password reset , which was just typical of my luck , but not the end of the world.
 
I personally really appreciate you apologising for something that was outside of your control. nice one. I don't even use twitter so I missed the all clear.
 
right , I'm off , /rm- r
 
 

mneylon

sysprogrammer wrote: »

The bkstatus.com webpage is overwritten with Paul lw Kellys note so , yeh, I'm just going to go back to work now.
 

 

No it isn't.
All posts are still there:
http://www.blacknightstatus.com/blog/  - all posts

Paul's post from yesterday: 
http://www.blacknightstatus.com/2014/12/shared-hosting-email-interim-reason-for-outage-notification.html

The original post from December 1st: 
http://www.blacknightstatus.com/2014/12/shared-hosting-email-performance.html

sysprogrammer

I can point out that Alan's posts and Pauls come from different URL's. so you can see the confusion.

but the tone is degenerating here, and as a paying customer I'm not liking the stance taken by Blacknight.

One less paying customer come renewal.

Good day

Hi Sysprog,

The URL of each post is unique but all posts are always available via http://www.blacknightstatus.com/blog/

I'm sorry you aren't happy with the responses you have received but we have been honest and up front at all times, both during and after the outage.

Alan

Here we go again mail down again on a busy morning where I really need it!

rob316

Yep down again. Can ye tell me what package I need to move over to, to avoid this? My business depends on email.

Apologies for the inconvenience we are working on getting services resumed as quickly as possible, updates are being posted to http://www.blacknightstatus.com/2015/01/shared-hosting-email-issue.html

You can see details on our Business Email at https://www.blacknight.com/hosted-exchange.html

What about cloud hosting like - Cloud VM Fixed Starter - Does this cover email? If using the standard CentOS 64 operating system, is it difficult to setup email & websites on this?

Axwell

Why should we be forced to look at other offerings when we are paying for a service as is that should provide working email.

Deleted User

Deleted User wrote: »

What about cloud hosting like - Cloud VM Fixed Starter - Does this cover email? If using the standard CentOS 64 operating system, is it difficult to setup email & websites on this?

A VM with a control panel would be pretty easy to setup but you would need at least Starter+

Axwell

Axwell wrote: »

Why should we be forced to look at other offerings when we are paying for a service as is that should provide working email.

Email is provided as a free add-on at present with hosting, the Business email platform is a premium product with an SLA, however we have invested heavily in our shared email platform and we will be working with the storage vendor to ensure we find the root cause of this issue and address it.

Services came back online at 13:35, we continue to work with the vendor to investigate the root cause.

ct5amr2ig1nfhp

Blacknight wrote: »

The updates were posted the day at issue at the times noted in the post. Please do not call me or my staff liars.

As for the support ticket system - it's an external system and its email was not impacted by any issues involving email. I can't check your ticket since you haven't give me any information to check it against eg. a real name an email address or something else

With respect to our staff's responses on the phones - I'll have to discuss with their manager

Regards

Michele

What an appalling response to a genuine issue/query from a customer. In no way did the poster call you or your staff liars.

After reading the entire thread I want to add in our own experience with another provider early last year.

We had a similar issue - downtime, status page not updated. The hosting provider claimed their status page was updated throughout an outage. We could not see any updates on their status page and took screenshots as proof that the page was not being updated.
Later it transpired that whatever system they used internally to update the page, wasn't updating the status page externally.

@Michele, perhaps you should review your procedures and process for updating your status page, rather than accuse a customer of calling you and/or your team liars. 

Hi ProjectMoose,

In the cases discussed here the status posts were updated by me and were accessible, many customers confirmed that to my team.

Alan

ct5amr2ig1nfhp

Alan, the status page query is not why I responded.

I felt that the overall tone of the post I referenced was not in line with how I expect a representative of BlackKnight to respond to a query. That is all.

Thanks ProjectMoose your feedback is appreciated