Zend still not working. Seeking answers!

[ThePants999]

Hi,

Zend Optimizer is still down on our server (Windows: http:// my.purepotential.org). This has been going on for a week now! Our website will not run without Zend, so from our perspective this has equated to a week of downtime. A week of downtime! And this is the second time this has happened in three weeks. In the last month, our website has essentially had 70% uptime (and falling).

It took nearly three days to get a reply to our ticket (DSY-97710-25, and even that only said "we're looking at it" - and there has been no further response in the four days since.

Please throw me a bone here - EUKhost looked fantastic as a prospective customer, and I'm still holding on to a faint hope that we can stay with you guys. There's nothing that would please me more than to be writing a testimonial in a year's time. But this support response has been shocking, and if our website suffered this level of downtime once it had gone live it would cripple us. Help!

- Chris

[ThePants999]

Well, the website has come back up again, so I'm grateful for that. However, I had to find that out for myself, as I've still had no response to the ticket. So, I need some important questions answered.

1. How did this happen?

Zend is pretty crucial, and it was working previously. Whatever change was made that broke it, should have been tested on a development server before being deployed to a live server with customer websites on it. You guys do that, right?

2. Why did it take over a week to resolve?

It's a single software package. It can be reinstalled in minutes. This could have had an enormous impact on us.

3. Why has nobody responded to us?

At my previous host, everything was sorted within an hour. No way would they ever make us wait three days for a "we're working on it" email, or then a further five (a working week) with no more response.

[eUKhost.com]

I have answered all your questions as below :-

1. How did this happen?
Zend is pretty crucial, and it was working previously. Whatever change was made that broke it, should have been tested on a development server before being deployed to a live server with customer websites on it. You guys do that, right?
Ans :- Not sure why Zend was not running on your server. We dont do any experiments on live servers.

2. Why did it take over a week to resolve?
Ans :- One of our windows server crashed and last 3 - 4 days have been a nightmare. Our response time for support tickets had an impact due to this problem.

3. Why has nobody responded to us?
Ans :- One of our system admins tried to make it work on the day you had opened the ticket but he was not successful with it. Next day we were hit by crash of our other windows server.

[ThePants999]

Sorry guys, I'm far from satisfied yet.

1. Well, can you find out please? A crucial component was down for over a week - you can't just shrug and say "not sure". It was up, and then it wasn't, so something changed.

2. Even assuming that 100% of your support resource was tied up with that server crash, which in itself would be a bit of a pity, your server crash was on Sunday. The ticket was opened the previous Wednesday. That in itself is four solid days of downtime, which needs serious justification.

3. As I've said, it wasn't the next day. It took three days to get a response - again, that needs some justification. Additionally, it took a day and a half to get a response here - I know this is the forum and not the standard support system, but you've posted to plenty of other threads in the meantime.

I'm not trying to cause hassle, and I'm not just here to let off steam and scream and shout. I know this hasn't been a good week for you folks, and you have my sympathy. But it hasn't been a good week for us either. I'm a customer with some serious concerns, and I'm asking you to work with me in resolving them. What I'd really like to happen is to get some satisfactory answers and some honestly-meant promises that this was a one-off, and in future we'll have 99.95% uptime, and ticket responses (preferably with a resolution) within an hour or two, not a week or two. I'd like nothing more than to be able to move forward from here and form a long-term relationship with this company. But my trust needs to be earned; we can't remain a customer unless there's no doubt in my mind that this is going to work out.

A prompt response would be appreciated, even if it's just to say "we're looking into it".

[eUKhost.com]

I Agree with the delay in response. Even I had a look at your ticket on couple of occasions and got our windows system admins to look into it. 2 of our admins already tried compiling zend with php but they were not successful and the ticket remained in the queue for more time. I agree that a respose should have been sent if the problem was not getting sorted but each person who tried to fix the problem escalated to another person and continued.

Finally I got it sorted yesterday from one of our senior system admins and I am not sure why he didn't succeed last week in doing the same.

Our CTO has been informed about this problem and he will take care of this matter on his level.

[ThePants999]

Thanks for the response. However, we still don't believe our points have been addressed. We're looking for explanations and assurances - please see my previous post - and we've had half an explanation and no assurances.

My boss is now very keen on changing host, but I've talked him into giving one last chance. We're looking for a full and complete response to my previous post, answering all points therein. Consider this a sales question now, rather than support, because the answer will determine whether we're a customer or not.

[eUKhost.com]

Please list out the questions which were not addressed. I will answer those in a sequence.

As far as assurance is concerned you have an assurance from my side that henceforth you will get 100% uptime for your websites as well as all the features included under your hosting account.

I have gone through your previous questions and I am not able to make out what exactly was not answered. Please donot take it in a wrong way, I just need questions and answers will be coming next to those.

[ThePants999]

100% uptime? That's an ambitious guarantee! But if you're happy to make it, I'm happy to take it, thanks

Okay, I'll provide specific questions again. I'll start from 4 to avoid confusion with previous posts.

4.
(a) What was the root cause of the problem?
(b) How did it come to happen on a live server?
(c) What are you doing to ensure it never happens again?

5.
(a) Why did it take so long to resolve? You've explained that the problem was repeatedly escalated, but a major component being non-functional should be assigned a sufficiently high priority that said escalation should happen within minutes rather than days.
(b) What guarantees can you give us about future resolution of issues like this?

6.
(a) Why did we not get responses from support? You've said you "agree" that it was a problem, and you've said that each person who looked at it escalated it, but to me neither of those are an explanation as to why we waited so long.
(b) What guarantees can you give us about future support response times?

Thanks,

- Chris

[eUKhost.com]

4.
(a) What was the root cause of the problem?
(b) How did it come to happen on a live server?
(c) What are you doing to ensure it never happens again?
Ans :- PHP version was upgraded on the server and zend was also needed to be upgraded. Zend was upgraded on receipt of your ticket but but php.ini was not able to recognize path for zend exe and thats where things got messed up.

5.
(a) Why did it take so long to resolve? You've explained that the problem was repeatedly escalated, but a major component being non-functional should be assigned a sufficiently high priority that said escalation should happen within minutes rather than days.
(b) What guarantees can you give us about future resolution of issues like this?
Ans :- Zend download website for new releases goes down many times and it happened with our windows system admins couple of time. I have suggested them to keep all the updates on our server as this should not repeat again. We wont upgrade php version in future without downloading zend updates as this problem is a good example for us.

6.
(a) Why did we not get responses from support? You've said you "agree" that it was a problem, and you've said that each person who looked at it escalated it, but to me neither of those are an explanation as to why we waited so long.
(b) What guarantees can you give us about future support response times?
Ans :- Such problems never occur with our service but the problems with zend download website and then the problem started with our other windows server got this ticket on the bottom of more than 100 support tickets. We managed to make proper utilization of our resources in that phase but the phase was tough for anyone to go through and we managed to come out of it with some harsh experience.

[ThePants999]

Thanks for the prompt reply Mark. I'm finding it difficult to build an overall picture here, given the piecemeal nature of this, but I'll try to keep raising specific questions.

7.
(a) You're giving me conflicting explanations: you're saying that you did upgrade Zend, and it didn't work, but you're also telling me that the delay was due to Zend's website being down, which would be irrelevant if you had already downloaded the upgrade. Which is the case?
(b) If it was the former, please re-answer question 5 (a), as you've only told me the problem without telling me why it took so long to resolve.
(c) If it was the latter - Zend's website was really down for four days straight?

8. You stated earlier (post 3) that "We dont do any experiments on live servers." Given the potential for problems like this, how come you were upgrading PHP on the live server without having tested said upgrade on another server?

9. In response to your answer to 6 (a) - as I've said, I sympathise with your major server crash, but that happened four days after this ticket was raised. Furthermore, Zend's website being down does not stop you talking to us. Please, without mentioning the problems with the other server or Zend's website, explain why we waited three days for a support response.

Finally - I don't think we've had an answer to 4 (c), 5 (b) or 6 (b). Remember that Zend was the problem this time, but we rely on many other components, and this has raised more general concerns for us about your processes.

[eUKhost.com]

7.
(a) You're giving me conflicting explanations: you're saying that you did upgrade Zend, and it didn't work, but you're also telling me that the delay was due to Zend's website being down, which would be irrelevant if you had already downloaded the upgrade. Which is the case?
(b) If it was the former, please re-answer question 5 (a), as you've only told me the problem without telling me why it took so long to resolve.
(c) If it was the latter - Zend's website was really down for four days straight?
Ans :- Initial problems occurred due to zend and we don't keep on trying continuously to see if zend download starts working or not. Once the update was available we got it installed but our system admins were not able to figure out if the update was successful or not though the problem was inside php.ini

8. You stated earlier (post 3) that "We don't do any experiments on live servers." Given the potential for problems like this, how come you were upgrading PHP on the live server without having tested said upgrade on another server?
Ans :- This was not a experiment as we do similar upgrades on all our servers and we have good experience of doing such upgrades. We could never imagine what we can do incase the zend download website is down and there was nothing included in our books to tackle such situation. php version once upgraded should not be downgraded again and thats why we needed to wait for the zend download website to function. As I mentioned earlier that we didn't keep on trying constantly but there was problem with zend download website for at least 2 days.

9. In response to your answer to 6 (a) - as I've said, I sympathize with your major server crash, but that happened four days after this ticket was raised. Furthermore, Zend's website being down does not stop you talking to us. Please, without mentioning the problems with the other server or Zend's website, explain why we waited three days for a support response.
Ans :- I am not able to find anyone to blame for this problem. Its our mistake and we accept our mistake.

[ThePants999]

Thanks Mark. So, in future, what can we expect in terms of:

(a) problem resolution times
(b) support response times?

[eUKhost.com]

(a) problem resolution times
Ans :- Problem would be resolved in 4 - 6 hours max even if its a major technical problem.

(b) support response times?
Ans :- We answer 90% of ticket in less than an hour. Our response time has no impact of any problems with our Servers but last week we got into a situation where backup restoration had to be done manually and things went wrong over there. We have configured good backup solutions for our windows servers and the newly added hosting controller windows servers have rebuild express which allows you to restore backup of all the data on respective server in 2 - 3 hours. So in case of a major problem with the server we will be in a position to restore everything as it is before customers start opening tickets.

[ThePants999]

Thanks for the answers Mark. You've convinced my boss to stay, conditional on 100% uptime between now and the end of 2006 - looking forward to happy times now!

[eUKhost.com]

Thats good to hear that your Boss is happy now

Hosting business is not as easy as it seems. New Problems keep on popping up which keep on testing your management and technical skills and all you need to do is trust eukhost as we really care for you and your business.