---EPISODE III, Revenge of the SIP---
First, thanks to everyone who has read the first two installments of this blogs. I really appreciated all of your feedback. Many of you had asked for me to write about the final phase of the phone project – rollout and conversion. As a result, I put something together and brought this project, and blog, to a close. So, let’s thrown a quarter into the jukebox, queue up the music, and finish this thing.
Just a small town girl, livin' in a lonely world
First, the overall project can be rated as nothing less than a success. We had zero day one issues after conversion in any of the four offices. We are now several weeks past the installation and practically no time is being spent dealing with the system other than day to day tasks. Things are working like they should. A good part of this was due to the quality of work from our engineers. The engineers that were assigned to us were highly experienced and had a combined tally of close to 200 Cisco VOIP installations under their belts. Many of their previous installations were on a much larger scale than my little operation of 4 offices and ~400+ phones. I think we may have been a nice break for them. Because of their experience, they anticipated some of the gotchas that inevitably came up and were more than ready to tackle them. Throughout the rollout, we stayed on target and on schedule. Most important, the conversion was smooth and we had zero disruptions to business. Things just worked on Day One.
Rollout
A singer in a smokey room…
For the rollout at the desks, we took the approach to place the new phones on people’s desks about a week before the actual conversion (with ample warning to not dial 911). People were allowed to touch and play with the new phones before training. Like most upgrades, the new phones had a lot of new features that the old phones did not. What features that were the same were obviously in a different location than before. This early rollout strategy was a huge benefit and time saver as my staff didn’t have to pull many late nights to accomplish a forklift type replacement. Rollout could be staggered, with time to spare. As a result of this early placement, the people in general were less intimidated with the phones and could begin to poke around and get their bearings before going live. They also were able to get a sense of ownership by setting ring tones and wallpaper (the most important aspect of any rollout). Since today’s smartphones are now more complicated to use than any desk phone, people showed up for training less intimidated and loaded with really good questions. For the attorneys, we scheduled one-on-one time with them as there were more features (mobility) that they would/could be utilizing. It was worth it to spend the time with them so they could see the additional tools at their fingertips. Plus configuring the mobility use with certain cellular carriers and phones can take some tweaking.
With Cisco mobility, you can configure a cell phone (or any other phone) to ring simultaneously with the desk phone without having to install any apps or software on your other phones. You can customize the length of time that mobility will attempt to ring a cell phone before pulling the call back and sending it to Unity Voicemail. You would want to customize this setting per user so a call destined for the desk phone does get passed through the cellphone’s voicemail. With some carriers and phone combinations, this is very seamless and works flawlessly out of the box. We are primarily a Verizon firm and there is something with how Verizon handles the calls passed to them and getting a smooth handoff from Cisco to Verizon takes some minor tweaking. For instance, using our default settings, my DROID would ring once and then pull the call back to Unity whereas the same call configuration to an AT&T iPhone worked as is. Obviously hearing one ring is not going to work, even if I am going to ignore your call, so different parameters had to be tuned accordingly. Fortunately this is really simple to do and can be done so through a web GUI. Smartphone model may play a factor in the settings as well.
Moving on, from the infrastructure side of things, deployment was really, really easy. As previously mentioned, our entire infrastructure was/is all new Cisco switches and routers. Cisco really makes it easy with their CDP protocols and other proprietary offerings to configure a network for voice. In order to turn on the voice capabilities of the network, there were minor engineering things that had to be configured and those could be completed in hours, not days. The experienced Cisco engineers were able to run through this rather quickly.
One thing to be careful with is Auto-QoS. The great philosopher Inigo Montoya once said “I do not think it means what you think it means.” Some of this is true with Auto-QoS. You cannot just turn it on and then slowly walk away like the hero in any action movie. There will be some explosions behind you if you do. By default, Cisco sets the maximum bandwidth allocation to some ridiculously low number even if your LAN is Gigabit (maybe 1kB, but don’t hold me to that). In hindsight, this makes total sense from Cisco’s perspective. QoS is something that needs proper thought and planning from an engineer for proper configuration. Once those settings get tuned for the speed and latency of your network, the “auto” feature shows its value by taking over and managing things for you by auto-discovering traffic on your network and assigning the appropriate QoS policies all by itself.
Gotchas
Strangers waiting, up and down the boulevard
No major IT initiative is not without its gotchas. Fortunately none of our gotchas had a negative impact to our deadlines. One gotcha that comes to mind is that we found out that in order for the survivability feature to work correctly with the new model phone that we were going to use, all of our routers were required to be on a much more recent code than what we were previously running. For those of you who really know Cisco, know that once you get on a stable code version, you really don’t want to move off of it. Cisco is very aggressive with its updates and as stands to reason, newer code isn’t as well baked as older code. Overall, this wasn’t really a bad thing and I know that this is really splitting hairs. This gives you an idea about how “bad” the bad news got with us. No surprises. Everything worked as advertised once we got all of the firmware in synch.
Phones
It goes on and on and on and on…
We chose to go with Cisco’s new generation of phones, the 89xx series. These phones have built in video capabilities for point to point video with any other Cisco phone on our network. The end user has to do nothing to initiate the video, unlike a video conferencing unit or software like Skype or Jabber. They do have controls on whether they want privacy or if they want the camera to auto answer. For the manual video, the caller simply pushes one button. Unifying the firm under a single phone system was indeed a primary objective and having video was not feature initially sought out. We went this route based on the no cost offer from Cisco during the contract proposal process.
It makes sense that people in different offices would make use of this technology. What surprised me was how much this video phone technology would be utilized and openly embraced by people working within the same office, and even in the same office area. The utilization is widespread. There is no doubt that this new technology has made a positive impact on firm communications just within a very short period of time. I encourage anyone who either has a Cisco telephone infrastructure in place already, or is looking at possibly acquiring a Cisco telephone system to add the 89xx series phones to your consideration. I think it’s time to ditch the old and tired design of the 79xx series phones. They still have their place in the world, but if you are starting new or looking to refresh your inventory, think 89xx versus 79xx model phones. I hear the 99xx series phones are really nice too.
On to one area, a little less than impressed.
Don’t stop…believing….
For our receptionists, we steered away from Cisco’s native attendant console and went with a third party called Vista Point. It is a relatively full featured and an inexpensive offering for running a console off of a pc. Overall impressions with the company made me wonder if they weren’t working out of their garage somewhere. The live, WebEx training session for our receptionist was lackluster to the point that after about 10 minutes into the training, our receptionist politely placed the trainer on mute and told me that she’d figure it out for herself and she did. The software does what it is supposed to and as I just mentioned, it is really easy to figure out. In the end I guess that is what matters despite first impressions.
Dealing with Telecom…just book your failures and disappointment now.
Don’t stop…believing….
Our only challenge during this entire process was dealing with the voice vendor providing services to three of our four offices. I won’t fully mention names here, so let’s refer to them as "TWTF". Let’s face it. Dealing with telecom companies in general just sucks, but this company took that to a whole new level. I mean they weren’t just bad, they were AT&T bad. I debated on whether or not to really go into great detail as to just how bad their failures were as I think if everyone knew the embarrassment felt and shaking of heads would be quite large. But, I decided to cut to the chase as our time is just about up on this blog and well, you get it. They dropped the ball and dropped the ball often. What was totally maddening was that most of the problems all stemmed from the simple request and paperwork handling process, or lack thereof. The failures of these simple requests was absolutely baffling. Clearly there are deep systemic problems within that organization. I know I shouldn’t do this, but I can’t help but offer some advice and I hope that they will take it and run with it. TWTF should take every person who answers a phone, sits in a center or does any touching of the paperwork, and force them to shadow the field technicians for several weeks. These people need to see firsthand what their poor work or inadequate procedures do to the real world customers and the poor field technician who has to bear the brunt of it all.
I know a lot of you are saying, but Mark, that’s why I use my account rep for moves and changes. I let them run things so that everything gets done right and on time. Yup, I know, me too. That is why this is story is even more egregious. Even though our account rep would herd the cattle on conference calls and plan the meetings, once the paperwork left his hands, he had zero control. He could do nothing but sit back and get his apologies warmed up. I took some serious heat for this company during this rollout and now it’s up to them to address appropriately or else they can begin to refer to us as a customer in the past tense. ‘Nuff said.
Wrapping things up
Streetlights people…
So I guess that neatly sums up the phone rollout process here at Rogers Townsend & Thomas. Everything was smooth minus the issues with the Telecom. In the beginning of the blog series, I intentionally withheld the name of the vendor selected for the phone system implementation. Many of you who have spoken to me privately know who I chose for the project. Many of the reasons of why they were selected over other vendors were posted here. Clearly, for us, the right product was selected, properly delivered and professionally installed. I debated whether or not to put the name of the vendor out there as I really wanted this blog to focus on the process and progress of this rollout. But, in the end, it was indeed this vendor who made this whole process work and ultimately delivered success at all phases of the project. So for those who care and really want to know, the vendor who was selected for this project was…
Don’t stop…
<Sopranos Ending>
Sorry, Michael H. I couldn't resist.