Beginning of Day 6 without FIOS:
Previous story continued.
Brian the Verizon technician came at 5:00 PM today. At about 5:15, John the Verizon service rep called to tell me a technician had been dispatched.
It’s a mystery why John couldn’t pick up a (hypothetical) GPS on Brian’s truck, mash up the GPS data with a mapping program and determine without asking me that the truck was in my driveway.
But no. I had to tell John the service rep that Brian the technician was already here. He was asking **me** to keep **him** informed.
Brian put a little red laser light on the end of my fiber that points back into the network and went about 750 m down the road to the splitter box. At the splitter box he literally tried to see the light at the other end of the fiber.
Only about 10-15% of the ports in the splitter box were occupied — this is not a sustainable take rate under anybody’s business model. FIOS has been in our neighborhood since 2005. You’d think they’d have signed up a few more FIOS customers.
Brian the technician explained the different ways that technicians map drops to splitters. There’s standard company practice, then there’s a variation on that (offset by one), then there’s the possibility that a repair or a mistake screws up the whole system. He tried all the logical alternatives and did not see the light. Then he proposed to unplug every fiber to rule out the (very real) possibility that my service had been mapped onto a random port. This, of course, would interrupt every FIOS phone call and Internet stream. I egged him on, saying that the whole neighborhood has been getting service glitches since the Big Storm. We unplugged every fiber momentarily, but we never saw the light.
The fault, dear Brutus, was somewhere between my house and the splitter box. We slowly went back towards my house, looking at the wire as we went. About four poles down the street, between my house and the splitter box, there’s a place where something obviously fell against the wires. This, of course, is exactly where the technician that arrived on Sunday told us the problem was. In other words, Verizon re-discovered what Verizon knew on Sunday.
Brian explained that a CX & M — construction and maintenance — crew would need to come out. That’s what the Verizon guy said on Sunday too. Five days with their thumbs in their orifices to find out what they already knew. At my expense.
Then, a few minutes ago, John, the Verizon service rep, called to tell me that apparently the trouble was somewhere between the splitter box and my house. I told him that I really appreciated the information. I didn’t ask when my FIOS would be back on, because I realized I probably have a better guess than he does.
Technorati Tags: CustomerService, FIOS, FTTH, Utility, Verizon
Mags says:
I was a Verizon customer for two years, and this was pretty common for them. Poor communication, bad scheduling, but usually really nice employees when they’re finally there.
Whatever they’re using for dispatch is obviously heinous. (odds that it’s just an Excel sheet?)
March 26, 2010, 5:25 amPhil Salkie says:
This is nothing new with FiOS – it is a pervasive problem with Verizon’s entire maintenance system. I’ve seen it with new installs of POTS lines (technician after technician arriving day after day, each one saying “I’m going to change the wire between the street and the house”, and I’d say “No, that was done three days ago, then the next guy changed the line to the box on the corner, yesterday he changed the pair up to Main Street – you have to start from there…”) and with T1 lines (undocumented repeaters installed underground, technicians demanding site access for facilities where all Verizon equipment is outside, technicians arriving at 8:30 and taking until 2:30 to realize that they’re not trained on T1 equipment, and that they’ll need another tech to come out…)
We actually had a _21 DAY_ Verizon outage on a T1 line, with techs arriving every day to repeat the previous day’s exercises (that’s when they accidentally sent out a guy who was there at the install, who said “Oh yeah, there’s that underground repeater” – of course, he wasn’t allowed to go find and fix it, that was two days later and a different guy) – if there was another way to get 64 IPs in a single drop, I’d be on it like a blanket.
Long ago, every splice chamber and every junction box had a log book in it – the techs kept their maintenance records and notes to each other in those books. A computerized system was meant to replace all that, but AFAIK the techs don’t have access to those records while they’re in the field – so it’s not anything like a replacement for what was abandoned.
March 26, 2010, 11:14 amRob says:
Verizon told me FIOS would be in our L.A. Area neighborhood next year… That was five years ago, but still no FIOS.
March 26, 2010, 11:42 amSeanK says:
I’ve had problems identical to yours with Telus and Rogers in Western Canada – for communications companies, they are singularly incapable of good business communications within their own company structure.
Good luck with your FIOS repair.
BTW, I now live outside of Saskatoon, and the DSL quality out here is extremely slow – late-90′s/early-2000′s slow, and if anyone in the house streams a hi-def Youtube video or run a torrent, it completely hoses the connection fr everyone else – no amount of router tweaking will fix it.
*sigh*
Sasktel and all the other North American telecoms are now seemingly loathe to upgrade their equipment, and have been for years.
March 26, 2010, 12:27 pmfm says:
Three long years ago I attempted to sign up for Verizon DSL. I paid my monies and received a modem, a dsl-filter and some extra wires. I hooked it up. Nothing. Not a signal, not a flicker of a signal, not a thing. I called Verizon. Verizon told me they had to send a technician. The technician could only come between 9 and 5 during the week. In order to pay for DSL I had decided to take a 9 to 5 job with a local establishment. This job required me to be there between 9 and 5. They further explained that no other times were available, not weekends, not holidays, only 9 to 5 during the week. I pleaded with them for assistance. They offered none; I canceled the service. Three months later I received a letter in the mail telling me I had was a thieving criminal because I had not returned their equipment. In order to return the equipment I had to pay to ship it back to them or pay for a technician to pick it up. I paid to ship it back to them with the understanding that they would not charge me additional monies. All in all; I received nothing, lost a few hours and five dollars.
March 26, 2010, 9:14 pmab3a says:
This is nothing new. Dispatch management systems that operate well on scales like this are one of a kind programs and policies.
It may amuse you to know that for decades prior to that laser tracer thing you saw, there were (and are still) “fox and hound” tone and tracer sets for finding signals on masses of copper wires. I’ve used them. Sometimes it’s best not to rely upon written or computer records after the fact.
Your technician, had he been better equipped, could have used an Optical Time Domain Reflectometer (OTDR) to figure out where the break was. Nevertheless, wherever the break was, it isn’t usually something a single person could handle.
The problem here is that once upon a time, technicians were assigned to specific areas and were supposed to know where the wires went. Then, succession planning took over when they began to realize that these technicians were carrying ridiculously valuable information with them when the retired or left the company. I can remember Bell Atlantic (!) boxes and even C&P boxes with funny graffiti inside them from the various technicians who worked on it. There were also wire numbering codes that if you knew what they meant would tell you exactly what you needed to know about where the wires came from and went to.
Today, everything goes on a computer. The people who may be on site may know what you have or they may not. Not one of them is smart enough to follow where the wire goes. Succession planning means giving everyone a work-order with very precise, but possibly inaccurate, information of what things should look like to the customer premises.
However, the person working there may not know anything about how the town, city, or neighborhood was wired. Those people who did that wiring are long gone. Today, we have managers who “manage” the wire.
This is what happens when you attempt to institutionalize common sense.
March 26, 2010, 9:38 pmJohnny says:
Ahh, Verizon… A few years back we bought a new house in a Verizon-monopolized area, and quickly came to hate their customer dis-service. We signed up for DSL, and waited and waited and waited and waited for it to be hooked up. At one point we were promised it would be working by Tuesday. The following Friday rolled around, still no connection, so we called Verizon, said we had been promised Tuesday. The woman on the other end asked–in all seriousness–if it was Tuesday yet where we were.
March 26, 2010, 10:48 pmJamie says:
I had a similar experience with Comcast. The line to our house no longer worked for some reason. After several visits by technicians (and promised visits that never happened) another tech came out and explained a new cable would be needed. But he didn’t have one. Imagine that. A cable company repair person without cable. What was he carrying in the back of his van? After about 3 or 4 weeks, it was finally fixed.
March 27, 2010, 12:32 amJimmy says:
Don’t feel too bad. It’s isn’t the just the newcomers that don’t know their @#$%! from a whole in the ground. Last year the company I was working for had their AT&T DSL go out. Bottom line: threes weeks without internet, seven calls to customer service (more than 10 hours on hold), a field tech that changed my server’s network settings so that no one could access the server, and that’s not even the best part. It turns out that the problem was at one of their “junctions” in Fresno. Our static connection was “too old” so they “upgraded” our service to a dynamic connection that could not figure how to change back. Do you know how fun it is to maintain VPN connections without a static IP? Lucky me though, AT&T is the only high-speed provider in my small town. One last complaint, I was disconnected while on the phone with them twice… disconnected from AT&T.
March 27, 2010, 3:38 pmNo Green Shoots Here (Even Though it’s Spring) « Out Of My Mind says:
[...] The good old days…: The phone companies used to be pro-privacy before they turned into snitches and rats…. Of course, now they’re also inept assholes. [...]
March 27, 2010, 6:11 pmGreho says:
Wait wait… something fell on a fiber line…
Why don’t we bury delicate cables so they are fire/tree/wind/snow/wtf resistant?
Direct-buried cable isn’t a panacea, either. They are waaay too easy to cut. But maybe some kind of resilient conduit? Power lines and gas lines and water pipes last years and years, don’t they?
March 29, 2010, 1:24 amGreho says:
FWIW… I live where hurricanes happen, not where earthquakes happen. Obviously, one size does not fit all. Buried utilities resist wind and storm, but I’d reckon the ground moving poses other challenges.
March 29, 2010, 1:28 am