GPS outage for 1 hour on 11/02/2011?

spaceHopper

I use GPS for work and was testing to see if our HW was loosing GPS lock from time to time. I set up a log of data for two hours and found something very strange. I lost GPS lock from 16:00:36 to 17:08:08 (log edned here) on 11:2:11 (last Friday) till my log ended. I’ve found a notice that there is some testing going on by the US DOD around Florida but nothing for Europe. Anybody else experience something similar? I’m in Dublin.

KoolKid

Have not noticed anything of the lenght anwhere.

Alun

A complete outage like that is highly improbable. Single sats get taken in and out of service all the time for maintenance, and these are all documented at http://www.navcen.uscg.gov/ in the so called NANU's, although these are complete gibberish unless you know how to decipher them.

OP, what's the application here? Network time sync by any chance?

spaceHopper

Alun wrote: »

A complete outage like that is highly improbable. Single sats get taken in and out of service all the time for maintenance, and these are all documented at http://www.navcen.uscg.gov/ in the so called NANU's, although these are complete gibberish unless you know how to decipher them.

OP, what's the application here? Network time sync by any chance?

Firstly I was wrong about the start time so I've updated it but I lost SV's for more than 1 hour! View of the sky could be better but I still find it odd.

Yep we are using it as a time reference

Alun

spaceHopper wrote: »

Firstly I was wrong about the start time so I've updated it but I lost SV's for more than 1 hour! View of the sky could be better but I still find it odd.

Yep we are using it as a time reference

Just sounds like a sub-optimal antenna placement to me. Especially in an urban environment surrounded by tall buildings it's remarkably easy to be unlucky and lose lock completely. Where is the antenna placed now?

If you're interested, you can download an application from the GPS manufacturer Trimble's website (http://www.trimble.com/planningsoftware.shtml) that allows you to plot the max available no of visible satellites for any location on earth and any time period. You'd be surprised how low that number drops to sometimes especially if you use the option to restrict the view to a number of degrees above the horizon.

Even at the default of 10 deg the number of visible sats regularly drops to only 6, and up it to say, 45 deg, which is quite possible if you're low down surrounded by buildings and you'll see drops to 2 sats only which will result in no lock.

Alun

There does indeed seem to be something a bit iffy with the GPS constellation at the moment. I ran the Trimble planning software for my location (Bray at 53m ASL, with a 10 degree cutoff) and there seems to be some very low satellite numbers around midday (dropping to only 2 at times!) and dangerously low numbers at other times too. I'll check it out tomorrow at midday with my GPS outside and see if that is actually happening, and if so try and check out what the story is.

I do know that they're currently in the process of bringing a few new satellites into service, and to do that they often have to nudge the orbits of some of the existing satellites to make space for the new ones, sometimes resulting in a less than optimal situation for some parts of the world for a limited time. Maybe that's what's happening here.

Scrap all that, I was using an almanac downloaded from the official US government website (http://www.navcen.uscg.gov/?pageName=gpsAlmanacs) and it appears I was importing the data using the wrong option which meant that the s/w wasn't compensating for a week number rollover sometime in 1999, and using the correct data give no lower than 7 sats for my location. Sorry about that, I thought something was wrong so double checked, and found it was my error

spaceHopper

Yea my antenna is on as stand in the car park but there isn't great view of the sky for the E/W, NS is fine and we don't normally have a problem. From looking at the NMEA ,messages I can see that the GPS receiver and see 12 SV's but went from tracking 7 down to tracking 2 then one.

$GPGSV,3,1,12,13,48,065,,10,86,191,,04,34,192,,02,52,240,27*7C
$GPGSV,3,2,12,08,32,169,,07,54,134,,05,39,292,,23,15,067,*7E
$GPGSV,3,3,12,29,13,319,,16,12,031,,26,06,243,,30,06,015,*78

Alun

Unfortunately also at these latitudes, the sky to the north is the area with the least number of sats in view that are at a decent elevation. Also even with a decent number of theoretically visible sats, their actual geometry can be sub optimal at times, for example with them all lining up in a row. If that row also happens to be at 90 degrees to the 'urban canyon' you find yourself in then you're really stuck.

You may want to look at getting the antenna up a bit higher on a pole on the roof or something.

Also, regarding time synchronization strategies in general, I hope you're not relying 100% on your GPS installation and employing secondary time servers as well (such as those from the Irish NTP pool) to keep going in case of outage?

coylemj

If you're using it as a time reference as opposed to using it to synchronize servers that are a long distance apart, surely a downtime of an hour is no big deal - wouldn't the server clock be sufficiently accurate over the space of an hour and you'd only be a small fraction of second out from UTC when you retrieve the GPS signal?

Alun

coylemj wrote: »

If you're using it as a time reference as opposed to using it to synchronize servers that are a long distance apart, surely a downtime of an hour is no big deal - wouldn't the server clock be sufficiently accurate over the space of an hour and you'd only be a small fraction of second out from UTC when you retrieve the GPS signal?

The problem is that clocks in various networked devices, servers, switches, routers etc. all drift at different rates, some quite badly, and one of the reasons many places do time synch at all is to ensure that it's possible to forensically recreate sequences of events that lead up to incidents such as security breaches etc. It's important then to be able to compare log files and the like to millisecond level accuracy, and a 'fraction of a second' just isn't good enough in such cases.

Anyway, as I mentioned it's common practice in such situations to not rely on one single time source, be that GPS or any other source, but to have additional sources such as internet based NTP servers or clocks with temperature stabilized crystals both as fall back and as a sanity check. The NTP protocol is quite sophisticated in this regard and can quickly identify and block out misbehaving clock sources.

Not every organization needs, or requires, this level of accuracy but they do exist.

coylemj

Alun wrote: »

The problem is that clocks in various networked devices, servers, switches, routers etc. all drift at different rates, some quite badly, and one of the reasons many places do time synch at all is to ensure that it's possible to forensically recreate sequences of events that lead up to incidents such as security breaches etc. It's important then to be able to compare log files and the like to millisecond level accuracy, and a 'fraction of a second' just isn't good enough in such cases.

Anyway, as I mentioned it's common practice in such situations to not rely on one single time source, be that GPS or any other source, but to have additional sources such as internet based NTP servers or clocks with temperature stabilized crystals both as fall back and as a sanity check. The NTP protocol is quite sophisticated in this regard and can quickly identify and block out misbehaving clock sources.

Not every organization needs, or requires, this level of accuracy but they do exist.

But my point is this: if the server with the GPS is being used as a time source, the time on that server will act as the master time source for the other servers so it doesn't really matter that the other servers are drifting by different amounts, that's why you have a time source in the first place. It's surely neither here or there whether over the course of an hour it drifts off UTC by a fraction of a second?

Alun

coylemj wrote: »

But my point is this: if the server with the GPS is being used as a time source, the time on that server will act as the master time source for the other servers so it doesn't really matter that the other servers are drifting by different amounts, that's why you have a time source in the first place. It's surely neither here or there whether over the course of an hour it drifts off UTC by a fraction of a second?

GPS based network time servers are standalone devices, and don't need to be, and often aren't, connected to a server machine, they just sit on the network and are available to all other devices on the network (not just servers) to synch to. If the GPS signal drops, they revert to their own internal clocks, which are also subject to drift.

Now, you're right, there are applications where absolute time is not that important, and the only really important thing is that everything is synched together relative to each other regardless of the absolute accuracy of that time, but there are also situations where absolute time accuracy is important, and in some cases, legally required. If you're comparing time stamps with logs from other organizations for example, then again, absolute accuracy is important.

In large networks it's common to have quite complex setups of hierarchical strata of time servers each supplied with a number of independent time sources, and also to have them peered with each other for robustness. See http://en.wikipedia.org/wiki/Network_Time_Protocol In many cases, these will be network devices such as switches and routers, many of which are capable of acting as both NTP clients and servers themselves. When you start adding multiple time sources for robustness, you also start adding extra complexity though, as you have to factor in what happens if one of those sources starts to misbehave. Luckily the NTP protocol takes care of that and can detect what it calls 'bad tickers' quite quickly.

I used to do this stuff for a living once upon a time

coylemj

I understand where're you're coming from. I suppose something like a geographically dispersed IBM mainframe sysplex could use independent GPS sources as time references, the reason being that the nodes would be too far apart to sync with one another over even a dedicated comms link. I could see where financial institutions would need to have complete accuracy with transaction timestamps because a gap of even a fraction of a second could potentially be exploited for fraudulent purposes.