Very difficult to diagnose network issues – professional help probably required

Black Amnesia

Hi all,

We've a strange network issue in our office (~ 20 people) which I'm struggling to diagnose (no dedicated IT person here yet, I have a decent knowledge of networking though).

We're a predominantly Wifi network with the following setup:

UPC -> Cisco EPC3925 -> Cisco unmanaged switch -> 2x Ubiquity AC access points

We have a Mac Mini server doing DNS and DHCP.

For the most part the network flies, no problems, and speeds are excellent.

However our Internet sporadically drops (absolutely no pattern) for our Mac users, when I say drops you can still ping local IPs, but you can't access external services (e.g. anything on the web).

I thought maybe it was related to the Yosemite/discoveryd issue, but alas it's still happening now that they've brought mDNSresponder back in OS 10.10.4.

I have tried everything, including removing the Mac Mini, different access points (Airport Extreme/Meraki), connecting the AP directly to the router, using only 1 access point, etc., etc.. But the problem still exists.

Note running a ping from the MacMini to the router (wired connection) hasn't revealed any issues.

Note: we have a couple of Windows machines which seem to work fine.

So my questions are:

1) Has anyone any advice?
2) Does anyone know someone we could get in for a bit of time to try and isolated the problem/recommend a remedy?

Thanks..

seamus

Does it drop for both Macs at the same time, or randomly for each?

Black Amnesia

Randomly for each – I could have perfect access while the person beside me is pounding their keyboard in frustration.

I should also mention dropouts last for around 30 - 40 seconds – DHCP lease renew sometimes works

Dardania

sounds like an awful lot of users for that gateway...I would be concerned that there is too much traffic through it, and it couldn't cope.
Is it well cooled?
And do any users run P2P type applications (stuff that opens many ports)?
Is the WiFi on Cisco switched off (e.g. to save CPU load, seeing as the APs can handle WiFi in lieu)

Running the DNS from Mac Mini probably helps...although does the Mac Mini put it's DNS lookups to the Cisco, or to an external DNS directly?

When you note that the internal macs can ping internally, but not externally - when you ping externally are you pinging IP addresses, or domain names? I'd be curious to see if pinging external IP addresses presents issues...

For an off-piste query - any chance that the Macs might be trying to connect over IPv6? I'v eheard in some cases UPC are starting IPv6 in some odd way, that results in slow downs for people...a sort of NAT'd IPv6

Black Amnesia

Thanks for the reply Dardania..

The router is in a service room with aircon, so it should be well enough cooled – interesting point though, it doesn't feel like a suitable router for a growing office alright..

To answer your queries:

There isn't much P2P usage on the network (have run wireshark repeatedly trying to diagnose this and have not seen it)

Wifi is off on the Cisco.

DNS goes directly to an external resolver if it can't fulfil the request locally

If I recall correctly I wasn't able to reach external IP addresses when I was having problems (definitely couldn't reach external domain addresses)

I did actually switch my machine to Link-local only ipv6 this morning, will wait and see if that helps (waiting for the problem to arise is a pain in itself)

Dardania

Black Amnesia wrote: »

Thanks for the reply Dardania..

The router is in a service room with aircon, so it should be well enough cooled – interesting point though, it doesn't feel like a suitable router for a growing office alright..

To answer your queries:

There isn't much P2P usage on the network (have run wireshark repeatedly trying to diagnose this and have not seen it)

Wifi is off on the Cisco.

DNS goes directly to an external resolver if it can't fulfil the request locally

If I recall correctly I wasn't able to reach external IP addresses when I was having problems (definitely couldn't reach external domain addresses)

I did actually switch my machine to Link-local only ipv6 this morning, will wait and see if that helps (waiting for the problem to arise is a pain in itself)

So overheating and excessive processing due to WiFi can be eliminated for sure...

The DNS going externally direct might still be an issue of too many socket connections through the router. Or maybe there is something else contributing to a massive number of socket connections exceeding what the router can handle
The datasheet (http://www.cisco.com/c/dam/en/us/products/collateral/video/cable-modems-digital-voice-emta-edva/7018345_C.pdf ) says it has session tracking - if you can access that info from the router, it might show if there's a peak of connections at a coincident time.

What else is the router configured to do - does it do the QOS, and firewall function?

Cuddlesworth

Since your pinging the local network, its the router. I would look at bridging the Cisco router, it's not capable of managing traffic for 30+ devices.

Black Amnesia

@Dardania the router isn't doing anything else, will try and tap into that session tracking and see if I can get catch anything

@Cuddlesworth – interesting – do you think putting a decent firewall/router in front of it would suffice? Any recommendations?

mass_debater

Since you're already seeing the brilliance if Ubiquiti, get one of their Edge routers and bridge that cheapo nasty Cisco

Black Amnesia

A new router sounds like an option to explore..

Related to this – does anyone know of anyone who could provide some installation and support on our network?

kippy

Black Amnesia wrote: »

A new router sounds like an option to explore..

Related to this – does anyone know of anyone who could provide some installation and support on our network?

Where are you based? There are usually a few small outfits around that would support small offices. I wouldn't buy anything new until you get that sorted out.

Black Amnesia

Yeah going to hold off buying – I've been looking after the network here, but am also running the office so no time – we're in Dublin 1