WPAD and Microsoft's NCSI

One of my 2017 goals is to remove as much NTLM authentication as is possible. The big one  at this site was the Squid Proxy.

After implementing the new proxy with the only change being the removal of the NTLM authentication mechanism I started seeing users complain that they had no internet...

That can't be right I tested the crap out of this... 

The conversation would generally go something like this.

User: Everything is broken?

Me: What are you having trouble with?

User: There is a yellow exclamation mark over the wifi, I CANT DO ANYTHING!

Me: Can you try to access google for me.

User: That worked, Thanks for fixing it.

Me: Just ignore the yellow exclamation mark, I will fix that soon.

Unsure of why this is now a problem I look at what has changed.

• I have changed the WPAD file to point to the new server. 
• I have a newer version of SQUID.
• I have changed the way authentication works in SQUID.

After reading up on Microsoft NCSI (here, here and here). I get down and dirty in WireShark trying to figure out what could be causing this.

Outbound port 80 and 443 are blocked so it is stopping the inital handshake to www.msftncsi.com or msftconnecttest.com, But this has always been the case port 80 and 443 have been blocked for the last 15 years and it's only happend after changing to this new proxy. 

As part of the squid config I have a rule that allowes both www.msftncsi.com and msftconnecttest.com to bypass all authentication but looking at the loggs there has not been any requests to either of these domains... Strange.

I have an alternative proxy at this site that does not need any auth but has a heavily restricted whitelist of sites that are allowed through including the NCSI sites. If i change the wpad.dat file to point to the other proxy the exclamation mark goes away! Looking at the logs on this other server the NCSI request come through like normal.

10.x.x.x TCP_MISS/200 471 GET http://www.msftncsi.com/ncsi.txt - FIRSTUP_PARENT/123.x.x.x text/plain

Looking at wireshark the client still attempts to make a request to the ip's directly but after a few seconds they attempt through the proxy like they should.

So looking at the WPAD file I just changed I am referencing the alternative proxy by IP not FQDN. I change my wpad to point to the new proxy by ip address and what do you know it works but obviously kerberos won't work unless I am using the FQDN...

So i add this and it seems to works.

// If the hostname matches, send direct.

    if (dnsDomainIs(host, "www.msftconnecttest.com")||

dnsDomainIs(host, "www.msftncsi.com"))

        return "PROXY 10.x.x.x:8080";

I don't know why after changing proxy this issue has arisen as the old proxy also was referenced by FQDN...

I hope this can help others as there is not a huge amount of documentation on Microsofts NCSI.

Have Fun.