niXforums Forum Index
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   PreferencesPreferences   Log in to check your private messagesLog in to check your private messages   Log inLog in 
·  nixdoc.net ·  man pages ·  Linux HOWTOs ·  FreeBSD Tips ·  Forums
navigation Forum index » *nix » BSD » FreeBSD
major DNS hiccup
Post new topic   Reply to topic Page 1 of 4 [54 Posts] View previous topic :: View next topic
Goto page:  1, 2, 3, 4 Next
Author Message
Mike Scott
*nix forums Guru Wannabe


Joined: 25 Feb 2005
Posts: 138

PostPosted: Thu Jul 20, 2006 2:07 pm    Post subject: Re: major DNS hiccup Reply with quote

Andrew Haley wrote:
....
Quote:
Good luck.

I wish it wasn't needed! Have you noticed how computer books in the
library are shelved next to the "unexplained and supernatural phenomena"
section? At least, they are round here. I'm beginning to wonder if they
know something I ought to :-)

But thanks! I'll return when I've heard something new.

--
Please use the corrected version of the address below for replies.
Replies to the header address will be junked, as will mail from
various domains listed at www.scottsonline.org.uk
Mike Scott Harlow Essex England.(unet -a-t- scottsonline.org.uk)
Back to top
Andrew Haley
*nix forums beginner


Joined: 11 Jul 2006
Posts: 7

PostPosted: Thu Jul 20, 2006 1:22 pm    Post subject: Re: major DNS hiccup Reply with quote

Mike Scott <usenet.10@spam.stopper.scottsonline.org.uk> wrote:
Quote:
Andrew Haley wrote:
...
Whatever the details, this /has/ to be something on the network - all
was well until July 6th when caching nameservers seem to have started
failing for various people on the same segment of ntl's network. I've
logged a second fault call with ntl, requesting very firmly that the
issue be escalated past front-line support to the relevant group.
Hopefully, they'll have some sort of change record, and will realise
from that at least what's going on.

I'll be interested to see how you get on.

I take it this is now of academic interest only, and you have a
fully working named when you set your forwarders to NTL's
nameservers?

The original reason for running my own nameserver was (a)
self-instruction, and (b) the reputation of ntl's servers' reliability.

In fact, even using their servers, I've had the odd transient glitch
that looks awfully like the same problem (web access failed; nslookup
returned a server failure message; - but then it all cleared again with
seconds).

Now, that is all very interesting to know. I was just about to go
back to using my NTL connection becasue it's so much faster than the
Demon ADSL connection, but maybe I shouldn't. I'll think on it.

Quote:
So not /quite/ academic interest, but I guess the urgency (if it was
ever there) has gone.

No response yet, btw, to the second formal fault report. I'm also
trying to raise the topic on chetnet (seems to be a back-door into
ntl support) but haven't managed to get a posting accepted
yet. Problems on top of problems Sad And all I was trying to do was
get some music transposed......

Good luck.

Andrew.
Back to top
Mike Scott
*nix forums Guru Wannabe


Joined: 25 Feb 2005
Posts: 138

PostPosted: Thu Jul 20, 2006 9:49 am    Post subject: Re: major DNS hiccup Reply with quote

Andrew Haley wrote:
....
Quote:
Whatever the details, this /has/ to be something on the network - all
was well until July 6th when caching nameservers seem to have started
failing for various people on the same segment of ntl's network. I've
logged a second fault call with ntl, requesting very firmly that the
issue be escalated past front-line support to the relevant group.
Hopefully, they'll have some sort of change record, and will realise
from that at least what's going on.

I'll be interested to see how you get on.

I take it this is now of academic interest only, and you have a fully
working named when you set your forwarders to NTL's nameservers?

The original reason for running my own nameserver was (a)
self-instruction, and (b) the reputation of ntl's servers' reliability.

In fact, even using their servers, I've had the odd transient glitch
that looks awfully like the same problem (web access failed; nslookup
returned a server failure message; - but then it all cleared again with
seconds).

So not /quite/ academic interest, but I guess the urgency (if it was
ever there) has gone.

No response yet, btw, to the second formal fault report. I'm also
trying to raise the topic on chetnet (seems to be a back-door into ntl
support) but haven't managed to get a posting accepted yet. Problems on
top of problems Sad And all I was trying to do was get some music
transposed......

--
Please use the corrected version of the address below for replies.
Replies to the header address will be junked, as will mail from
various domains listed at www.scottsonline.org.uk
Mike Scott Harlow Essex England.(unet -a-t- scottsonline.org.uk)
Back to top
Andrew Haley
*nix forums beginner


Joined: 11 Jul 2006
Posts: 7

PostPosted: Wed Jul 19, 2006 6:17 pm    Post subject: Re: major DNS hiccup Reply with quote

Mike Scott <usenet.10@spam.stopper.scottsonline.org.uk> wrote:
Quote:
Mark Andrews wrote:
...
I suspect that they have installed a "transparent DNS cache".

Nameservers require authoritative answers (aa flag set)
from authoritative servers. A "transparent DNS cache" will
result in non-authoritative answers. Lack of "aa" is a
indication that the zone is not loaded or partially loaded
on the authoritative server.

OK, I take your point. (I am, btw, headed well out of my depth here;
nameservers are something I've used and accepted as "just working" in
the past, without worrying to much about the nitty gritty.)


A "transparent DNS cache" will not be visible to a stub
resolver which is expecting to get answers from a cache.

A interative resolver (which named is) exects to get
authoritative answers or referrals.

When named is taking to a forwarder it is expecting to
get non-authoritative answers from the forwarder which
is why things work when you configure forwarders.

OK, so what happens when named thinks it's talking to an authoritative
server, and it's actually something different?

Checking a handful of queries that actually do work using my nameserver,
I find they all have "flags: qr rd ra", none with aa.

Whatever the details, this /has/ to be something on the network - all
was well until July 6th when caching nameservers seem to have started
failing for various people on the same segment of ntl's network. I've
logged a second fault call with ntl, requesting very firmly that the
issue be escalated past front-line support to the relevant group.
Hopefully, they'll have some sort of change record, and will realise
from that at least what's going on.

I'll be interested to see how you get on.

I take it this is now of academic interest only, and you have a fully
working named when you set your forwarders to NTL's nameservers?

Andrew
Back to top
Mike Scott
*nix forums Guru Wannabe


Joined: 25 Feb 2005
Posts: 138

PostPosted: Wed Jul 19, 2006 11:16 am    Post subject: Re: major DNS hiccup Reply with quote

jpd wrote:
Quote:
Begin <o23vg.15527$EK1.13934@newsfe4-gui.ntli.net
On 2006-07-18, Mike Scott <usenet.10@spam.stopper.scottsonline.org.uk> wrote:
The front line support have already denied any network changes affecting
DNS. I've no idea at all how to get things like this escalated - front
line seems to take the view that rebooting cures almost everything,
otherwise they call an engineer to come and look at the modem Smile The
very concept of networking problems seems foreign to them.

Don't charge the first line.
....
If you encounter particularly helpful and clueful people, keep track
of that too, and after the problem is fixed, send a thank you note
with a commendation about those people to their bosses.



Thanks for the tips - a followup problem report is already with them --
firm but saccharine sweet I hope in its politeness, and asking for the
problem to be escalated to the appropriate group.

Actually, thinking of helpful people, there may be a 'back door' into
support. I'll check!

--
Please use the corrected version of the address below for replies.
Replies to the header address will be junked, as will mail from
various domains listed at www.scottsonline.org.uk
Mike Scott Harlow Essex England.(unet -a-t- scottsonline.org.uk)
Back to top
jpd
*nix forums Guru


Joined: 22 Feb 2005
Posts: 877

PostPosted: Wed Jul 19, 2006 11:11 am    Post subject: Re: major DNS hiccup Reply with quote

Begin <ucnvg.16276$EK1.6129@newsfe4-gui.ntli.net>
On 2006-07-19, Mike Scott <usenet.10@spam.stopper.scottsonline.org.uk> wrote:
Quote:
Incidentally, as a result of all this, I'm wondering about the relative
fragility of protocols that work only on a defined port number (DNS,
SMTP particularly) - these can all be broken or hijacked very readily by
some central authority: contrast with http, where any destination port
is possible (so, eg, when ntl's /web/ proxies break, it's quite easy to
bypass them).

No, it isn't. You still need to find somewhere off-site that will proxy
your requests from your alternative destination port to port 80 again,
if only because of the millions of webservers that expect you to use
port 80.

Likewise, it isn't exactly hard to run an smptd or a dns server on a
port other than their well known ones. It's just that nobody ever does
so, whereas too many companies had firewalls blocking just port 80 so
many people implemented all sorts of workarounds. The difference is
historic, not technical.


--
j p d (at) d s b (dot) t u d e l f t (dot) n l .
This message was originally posted on Usenet in plain text.
Any other representation, additions, or changes do not have my
consent and may be a violation of international copyright law.
Back to top
jpd
*nix forums Guru


Joined: 22 Feb 2005
Posts: 877

PostPosted: Wed Jul 19, 2006 11:04 am    Post subject: Re: major DNS hiccup Reply with quote

Begin <o23vg.15527$EK1.13934@newsfe4-gui.ntli.net>
On 2006-07-18, Mike Scott <usenet.10@spam.stopper.scottsonline.org.uk> wrote:
Quote:
The front line support have already denied any network changes affecting
DNS. I've no idea at all how to get things like this escalated - front
line seems to take the view that rebooting cures almost everything,
otherwise they call an engineer to come and look at the modem Smile The
very concept of networking problems seems foreign to them.

Don't charge the first line.

First line support is very good at keeping people at bay, calming irate
customers, filtering out the user error problems, and so on. That's what
they're there for. They're not there to fix anything.

At this point you really just want to pass their firewall and talk to
someone else. Someone started with saying literally that, and after
a long conversation (he can be quite persistent and can keep that up
for hours) said ``well, I said I wanted to talk to someone else at the
start, didn't I?'' when he finally got passed off to the second line.
That isn't particularly effective: The first line isn't allowed to pass
you off just because you want to talk to someone else.

Tell them exactly what you're seeing until their heads explode so they
pass you onto someone more knowledgeable. Mix in strong hints that you
checked and doublechecked your stuff and that you don't need handholding
while you support your own system, thank you, but carefully.

Describe your problem, not as if it's their fault, but as if you're
seeing an anomaly and need it investigated. Make them conclude that it
really must be their fault in their network on their side and that you
have proof. Lather, rinse, repeat until you talk to someone with actual
DNS clue. Get a trouble ticket written (get the number) with a promise
to call you back, then follow up on that after a day or two.

Be honest, have the proof ready, and don't act all superior. Just walk
them through your problems and your evidence. Give them a crash course
network monitoring and packet inspecting in that manner if you have to.

If that fails, take it to paper (fax or letter, not email). Keep track
of the names of everybody you talk to, and the times, and maybe the
names of their direct superiors, too. Also, re-read contracts and SLA
documents to see exactly what hoops you need to jump through so you can
put their noses to the grindstone.

Understandably, they'll be miffed you're expecting them to live up to
their SLA for a residential contract, but you do pay them a little for
that low service level so them being miffed really is their problem.
Your problem is to make clear they have a problem in their network and
that they're not living up to their SLA and that you've done your part
to notify them of that. In writing, if necessairy.

If you encounter particularly helpful and clueful people, keep track
of that too, and after the problem is fixed, send a thank you note
with a commendation about those people to their bosses.


--
j p d (at) d s b (dot) t u d e l f t (dot) n l .
This message was originally posted on Usenet in plain text.
Any other representation, additions, or changes do not have my
consent and may be a violation of international copyright law.
Back to top
Mike Scott
*nix forums Guru Wannabe


Joined: 25 Feb 2005
Posts: 138

PostPosted: Wed Jul 19, 2006 9:47 am    Post subject: Re: major DNS hiccup Reply with quote

Mark Andrews wrote:
....
Quote:
I suspect that they have installed a "transparent DNS cache".

Nameservers require authoritative answers (aa flag set)
from authoritative servers. A "transparent DNS cache" will
result in non-authoritative answers. Lack of "aa" is a
indication that the zone is not loaded or partially loaded
on the authoritative server.

OK, I take your point. (I am, btw, headed well out of my depth here;
nameservers are something I've used and accepted as "just working" in
the past, without worrying to much about the nitty gritty.)

Quote:

A "transparent DNS cache" will not be visible to a stub
resolver which is expecting to get answers from a cache.

A interative resolver (which named is) exects to get
authoritative answers or referrals.

When named is taking to a forwarder it is expecting to
get non-authoritative answers from the forwarder which
is why things work when you configure forwarders.

OK, so what happens when named thinks it's talking to an authoritative
server, and it's actually something different?

Checking a handful of queries that actually do work using my nameserver,
I find they all have "flags: qr rd ra", none with aa.

Whatever the details, this /has/ to be something on the network - all
was well until July 6th when caching nameservers seem to have started
failing for various people on the same segment of ntl's network. I've
logged a second fault call with ntl, requesting very firmly that the
issue be escalated past front-line support to the relevant group.
Hopefully, they'll have some sort of change record, and will realise
from that at least what's going on.

===
Incidentally, as a result of all this, I'm wondering about the relative
fragility of protocols that work only on a defined port number (DNS,
SMTP particularly) - these can all be broken or hijacked very readily by
some central authority: contrast with http, where any destination port
is possible (so, eg, when ntl's /web/ proxies break, it's quite easy to
bypass them). I guess this isn't the right group for such musings - but
I'd hope anyway I'm not the only one to think about this.

--
Please use the corrected version of the address below for replies.
Replies to the header address will be junked, as will mail from
various domains listed at www.scottsonline.org.uk
Mike Scott Harlow Essex England.(unet -a-t- scottsonline.org.uk)
Back to top
Mark Andrews
*nix forums beginner


Joined: 02 Mar 2005
Posts: 25

PostPosted: Wed Jul 19, 2006 4:17 am    Post subject: Re: major DNS hiccup Reply with quote

Quote:
Incidentally, I also tried accessing the two nameservers shown in those
packets directly. That seems to work for dig:

data# dig @194.74.151.194 www.yell.co.uk

; <<>> DiG 9.3.2 <<>> @194.74.151.194 www.yell.co.uk
; (1 server found)
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 7427
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;www.yell.co.uk. IN A

;; ANSWER SECTION:
www.yell.co.uk. 85376 IN A 194.72.108.2

;; Query time: 18 msec
;; SERVER: 194.74.151.194#53(194.74.151.194)
;; WHEN: Mon Jul 17 11:47:47 2006
;; MSG SIZE rcvd: 48

Actually it didn't work and it proves my supposition in
the other messages. If you were actually talking to the
authoritative servers then you would see "aa" in the
flags. You are in fact talking to a cache.

This is what you should be seeing. Note "aa" is set and
"ra" is not set. In addition you get the NS RRset and
associated glue.

Mark

; <<>> DiG 9.3.2 <<>> @194.74.151.194 www.yell.co.uk
; (1 server found)
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4659
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 2

;; QUESTION SECTION:
;www.yell.co.uk. IN A

;; ANSWER SECTION:
www.yell.co.uk. 86400 IN A 194.72.108.2

;; AUTHORITY SECTION:
yell.co.uk. 86400 IN NS redgate.yellowpages.co.uk.
yell.co.uk. 86400 IN NS redgate2.yellowpages.co.uk.

;; ADDITIONAL SECTION:
redgate.yellowpages.co.uk. 86400 IN A 194.74.151.200
redgate2.yellowpages.co.uk. 86400 IN A 194.74.151.194

;; Query time: 307 msec
;; SERVER: 194.74.151.194#53(194.74.151.194)
;; WHEN: Wed Jul 19 14:12:16 2006
;; MSG SIZE rcvd: 137

Quote:
Also

data# dig @194.74.151.200 www.yell.co.uk

; <<>> DiG 9.3.2 <<>> @194.74.151.200 www.yell.co.uk
; (1 server found)
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 17678
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;www.yell.co.uk. IN A

;; ANSWER SECTION:
www.yell.co.uk. 85190 IN A 194.72.108.2

;; Query time: 9 msec
;; SERVER: 194.74.151.200#53(194.74.151.200)
;; WHEN: Mon Jul 17 11:50:52 2006
;; MSG SIZE rcvd: 48




I don't know what that proves though.

--
Please use the corrected version of the address below for replies.
Replies to the header address will be junked, as will mail from
various domains listed at www.scottsonline.org.uk
Mike Scott Harlow Essex England.(unet -a-t- scottsonline.org.uk)
Back to top
Mark Andrews
*nix forums beginner


Joined: 02 Mar 2005
Posts: 25

PostPosted: Wed Jul 19, 2006 4:03 am    Post subject: Re: major DNS hiccup Reply with quote

In article <12b7gv9jth8jdaa@news.supernews.com>,
Andrew Haley <andrew29@littlepinkcloud.invalid> wrote:
Quote:
Andrew Haley <andrew29@littlepinkcloud.invalid> wrote:
Mike Scott <usenet.10@spam.stopper.scottsonline.org.uk> wrote:
Chronos wrote:
After replacing Mike Scott with a small shell script on Tuesday 11 Jul
2006 09:06, the following appeared on stdout:

Thanks, but as it's only DNS that's not working

And then only if you're doing fully recursive lookups. Is your hints
file up to date?

Yes.

Have you tried an alternate root such as ORSN?

I have now. Same problem (sample of 2 failure, 1 working).

But thanks for the ideas. Somehow I get the most horrible feeling this
will all be down to something incredibly obvious that I've just been
missing totally.....

FWIW, my failures began at 11:03:54 on Jul 6.

Mike, can you have a look at your named logs, and see when it started?

Andrew.


Jul 6 11:03:54 mammoth named[2671]: lame server resolving 'gmail.com'
(in 'gmail.com'?): 216.239.34.10#53
Jul 6 11:03:54 mammoth named[2671]: lame server resolving
'174.176.132.209.sbl-xbl.spamhaus.org' (in 'sbl-xbl.spamhaus.org'?):
193.70.192.64#53
Jul 6 11:03:54 mammoth named[2671]: lame server resolving
'190.182.233.64.sbl-xbl.spamhaus.org' (in 'sbl-xbl.spamhaus.org'?):
193.70.192.64#53
Jul 6 11:03:54 mammoth named[2671]: lame server resolving
'rbldns4.sorbs.net' (in 'sorbs.NET'?): 213.155.150.206#53
Jul 6 11:03:54 mammoth named[2671]: lame server resolving
'rbldns4.sorbs.net' (in 'sorbs.NET'?): 63.209.15.211#53
Jul 6 11:03:54 mammoth named[2671]: lame server resolving
'190.182.233.64.sa-other.bondedsender.org' (in
'sa-other.bondedsender.org'?): 64.92.165.122#5
3
Jul 6 11:03:54 mammoth named[2671]: lame server resolving
'174.176.132.209.sa-trusted.bondedsender.org' (in
'sa-trusted.bondedsender.org'?): 64.92.165.
122#53
Jul 6 11:03:54 mammoth named[2671]: lame server resolving
'rbldns4.sorbs.net' (in 'sorbs.NET'?): 217.70.177.40#53
Jul 6 11:03:54 mammoth named[2671]: lame server resolving
'rbldns4.sorbs.net' (in 'sorbs.NET'?): 209.97.199.116#53
Jul 6 11:07:21 mammoth named[2671]: lame server resolving
'backmx.iol.cz' (in 'iol.cz'?): 194.228.2.1#53
Jul 6 11:07:21 mammoth named[2671]: lame server resolving
'backmx.iol.cz' (in 'iol.cz'?): 194.228.2.61#53
Jul 6 11:07:21 mammoth named[2671]: lame server resolving
'backmx.iol.cz' (in 'iol.cz'?): 194.228.2.1#53
Jul 6 11:07:21 mammoth named[2671]: lame server resolving
'backmx.iol.cz' (in 'iol.cz'?): 194.228.2.61#53
Jul 6 11:07:21 mammoth named[2671]: lame server resolving
'backmx.iol.cz' (in 'iol.cz'?): 194.228.2.61#53
Jul 6 11:07:21 mammoth named[2671]: lame server resolving
'backmx.iol.cz' (in 'iol.cz'?): 194.228.2.1#53
Jul 6 11:07:21 mammoth named[2671]: lame server resolving
'backmx.iol.cz' (in 'iol.cz'?): 194.228.2.1#53
Jul 6 11:07:21 mammoth named[2671]: lame server resolving
'backmx.iol.cz' (in 'iol.cz'?): 194.228.2.61#53
Jul 6 11:07:21 mammoth named[2671]: lame server resolving
'backmx.iol.cz' (in 'iol.cz'?): 194.228.2.61#53
...

I suspect that they have installed a "transparent DNS cache".

Nameservers require authoritative answers (aa flag set)
from authoritative servers. A "transparent DNS cache" will
result in non-authoritative answers. Lack of "aa" is a
indication that the zone is not loaded or partially loaded
on the authoritative server.

A "transparent DNS cache" will not be visible to a stub
resolver which is expecting to get answers from a cache.

A interative resolver (which named is) exects to get
authoritative answers or referrals.

When named is taking to a forwarder it is expecting to
get non-authoritative answers from the forwarder which
is why things work when you configure forwarders.

"dig backmx.iol.cz @194.228.2.61" should return something
like this. Note "aa" is set in flags. If you are getting
a answer without "aa" being set then you should complain.

Mark

; <<>> DiG 9.3.2 <<>> backmx.iol.cz @194.228.2.61
; (1 server found)
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 37692
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 3, ADDITIONAL: 3

;; QUESTION SECTION:
;backmx.iol.cz. IN A

;; ANSWER SECTION:
backmx.iol.cz. 3600 IN A 194.228.41.112

;; AUTHORITY SECTION:
iol.cz. 3600 IN NS dns-polvezni.iol.cz.
iol.cz. 3600 IN NS dns.iol.cz.
iol.cz. 3600 IN NS ns2.tel.cz.

;; ADDITIONAL SECTION:
dns.iol.cz. 3600 IN A 194.228.2.61
ns2.tel.cz. 86400 IN A 194.228.2.1
dns-polvezni.iol.cz. 3600 IN A 194.228.41.113

;; Query time: 330 msec
;; SERVER: 194.228.2.61#53(194.228.2.61)
;; WHEN: Wed Jul 19 14:01:58 2006
;; MSG SIZE rcvd: 162
Back to top
Mike Scott
*nix forums Guru Wannabe


Joined: 25 Feb 2005
Posts: 138

PostPosted: Tue Jul 18, 2006 10:51 am    Post subject: Re: major DNS hiccup Reply with quote

Per Hedeland wrote:
Quote:
In article <X6Oug.14695$EK1.11536@newsfe4-gui.ntli.net> Mike Scott
usenet.10@spam.stopper.scottsonline.org.uk> writes:
Out of curiosity, I tried traceroute -p 53 to one of the root
nameservers. The hypothesis was that if ntl have put some sort of
transparent cache into place, this /ought/ not to reach the root server
- maybe!

That won't work - traceroute always uses multiple ports (increments by
one for each probe), -p just gives the "base" one. I.e. by the time the
probes reach the "interesting" place, the port won't be 53 anymore.

Yes, thanks; I figured that out the hard way after posting (maybe I
should start reading man pages Smile ).

And I didn't know that it uses the remote port number to encode the
attempt identification, icmp replies apparently not including enough
data to match them to the particular sent packets. So it's not quite
straightforward to send DNS probes in the same way. I'll keep it on the
back burner - if ntl are intercepting DNS packets, there has to be a way
to prove it!

....
Quote:
But I really don't think that your ISP would deny that they are doing
some sort of "intercept" of the DNS traffic (assuming that you can get

The front line support have already denied any network changes affecting
DNS. I've no idea at all how to get things like this escalated - front
line seems to take the view that rebooting cures almost everything,
otherwise they call an engineer to come and look at the modem Smile The
very concept of networking problems seems foreign to them.

Thanks again for your help, and of course to the others who've put time
into this!

--
Please use the corrected version of the address below for replies.
Replies to the header address will be junked, as will mail from
various domains listed at www.scottsonline.org.uk
Mike Scott Harlow Essex England.(unet -a-t- scottsonline.org.uk)
Back to top
Per Hedeland
*nix forums Guru Wannabe


Joined: 20 Feb 2005
Posts: 182

PostPosted: Mon Jul 17, 2006 9:09 pm    Post subject: Re: major DNS hiccup Reply with quote

In article <4i1phlF1ogn2U1@individual.net> jpd
<read_the_sig@do.not.spam.it.invalid> writes:
Quote:
Begin <e9g4th$o7f$1@hedeland.org
On 2006-07-17, Per Hedeland <per@hedeland.org> wrote:
If all else fails, you could perhaps hack named to not use EDSN0 at all
- I didn't find any such setting,

IIRC from last time I looked that there is a global setting that regulates
the maximum value to ever advertise. Setting it to 512 might cause bind
to never include EDNS0 options, or at least reduce the probability that
packets get truncated.

There is such a setting, but I believe Mike already tried setting it to
512 w/o improvement - and IIRC BIND will send the EDNS0 option anyway,
just with the value specified. Also, the response in the example he
posted is "small", it doesn't actually use the offer of sending bigger
packets.

--Per Hedeland
per@hedeland.org
Back to top
Per Hedeland
*nix forums Guru Wannabe


Joined: 20 Feb 2005
Posts: 182

PostPosted: Mon Jul 17, 2006 9:02 pm    Post subject: Re: major DNS hiccup Reply with quote

In article <X6Oug.14695$EK1.11536@newsfe4-gui.ntli.net> Mike Scott
<usenet.10@spam.stopper.scottsonline.org.uk> writes:
Quote:

Out of curiosity, I tried traceroute -p 53 to one of the root
nameservers. The hypothesis was that if ntl have put some sort of
transparent cache into place, this /ought/ not to reach the root server
- maybe!

That won't work - traceroute always uses multiple ports (increments by
one for each probe), -p just gives the "base" one. I.e. by the time the
probes reach the "interesting" place, the port won't be 53 anymore.

Quote:
In fact, it did eventually, but only after some very long
delays; I'm not sure a one-off proves much anyway - perhaps some further
thought along those lines might give an indicator (eg craft a
traceroute-like thing that sends a genuine dns query, and sees where it
reaches. Maybe I could hack the traceroute sources).

Yes - there is a 'tcptraceroute' that really does the same thing as a
TCP connection attempt would, and which can tell you "interesting"
things (like proceeding across a DNAT/"port forwarding" firewall and
mapping out the network behind it:-) - but I haven't heard of a
'udptraceroute' along those lines. It might be enough just to keep the
port fixed for traceroute - you won't get the "final hit" line which is
based on receiveing an ICMP "port unreachable", but you would see where
the intermediate responses (from ICMP "time exceeded") stop coming back.

But I really don't think that your ISP would deny that they are doing
some sort of "intercept" of the DNS traffic (assuming that you can get
hold of someone there that even understands what you're talking about) -
"proving" (to their satisfaction) that they are doing it incorrectly may
be harder.

Quote:
Thanks for the patch. I'm not sure it'll help though - I'm almost sure
the other nameserver I tried from the ports (the name /still/ escapes
me, and the PC is turned off right now) explicitly does not support EDNS
- and that returns the same symptoms as BIND does. But I'll give it
another look.

Right, I had forgotten about that - and of course the fact that some
queries succeed indicates that EDNS0 can't really be fatal, since BIND
will do it on all queries by default. The thing that did point to EDNS0
was that 'dig' queries succeeded, but there is another difference
between BIND's and dig's queries, that may be the significant one given
what your trace showed: BIND (or any "real" name server) will not ask
for recursion, while dig will. You could try dig with +norecurse (this
should work when querying the authoritative servers, or a server that
happens to have the data in its cache, but not otherwise - works fine
with both the servers in this case from here, as expected).

Quote:
One 'evil' thought might be to contact the operators of the failing
domains and tell them (some) ntl users can't get through and why.....

Well, I don't know any specifics in your case, but my general impression
is that if getting an ISP to fix things is possible at all, it is only
by request from its paying customers (unless the brokenness is downright
harmful to the rest of the network). Inquiries from third parties would
probably at most result in them (and you) being told that "it works fine
when using the DNS servers we provide for our customers".

--Per Hedeland
per@hedeland.org
Back to top
jpd
*nix forums Guru


Joined: 22 Feb 2005
Posts: 877

PostPosted: Mon Jul 17, 2006 3:43 pm    Post subject: Re: major DNS hiccup Reply with quote

Begin <e9g4th$o7f$1@hedeland.org>
On 2006-07-17, Per Hedeland <per@hedeland.org> wrote:
Quote:
If all else fails, you could perhaps hack named to not use EDSN0 at all
- I didn't find any such setting,

IIRC from last time I looked that there is a global setting that regulates
the maximum value to ever advertise. Setting it to 512 might cause bind
to never include EDNS0 options, or at least reduce the probability that
packets get truncated.


--
j p d (at) d s b (dot) t u d e l f t (dot) n l .
This message was originally posted on Usenet in plain text.
Any other representation, additions, or changes do not have my
consent and may be a violation of international copyright law.
Back to top
Mike Scott
*nix forums Guru Wannabe


Joined: 25 Feb 2005
Posts: 138

PostPosted: Mon Jul 17, 2006 3:35 pm    Post subject: Re: major DNS hiccup Reply with quote

Thanks for the diagnosis. I've put some notes on the missing bits (only
missing because the trace seemed too long anyway) from the binary dump file.

Per Hedeland wrote:
Quote:
In article <8WJug.7561$i32.3378@newsfe2-gui.ntli.net> Mike Scott
usenet.10@spam.stopper.scottsonline.org.uk> writes:
OK, I've come back to this at last. Hoping it might have gone away as
suddenly as it arrived; but no such luck :-(

I ran 'dig' to look up the address of one of the always-failing names.
'dig' output plus ethereal diagnosis follow. named running as caching
nameserver on localhost.

"Interesting" stuff - IMHO it's quite clear that your ISP is messing
things up with some sort of "transparent" filter/proxy/cache/firewall -
it might be specifically b0rken where EDNS0 is concerned, while their
"normal" name servers can deal with that one way or another (as they
should) - but it's not really conclusive that this is the trigger.
Unfortunately your ethereal output was missing some parts, notably
details of the "additional records", but they can be guessed...

The 'additional record' in the query (sorry, should have been included)
was specifying the EDNS0 option, also had a DNSSEC bit set.

....
Quote:
But here comes the real brokenness - the reply to this query is
non-authoritative, has no error but no answers either, but has authority
records:

No. Time Source Destination
Protocol Info
6 11:32:48.355006 194.74.151.194 86.22.67.158 DNS
Standard query response


Flags: 0x8080 (Standard query response, No error)

Questions: 1
Answer RRs: 0
Authority RRs: 2

The authority records weren't shown, but a fair guess is that they were
the two "proper" NS records, one of which is for the very server
purportedly sending this response. This is the signature of a "lame
delegation", and named would normally log that (but it can be turned off
IIRC).

They named the two 'yell' nameservers, redgate and
redgate2.yellowpages.co.uk, as you surmise.

Odd about the logging - I currently only get minimal logging, using the
default logging configuration.

....
Quote:
Anyway, for me this clearly proves that your ISP is at fault - whether

I'm not sure whether that's good news or bad :-)

Quote:
it is proof enough for them I have no idea. They could e.g. claim that
your named sends some broken stuff while mine doesn't - but then there
was the other poster here that had success with the exact same setup
that failed with your ISP when he switched to another ISP.

Out of curiosity, I tried traceroute -p 53 to one of the root
nameservers. The hypothesis was that if ntl have put some sort of
transparent cache into place, this /ought/ not to reach the root server
- maybe! In fact, it did eventually, but only after some very long
delays; I'm not sure a one-off proves much anyway - perhaps some further
thought along those lines might give an indicator (eg craft a
traceroute-like thing that sends a genuine dns query, and sees where it
reaches. Maybe I could hack the traceroute sources).

Thanks for the patch. I'm not sure it'll help though - I'm almost sure
the other nameserver I tried from the ports (the name /still/ escapes
me, and the PC is turned off right now) explicitly does not support EDNS
- and that returns the same symptoms as BIND does. But I'll give it
another look.

One 'evil' thought might be to contact the operators of the failing
domains and tell them (some) ntl users can't get through and why.....

--
Please use the corrected version of the address below for replies.
Replies to the header address will be junked, as will mail from
various domains listed at www.scottsonline.org.uk
Mike Scott Harlow Essex England.(unet -a-t- scottsonline.org.uk)
Back to top
Google

Back to top
Display posts from previous:   
Post new topic   Reply to topic Page 1 of 4 [54 Posts] Goto page:  1, 2, 3, 4 Next
View previous topic :: View next topic
The time now is Sat Nov 22, 2008 8:12 am | All times are GMT
navigation Forum index » *nix » BSD » FreeBSD
Jump to:  

Similar Topics
Topic Author Forum Replies Last Post
No new posts MAJOR RANT: That DK(IM)? crap Ralf Hildebrandt Postfix 4 Thu Jul 06, 2006 11:57 am
No new posts what is the next major release for IA after B.11.23? Bujji HP-UX 2 Tue Jun 20, 2006 6:48 pm
No new posts Flight performance hiccup (decnet ?) JF Mezei VMS 4 Thu May 18, 2006 8:33 am
No new posts SUMMARY: doconfig fails with "Major number 55 requested i... Roberto Mackun Tru64 managers mail-list 0 Tue May 16, 2006 1:42 am
No new posts doconfig fails with "Major number 55 requested in stanza.... Roberto Mackun Tru64 managers mail-list 0 Mon May 15, 2006 12:49 pm

Budapest | Magazine Subscriptions | MySpace Layouts | Credit Card | Internet Advertising
Copyright © 2004-2005 DeniX Solutions SRL
 
Other DeniX Solutions sites: Unix/Linux blog |  electronics forum |  medicine forum |  science forum | 
Privacy Policy


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.4665s ][ Queries: 16 (0.2455s) ][ GZIP on - Debug on ]