|
|
|
|
|
|
| Author |
Message |
Xin LI
*nix forums beginner
Joined: 09 Nov 2004
Posts: 44
|
 Posted: Thu Jul 20, 2006 5:21 pm Post subject:
Proposal: Allow dict_tcp server to disconnect without causing smtpd to say Server configuration error [PATCH]
|
|
|
Hi,
Currently, postfix dict_tcp implementation reuses connection if the
query returns 200 or 500. However, sometimes the other end (say, the
tcp map server) may have some reasons to shutdown the connection; for
instance, it may have an idle timeout which can reclaim threads that
is no longer necessary for the near future, etc.
We have a tcp map server implementation which does timeout and
shutdown the connection after 10 minutes. After some tcpdump
investigation for recently raising "Server configuration error" and
"Server configuration problem" we have found postfix's processing
logic is roughly like this:
+ BEGIN
++ (*) If we have cached connection descriptor or a new connection is
successfully established, then use the connection.
++ Send the request, and read response
++ If the send request failed, or connection is not available (yet),
then loop until either we got a connection or retries is exhausted.
+ LOOP
+ Give response to caller, if it passed basic sanity check.
Note that because the sanity check is done after the loop and
postfix's smtpd does not distinguish between server generated 400
error and tcp connection problem when querying a TCP map, and
considers that as a server configuration problem, there are chances
where the dropped connection (still a valid fd at smtpd side) would
trigger this situation.
The proposed patch moves the check into the loop and does some retry
attempts before actually saying "Bang! Fail!" which is potentially
useful for others, so we share it here and hopes that the idea can be
adopted or be implemented in other way. The patch is done against
postfix 2.3.0 release, but should apply cleanly on 2.4.x development
snapshot as well.
Cheers,
--
Xin LI <delphij@delphij.net> http://www.delphij.net |
|
| Back to top |
|
 |
Wietse Venema
*nix forums Guru
Joined: 27 Feb 2005
Posts: 2697
|
| Posted: Thu Jul 20, 2006 5:58 pm Post subject:
Re: Proposal: Allow dict_tcp server to disconnect without causing smtpd to say Server configuration error [PATCH]
|
|
|
Xin LI:
| Quote: | Hi,
Currently, postfix dict_tcp implementation reuses connection if the
query returns 200 or 500. However, sometimes the other end (say, the
tcp map server) may have some reasons to shutdown the connection; for
instance, it may have an idle timeout which can reclaim threads that
is no longer necessary for the near future, etc.
We have a tcp map server implementation which does timeout and
shutdown the connection after 10 minutes. After some tcpdump
investigation for recently raising "Server configuration error" and
"Server configuration problem" we have found postfix's processing
logic is roughly like this:
+ BEGIN
++ (*) If we have cached connection descriptor or a new connection is
successfully established, then use the connection.
++ Send the request, and read response
++ If the send request failed, or connection is not available (yet),
then loop until either we got a connection or retries is exhausted.
+ LOOP
+ Give response to caller, if it passed basic sanity check.
|
Hmm. I have one question. Are we solving the right problem?
It looks like you are trying to ignore malformed server replies.
The sanity check, as implemented today, happens only after the
server has sent newline-terminated text.
If I understand your patch correctly, your change makes a difference
only when your server sometimes sends text that ends in newline
but that does not have the required format of (three digits, space,
encoded text).
Instead of forcing Postfix to ignore malformed server replies,
perhaps the effort would be better spent fixing the server so
that it does not send malformed replies in the first place.
Wietse |
|
| Back to top |
|
|
Xin LI
*nix forums beginner
Joined: 09 Nov 2004
Posts: 44
|
| Posted: Fri Jul 21, 2006 1:48 am Post subject:
Re: Proposal: Allow dict_tcp server to disconnect without causing smtpd to say Server configuration error [PATCH]
|
|
|
Hi, Wietse,
On 7/21/06, Wietse Venema <wietse@porcupine.org> wrote:
[...]
| Quote: | Hmm. I have one question. Are we solving the right problem?
It looks like you are trying to ignore malformed server replies.
The sanity check, as implemented today, happens only after the
server has sent newline-terminated text.
If I understand your patch correctly, your change makes a difference
only when your server sometimes sends text that ends in newline
but that does not have the required format of (three digits, space,
encoded text).
Instead of forcing Postfix to ignore malformed server replies,
perhaps the effort would be better spent fixing the server so
that it does not send malformed replies in the first place.
|
Well, not only insane responses, but also 400 temporarily problem can
cause postfix to give "Server configuration problem". Yes, there was
a bug in our first implementation of the tcp table server, which did
the following (C=smtpd, S=tcp map server)
C> get foo
S> 200 [the response]
(... C did not contact with S for a long while, e.g. 600 seconds)
S> 400 Timeout <---- (*) This is not correct, and S closes the TCP
socket afterhand.
(... some time later)
C> get foo
(C gets a RST)
I think in the last step, C gets "400 Timeout", and it in turn tell
the remote smtp client "Server configuration problem".
We have addressed the case marked as (*), but only having that does
not make changes where the following happens:
C> get foo
S> 400 [Connection to database backend problem, try again later]
C will give remote client "Server configuration problem" in this case as well.
Because the tcp_table(5) says that 400 "indicates an error
condition.", and 'the client should retry the request later.", I am
not very sure if providing "400 connection problem" here is a misuse.
If it is not then I think the retry mechainsm should be implemented in
Postfix.
Cheers,
--
Xin LI <delphij@delphij.net> http://www.delphij.net |
|
| Back to top |
|
|
Wietse Venema
*nix forums Guru
Joined: 27 Feb 2005
Posts: 2697
|
| Posted: Fri Jul 21, 2006 3:21 am Post subject:
Re: Proposal: Allow dict_tcp server to disconnect without causing smtpd to say Server configuration error [PATCH]
|
|
|
Indeed, Postfix does not distinguish between:
- unable to query the dict_tcp server
- 4xx reply from the dict_tcp server
In other words, the dict_tcp server must reply with 4xx only if it
is unable to answer a query. In all other cases it replies with
2xx (the requested item was found) or 5xx (the requested item does
not exist).
Wietse |
|
| Back to top |
|
|
Xin LI
*nix forums beginner
Joined: 09 Nov 2004
Posts: 44
|
| Posted: Fri Jul 21, 2006 6:34 am Post subject:
Re: Proposal: Allow dict_tcp server to disconnect without causing smtpd to say Server configuration error [PATCH]
|
|
|
Hi,
On 7/21/06, Wietse Venema <wietse@porcupine.org> wrote:
| Quote: | Indeed, Postfix does not distinguish between:
- unable to query the dict_tcp server
- 4xx reply from the dict_tcp server
In other words, the dict_tcp server must reply with 4xx only if it
is unable to answer a query. In all other cases it replies with
2xx (the requested item was found) or 5xx (the requested item does
not exist).
|
Indeed.
So what we feel useful is that the dict_tcp client (or its caller)
could make limited retries if it has encounted "unable to query" or
"4xx"s. Or, perhaps this would be suitible to warrant as a main.cf
option (dict_tcp_retry)?
Cheers,
--
Xin LI <delphij@delphij.net> http://www.delphij.net |
|
| Back to top |
|
|
Xin LI
*nix forums beginner
Joined: 09 Nov 2004
Posts: 44
|
| Posted: Fri Jul 21, 2006 7:34 am Post subject:
Re: Proposal: Allow dict_tcp server to disconnect without causing smtpd to say Server configuration error [PATCH]
|
|
|
On 7/21/06, Wietse Venema <wietse@porcupine.org> wrote:
| Quote: | Indeed, Postfix does not distinguish between:
- unable to query the dict_tcp server
- 4xx reply from the dict_tcp server
In other words, the dict_tcp server must reply with 4xx only if it
is unable to answer a query. In all other cases it replies with
2xx (the requested item was found) or 5xx (the requested item does
not exist).
|
I have re-read the code and the above text. Perhaps I can conclude
the tcp map behavior as:
When a "get" request is received, server should:
- 200 result, in case something is found.
- 400 reason, in case that there is a problem which is really unrecoverable.
- 500 reason, in case that the result is not found.
*and*
- disconnect immediately, if we want Postfix to retry 1 second later
until the count is exhausted?
If that's the case I think we may want to make some changes to
tcp_table(5), though... I would be happy do the work if you wish :-)
Cheers,
--
Xin LI <delphij@delphij.net> http://www.delphij.net |
|
| Back to top |
|
|
Wietse Venema
*nix forums Guru
Joined: 27 Feb 2005
Posts: 2697
|
| Posted: Fri Jul 21, 2006 2:13 pm Post subject:
Re: Proposal: Allow dict_tcp server to disconnect without causing smtpd to say Server configuration error [PATCH]
|
|
|
Xin LI:
[ Charset ISO-8859-1 unsupported, converting... ]
| Quote: | On 7/21/06, Wietse Venema <wietse@porcupine.org> wrote:
Indeed, Postfix does not distinguish between:
- unable to query the dict_tcp server
- 4xx reply from the dict_tcp server
In other words, the dict_tcp server must reply with 4xx only if it
is unable to answer a query. In all other cases it replies with
2xx (the requested item was found) or 5xx (the requested item does
not exist).
I have re-read the code and the above text. Perhaps I can conclude
the tcp map behavior as:
When a "get" request is received, server should:
- 200 result, in case something is found.
- 400 reason, in case that there is a problem which is really unrecoverable.
|
No. 4xx is for "Cannot answer the query, NOW, try again LATER".
Use 5xx when the reply does not exist, or when the request
should not be retried.
Wietse |
|
| Back to top |
|
|
Google
|
|
| Back to top |
|
|
|
|
The time now is Sat Nov 22, 2008 8:36 pm | All times are GMT
|
|
Loans | Mobile Phones | Myspace Layouts | Loans | Mobile Phones
|
|
Copyright © 2004-2005 DeniX Solutions SRL
|
|
|
|
Other DeniX Solutions sites:
Unix/Linux blog |
electronics forum |
medicine forum |
science forum |
|
|
Privacy Policy
|
Powered by phpBB © 2001, 2005 phpBB Group
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
Forum index
»
Apps
»
Postfix
