|
|
|
|
|
|
| Author |
Message |
Kyle Wheeler
*nix forums Guru Wannabe
Joined: 07 Jan 2005
Posts: 208
|
 Posted: Wed Jul 12, 2006 10:01 pm Post subject:
Re: Binding qmail-remote to multiple IP addresses
|
|
|
On Wednesday, July 12 at 01:26 PM, quoth Paul Theodoropoulos:
| Quote: | what's really screwed is that - in the day and age of commodity,
cheap, massive, bandwidth, and commodity, cheap, powerful, hardware
- that this is an issue at all.
ATTworldnet won't accept mail directly from my servers. they deny
that we're blacklisted. their 'help' page for this particular
connection rejection makes noises about DoS attacks and throttling
people temporarily who make too many connections.
|
Amen to that. Anyone that thinks 20 concurrent connections is a DoS
attack has their threshold too low (don’t even get me started on
people who set their threshold to 1).
In my experience, it seems to be a Barracuda thing: every domain I’ve
had problems with has been running a Barracuda spam-firewall.
~Kyle
--
Time is an illusion. Lunchtime doubly so.
-- Douglas Adams |
|
| Back to top |
|
 |
Paul Theodoropoulos
*nix forums beginner
Joined: 19 Jul 2005
Posts: 17
|
| Posted: Wed Jul 12, 2006 8:26 pm Post subject:
Re: Binding qmail-remote to multiple IP addresses
|
|
|
At 03:32 AM 7/12/2006, Reuven M. Lerner wrote:
| Quote: | My impression is that qmail might indeed be faster and/or more
efficient in a world where ISPs accept unlimited connections, or in
which members of a mailing list are spread across many different
ISPs. But on a list where members are concentrated in a small
number of ISPs, and when those ISPs limit the number of connections,
the Postfix approach seems to be better.
|
Metacomment:
what's really screwed is that - in the day and age of commodity,
cheap, massive, bandwidth, and commodity, cheap, powerful, hardware -
that this is an issue at all.
ATTworldnet won't accept mail directly from my servers. they deny
that we're blacklisted. their 'help' page for this particular
connection rejection makes noises about DoS attacks and throttling
people temporarily who make too many connections. before i gave up on
the issue - after more than a dozen messages back and forth with
truly brick-like support people - my customers were sending fewer
than 150 messages *per week* to ATTworldnet addresses.
I set up a dinky qmail server on my brother's static-IP dsl
connection, and use smtproutes to push the mail there, and from there
it goes to ATTworldnet - just fine. have been doing this for three
months now. turned it off briefly a few weeks ago - still blocked.
pathetic. okay, i'm done ranting.
Paul Theodoropoulos
http://www.anastrophe.com |
|
| Back to top |
|
|
Dave Sill
*nix forums Guru Wannabe
Joined: 09 May 2002
Posts: 235
|
| Posted: Wed Jul 12, 2006 5:24 pm Post subject:
Re: Binding qmail-remote to multiple IP addresses
|
|
|
Amitai Schlair <schmonz@schmonz.com> wrote:
| Quote: | Reuven M. Lerner wrote:
Several people also suggested delivering mail to a local Maildir instead
of the problematic domains, and then using serialmail to deliver mail
from those directories on a regular basis. Maybe it's just me, but this
sounded like it would be hard to do.
If Postfix does what you want (and it sounds like it does) and you're
willing to administer it, then sure, it's a solution to the problem. But
you sure have a funny definition of "hard to do." 
|
Exactly. Setting up serialmail to do the job would have been "hard" in
some sense, but it certainly would have been easier than uninstalling
qmail and installing/learning/configuring Postfix. Not that there's
anything wrong with Postfix.
-Dave |
|
| Back to top |
|
|
Amitai Schlair
*nix forums beginner
Joined: 03 Mar 2005
Posts: 14
|
| Posted: Wed Jul 12, 2006 3:23 pm Post subject:
Re: Binding qmail-remote to multiple IP addresses
|
|
|
Reuven M. Lerner wrote:
| Quote: | Several people also suggested delivering mail to a local Maildir instead
of the problematic domains, and then using serialmail to deliver mail
from those directories on a regular basis. Maybe it's just me, but this
sounded like it would be hard to do.
[...]
This got me thinking: If the problem is the number of SMTP connections
that I'm making, rather than the number of messages, and if the most
popular ISPs are the ones that are giving me trouble, then perhaps I
would be wise to switch to Postfix.
|
If Postfix does what you want (and it sounds like it does) and you're
willing to administer it, then sure, it's a solution to the problem. But
you sure have a funny definition of "hard to do." :-)
- Amitai |
|
| Back to top |
|
|
Reuven M. Lerner
*nix forums beginner
Joined: 05 Jul 2006
Posts: 2
|
| Posted: Wed Jul 12, 2006 10:32 am Post subject:
Re: Binding qmail-remote to multiple IP addresses
|
|
|
Hi, everyone. I e-mailed this list about a week ago, asking for help
with my mail server, which was unable to delivery mail to many list
subscribers because the ISPs were limiting the number of connections I
could make per minute. Since qmail puts each message into its own SMTP
connection, and at two of my lists had 50 messages/day and 1,000
subscribers, I was hitting a wall.
I received a lot of good suggestions from this list. In particular,
Jeremy Eder wrote:
| Quote: | Sadly, most everyone with a busy server has run into this.
I've gone the 'contact recipient isp, beg for whitelisting' thing, and
got nowhere.
Take a look at the url and posts below, we had a talk about this a
while back.
I haven't had a chance to test Richard's code, but if you do please
let us know as I believe it will benefit all.
http://marc.theaimsgroup.com/?l=qmail&w=2&r=1&s=control+delivery+rate&q=b
I tried the code posted in the above thread, and it seemed to make |
things worse; my CPU usage went through the roof, and delivery success
became more variable -- sometimes working well, but often getting stuck
even more than before. I fiddled with the configuration quite a bit,
but nothing changed the feeling that I got of doing a busy wait, with
qmail-remote trying (and failing) repeatedly to lock the
rate-controlling file.
Several people also suggested delivering mail to a local Maildir instead
of the problematic domains, and then using serialmail to deliver mail
from those directories on a regular basis. Maybe it's just me, but this
sounded like it would be hard to do.
I also received some advice on how I could use netfilter to send from
several IP addresses. Unfortunately, my hosting provider doesn't offer
the netfilter modules that I would need.
Establishing a second mail server would normally be a good idea, I
think, except that this is all supposed to be a fun, volunteer thing,
and I didn't want to spend money on another server.
Erwin Hoffmann wrote:
| Quote: | In fact, this is based on qmail's send-one-message-then-quit strategy for
sending emails. Postfix, for instances, bundles all requist for a
receipient domain and sends them in one go.
This got me thinking: If the problem is the number of SMTP connections |
that I'm making, rather than the number of messages, and if the most
popular ISPs are the ones that are giving me trouble, then perhaps I
would be wise to switch to Postfix. So I decided to give it a shot,
keeping my qmail configuration intact -- and I must say that the results
are overwhelming and dramatic. Since installing Postfix yesterday, I
have literally no messages waiting in the queue, despite the same
traffic patterns as I had using qmail. Memory usage is way down, CPU
usage is way down, and people are getting their messages within
seconds. Configuration and installation was pretty quick and easy, as
well; I went from knowing nothing about Postfix to having it working
within about 36 hours, with the actual time invested being much less
than that.
My impression is that qmail might indeed be faster and/or more efficient
in a world where ISPs accept unlimited connections, or in which members
of a mailing list are spread across many different ISPs. But on a list
where members are concentrated in a small number of ISPs, and when those
ISPs limit the number of connections, the Postfix approach seems to be
better.
Thanks to those who responded for being so helpful!
Reuven |
|
| Back to top |
|
|
Erwin Hoffmann
*nix forums addict
Joined: 24 Jan 2005
Posts: 71
|
| Posted: Wed Jul 05, 2006 6:41 pm Post subject:
Re: Binding qmail-remote to multiple IP addresses
|
|
|
Hi,
At 11:52 05.07.2006 -0500, you wrote:
| Quote: | Hi, everyone. I've been using qmail (unpatched 1.03) as my MTA for a
number of years on a virtual server running an old version of Red Hat
Linux. Among other things, my server hosts two medium-sized
(1,000-subscriber, 50 postings/day) mailing lists, managed by Mailman.
About two months ago, I learned that some subscribers to my mailing
lists were having their mail delayed or dropped. Upon further
investigation, I discovered that this was only true for three ISPs --
Yahoo Mail, NetVision, and 012.net.il, the latter two of which are
popular Israeli ISPs. (About 10 percent of the subscribers to each list
use NetVision, with about half that on Yahoo and 012.) The problem, it
would seem, is that I am trying to send too much e-mail to these ISPs.
My server is mistaken for a spammer, or perhaps a denial-of-service
attack, and all but the first dozen or so messages are rejected.
|
Yes. The same happens in Germany for the T-Online users:
T-Online MTAs reject incoming SMTP sessions in case they have received from
the very same SMTP sending IP over a certain period too many connections.
In fact, this is based on qmail's send-one-message-then-quit strategy for
sending emails. Postfix, for instances, bundles all requist for a
receipient domain and sends them in one go.
There have been serveral debats on this issues regarding use of bandwith;
however the behavoir of T-Online MTAs -- though it is understandable in
terms of Spam rejection -- is simply a matter of 'policy' and not covered
positively by any RFC.
Instead of requireing different IP addresses to send mail from by
qmail-remote another solution would be to contact in case of a failure a
second MX instead of trying to establish sessions only to the primary
(which in case of identical MX record weights don't buy).
DJB has had the same idea (read THOUGHTS in vanilla qmail) and discussed
that in terms of contacting the secondary MTAS after a failure of the
primary one; however -- at that time -- seems to be not necessary.
In SPAMCONTROL 2.4 I re-used Dan's code (which actually was pointed out in
the past by Matthias Andree).
Changes to qmail-remote are minor and are not harmful in any case.
@Charles: I guess this should be included into netqmail-1.06.
Here is a snipset of the relevant part of qmail-remote.c:
void smtp()
{
unsigned long code;
int flagbother;
int i;
code = smtpcode();
if (code >= 400) return; /* try next MX */
if (code != 220) quit("ZConnected to "," but greeting failed");
substdio_puts(&smtpto,"EHLO ");
substdio_put(&smtpto,helohost.s,helohost.len);
substdio_puts(&smtpto,"\r\n");
substdio_flush(&smtpto);
if (smtpcode() != 250) {
substdio_puts(&smtpto,"HELO ");
substdio_put(&smtpto,helohost.s,helohost.len);
substdio_puts(&smtpto,"\r\n");
substdio_flush(&smtpto);
code = smtpcode();
authsender = 0;
if (code >= 500) quit("DConnected to "," but my name was rejected");
if (code != 250) quit("ZConnected to "," but my name was rejected");
}
/* if (authsender)
smtp_auth();
else */
mailfrom();
code = smtpcode();
if (code >= 500) quit("DConnected to "," but sender was rejected");
if (code >= 400) quit("ZConnected to "," but sender was rejected");
flagbother = 0;
for (i = 0;i < reciplist.len;++i) {
substdio_puts(&smtpto,"RCPT TO:<");
substdio_put(&smtpto,reciplist.sa[i].s,reciplist.sa[i].len);
substdio_puts(&smtpto,">\r\n");
substdio_flush(&smtpto);
code = smtpcode();
if (code >= 500) {
out("h"); outhost(); out(" does not like recipient.\n");
outsmtptext(); zero();
}
else if (code >= 400) {
out("s"); outhost(); out(" does not like recipient.\n");
outsmtptext(); zero();
}
else {
out("r"); zero();
flagbother = 1;
}
}
if (!flagbother) quit("DGiving up on ","");
substdio_putsflush(&smtpto,"DATA\r\n");
code = smtpcode();
if (code >= 500) quit("D"," failed on DATA command");
if (code >= 400) quit("Z"," failed on DATA command");
blast();
code = smtpcode();
flagcritical = 0;
if (code >= 500) quit("D"," failed after I sent the message");
if (code >= 400) quit("Z"," failed after I sent the message");
quit("K"," accepted message");
}
Dr. Erwin Hoffmann | FEHCom | http://www.fehcom.de/
Wiener Weg 8, 50858 Cologne | T: +49 221 484 4923 | F: ...24 |
|
| Back to top |
|
|
Jeremy Eder
*nix forums beginner
Joined: 11 Jan 2005
Posts: 15
|
| Posted: Wed Jul 05, 2006 5:39 pm Post subject:
Re: Binding qmail-remote to multiple IP addresses
|
|
|
Charles Cazabon wrote:
| Quote: | Reuven M. Lerner <reuven@lerner.co.il> wrote:
I have contacted the Israelis ISPs, and got nowhere. One argues that my
MTA shouldn't be trying to deliver more than 12 messages per minute --
yes, per minute! -- from the same originating IP address.
Ouch. And if you have tens of thousands of users, more than 12 of which might
want to send mail to them at a given time?
My hosting provider has said that I'm entitled to additional IP
addresses. What I would like to do, in lieu of any better solution, is
bind qmail-remote to a randomly selected IP address from a list
Better solution would be to set up those three domains as virtual domains
delivering to a local maildir, then use serialmail to periodically deliver
those maildirs to the servers in question. You end up with only one
connection at a time, so it shouldn't trigger their bogus filters on incoming
connections.
Charles
|
Sadly, most everyone with a busy server has run into this.
I've gone the 'contact recipient isp, beg for whitelisting' thing, and
got nowhere.
Take a look at the url and posts below, we had a talk about this a while
back.
I haven't had a chance to test Richard's code, but if you do please let
us know as I believe it will benefit all.
http://marc.theaimsgroup.com/?l=qmail&w=2&r=1&s=control+delivery+rate&q=b
--
Jeremy Eder
UNIX Administrator
INVISION.COM
631.543.1000 x334 |
|
| Back to top |
|
|
Fabio Busatto
*nix forums beginner
Joined: 02 Feb 2006
Posts: 24
|
| Posted: Wed Jul 05, 2006 5:34 pm Post subject:
Re: Binding qmail-remote to multiple IP addresses
|
|
|
On Wed, Jul 05, 2006 at 11:52:04AM -0500, Reuven M. Lerner wrote:
| Quote: | My hosting provider has said that I'm entitled to additional IP
addresses. What I would like to do, in lieu of any better solution, is
bind qmail-remote to a randomly selected IP address from a list that I
will specify, presumably in a control file. This will presumably allow
me to get around the ISPs' restrictions, enabling me to deliver the mail
in a timely manner. I'm wondering if there is a patch that already does
this; I didn't see any, but I might well be missing something. And of
course, if there's a better solution than what I'm suggesting, I would
be delighted to hear about it.
|
If you use linux and netfilter, you can use iptables to do that:
if a connection matches specific criteria (destination ip - destination port),
do a source nat using a simple round robin algo to compute the source ip.
Something like this:
iptables -t nat -A POSTROUTING -d mx.lazydomain.tld -p tcp --dport smtp -j SNAT --to-source ip1 --to-source ip3-ip4
Bye
-fabio
--
Fabio Busatto <fabio.busatto@sikurezza.org> |
|
| Back to top |
|
|
Charles Cazabon
*nix forums Guru
Joined: 08 Jan 2005
Posts: 805
|
| Posted: Wed Jul 05, 2006 5:21 pm Post subject:
Re: Binding qmail-remote to multiple IP addresses
|
|
|
Reuven M. Lerner <reuven@lerner.co.il> wrote:
| Quote: |
I have contacted the Israelis ISPs, and got nowhere. One argues that my
MTA shouldn't be trying to deliver more than 12 messages per minute --
yes, per minute! -- from the same originating IP address.
|
Ouch. And if you have tens of thousands of users, more than 12 of which might
want to send mail to them at a given time?
| Quote: | My hosting provider has said that I'm entitled to additional IP
addresses. What I would like to do, in lieu of any better solution, is
bind qmail-remote to a randomly selected IP address from a list
|
Better solution would be to set up those three domains as virtual domains
delivering to a local maildir, then use serialmail to periodically deliver
those maildirs to the servers in question. You end up with only one
connection at a time, so it shouldn't trigger their bogus filters on incoming
connections.
Charles
--
--------------------------------------------------------------------------
Charles Cazabon <qmail@discworld.dyndns.org>
Read http://pyropus.ca/personal/writings/12-steps-to-qmail-list-bliss.html
My services include qmail consulting. See http://pyropus.ca/ for details.
-------------------------------------------------------------------------- |
|
| Back to top |
|
|
Reuven M. Lerner
*nix forums beginner
Joined: 05 Jul 2006
Posts: 2
|
| Posted: Wed Jul 05, 2006 4:52 pm Post subject:
Binding qmail-remote to multiple IP addresses
|
|
|
Hi, everyone. I've been using qmail (unpatched 1.03) as my MTA for a
number of years on a virtual server running an old version of Red Hat
Linux. Among other things, my server hosts two medium-sized
(1,000-subscriber, 50 postings/day) mailing lists, managed by Mailman.
About two months ago, I learned that some subscribers to my mailing
lists were having their mail delayed or dropped. Upon further
investigation, I discovered that this was only true for three ISPs --
Yahoo Mail, NetVision, and 012.net.il, the latter two of which are
popular Israeli ISPs. (About 10 percent of the subscribers to each list
use NetVision, with about half that on Yahoo and 012.) The problem, it
would seem, is that I am trying to send too much e-mail to these ISPs.
My server is mistaken for a spammer, or perhaps a denial-of-service
attack, and all but the first dozen or so messages are rejected.
The volume of mail to these lists is such that it doesn't help for qmail
to retry according to its normal schedule; messages continue to pile up,
until they are simply discarded.
I have contacted the Israelis ISPs, and got nowhere. One argues that my
MTA shouldn't be trying to deliver more than 12 messages per minute --
yes, per minute! -- from the same originating IP address. The other
claims that there aren't any restrictions on sending e-mail to their
servers, even though I have clear evidence that they are now refusing to
accept SMTP connections from my IP address for more than a token number
of messages. Both claim that the problem is squarely with my server,
even though I only appear guilty of trying to deliver legitimate
messages to list subscribers.
I have tried a number of techniques to get around this problem, the most
successful of which involved sending SIGALRM to qmail-send every three
minutes or so, via cron. (Yes, I realize that this is an ugly,
brute-force approach.) However, after a week or so of success with this
technique, the ISPs have once again begun blocking my outgoing mail.
This cron-resend technique did work with Yahoo, for what it's worth.
There are no obvious problems with sending e-mail to any other ISPs or
hosts.
I have tried to adjust a number of control files, in the hopes that this
would improve things. I have changed concurrencyremote to its default
value of 20, but that proved to be too slow, especially with so many
processes timing out to 012 and NetVision. I changed timeoutremote and
timeoutconnect to very small values (e.g., 20 or 30), and that did seem
to improve things somewhat, but not enough to get the mail delivered. I
have adjusted the frequency with which I send SIGALRM to qmail-send, to
no avail.
My hosting provider has said that I'm entitled to additional IP
addresses. What I would like to do, in lieu of any better solution, is
bind qmail-remote to a randomly selected IP address from a list that I
will specify, presumably in a control file. This will presumably allow
me to get around the ISPs' restrictions, enabling me to deliver the mail
in a timely manner. I'm wondering if there is a patch that already does
this; I didn't see any, but I might well be missing something. And of
course, if there's a better solution than what I'm suggesting, I would
be delighted to hear about it.
Thanks for any and all help you can offer!
Reuven |
|
| Back to top |
|
|
Google
|
|
| Back to top |
|
|
|
|
The time now is Sat Nov 22, 2008 8:38 am | All times are GMT
|
|
Debt | Remortgages | Credit Cards | Dutch Bodybuilding Forums | Remortgages
|
|
Copyright © 2004-2005 DeniX Solutions SRL
|
|
|
|
Other DeniX Solutions sites:
Unix/Linux blog |
electronics forum |
medicine forum |
science forum |
|
|
Privacy Policy
|
Powered by phpBB © 2001, 2005 phpBB Group
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
Forum index
»
Apps
»
Qmail
