|
|
|
|
|
|
| Author |
Message |
Marc Perkel *nix forums Guru
Joined: 27 Jan 2005
Posts: 502
|
Posted: Thu Jul 20, 2006 11:21 pm Post subject:
Need a little unix help
|
|
|
Some of you I'm srue are experts at this. I can probably get something
to work but I want to do it right. And it's fairly simple.
What I want to do is create a simple client/server application where
clients running Exim can easily report IP addresses to the server. The
server will accept the IP addresses and execute a command to add the IP
address to a MySQL database and increment a counter. What I'm
specifically doing is counting spam/ham based on IP address. But - if I
can just receive the IP address I'll figure put how to process it.
I have xinentd if that maes things easy.
On the client side I want to in an efficient way be able to pass a
string containing the IP address to the server. Maybe talk to an open port?
So - what's the simplest way to accomplish this?
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/ |
|
| Back to top |
|
 |
Chris Lightfoot *nix forums addict
Joined: 22 Mar 2005
Posts: 62
|
Posted: Thu Jul 20, 2006 11:31 pm Post subject:
Re: Need a little unix help
|
|
|
On Thu, Jul 20, 2006 at 04:21:03PM -0700, Marc Perkel wrote:
| Quote: | Some of you I'm srue are experts at this. I can probably get something
to work but I want to do it right. And it's fairly simple.
What I want to do is create a simple client/server application where
clients running Exim can easily report IP addresses to the server. The
server will accept the IP addresses and execute a command to add the IP
address to a MySQL database and increment a counter. What I'm
specifically doing is counting spam/ham based on IP address. But - if I
can just receive the IP address I'll figure put how to process it.
I have xinentd if that maes things easy.
On the client side I want to in an efficient way be able to pass a
string containing the IP address to the server. Maybe talk to an open port?
So - what's the simplest way to accomplish this?
|
I'd be tempted to use HTTP perl -- put in a perl startup
file containing something like,
use LWP::Simple;
use Digest::HMAC_SHA1 qw(hmac_sha1_hex);
my $url = 'http://somewhere/something';
my $secret = 'some secret value shared with the server';
sub transmit_ip_to_server ($) {
my $ip = shift;
get("$url?ip=$ip;mac=" . hmac_sha1_hex($ip, $secret));
}
and write the corresponding CGI script to check the MAC
and accept the submissions. HTTP purists will note I
should be using POST not GET and crypto purists would
probably want some kind of random padding in the protocol,
but you get the idea. Then use something like
${perl{transmit_ip_to_server}{$address}}
in an ACL to send the information.
--
``I had one [good seventh grade teacher]. Smart, witty, excellent
communicator, young, and beautiful. Naturally, she fell over
dead in class from a brain haemorrhage.'' (seen on the internet)
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/ |
|
| Back to top |
|
 |
Tony Finch *nix forums Guru
Joined: 22 Mar 2002
Posts: 1222
|
Posted: Thu Jul 20, 2006 11:45 pm Post subject:
Re: Need a little unix help
|
|
|
On Fri, 21 Jul 2006, Chris Lightfoot wrote:
| Quote: | On Thu, Jul 20, 2006 at 04:21:03PM -0700, Marc Perkel wrote:
What I want to do is create a simple client/server application where
clients running Exim can easily report IP addresses to the server.
So - what's the simplest way to accomplish this?
I'd be tempted to use HTTP perl
|
I'd use syslog :-)
Tony.
--
<fanf@exim.org> <dot@dotat.at> http://dotat.at/ ${sg{\N${sg{\
N\}{([^N]*)(.)(.)(.*)}{\$1\$3\$2\$1\$3\n\$2\$3\$4\$3\n\$3\$2\$4}}\
\N}{([^N]*)(.)(.)(.*)}{\$1\$3\$2\$1\$3\n\$2\$3\$4\$3\n\$3\$2\$4}}
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/ |
|
| Back to top |
|
 |
Marc Perkel *nix forums Guru
Joined: 27 Jan 2005
Posts: 502
|
Posted: Thu Jul 20, 2006 11:51 pm Post subject:
Re: Need a little unix help
|
|
|
Tony Finch wrote:
| Quote: | On Fri, 21 Jul 2006, Chris Lightfoot wrote:
On Thu, Jul 20, 2006 at 04:21:03PM -0700, Marc Perkel wrote:
What I want to do is create a simple client/server application where
clients running Exim can easily report IP addresses to the server.
So - what's the simplest way to accomplish this?
I'd be tempted to use HTTP perl
I'd use syslog :-)
Tony.
|
How do you do that? :)
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/ |
|
| Back to top |
|
 |
Bill Hacker *nix forums Guru
Joined: 27 Jan 2005
Posts: 427
|
Posted: Fri Jul 21, 2006 1:19 am Post subject:
Re: Need a little unix help
|
|
|
Marc Perkel wrote:
| Quote: | Some of you I'm srue are experts at this. I can probably get something
to work but I want to do it right. And it's fairly simple.
What I want to do is create a simple client/server application where
clients running Exim can easily report IP addresses to the server.
The server will accept the IP addresses and execute a command to add the IP
address to a MySQL database and increment a counter. What I'm
specifically doing is counting spam/ham based on IP address. But - if I
can just receive the IP address I'll figure put how to process it.
I have xinentd if that maes things easy.
On the client side I want to in an efficient way be able to pass a
string containing the IP address to the server. Maybe talk to an open port?
So - what's the simplest way to accomplish this?
|
It seems you would want near-real-time posting of those IP.
Since you said that your 'target' is a MySQL DB, you might want to simply use
SQL to write them directly into the DB from within an acl.
There are examples available in SQL greylisting code - not cut-and-paste usable,
but which you should be able to readily adapt to your needs.
An Exim MACRO from Toleff Fog Heen is illustrative of the 'INSERT' code concept:
GRAYLIST_ADD = INSERT INTO relaytofrom (relay_ip, from_domain) \
VALUES ( '${quote_mysql:$sender_host_address}', \
'${quote_mysql:$sender_address_domain}')
One might add timestamps and an autoincrementing 'serial' field, then a trigger
on the count if MySQL has triggers Else PostgreSQL sells for the same price, and
does have.
Note that 'read' privileges are not enough, so make sure that any other SQL code
is resistant to exploits that could modify the DB improperly.
I suspect this would run faster if you used something simpler than an SQL DB.
HTH,
Bill
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/ |
|
| Back to top |
|
 |
Marc Perkel *nix forums Guru
Joined: 27 Jan 2005
Posts: 502
|
Posted: Fri Jul 21, 2006 2:10 am Post subject:
Re: Need a little unix help
|
|
|
W B Hacker wrote:
| Quote: | Marc Perkel wrote:
Some of you I'm srue are experts at this. I can probably get something
to work but I want to do it right. And it's fairly simple.
What I want to do is create a simple client/server application where
clients running Exim can easily report IP addresses to the server.
The server will accept the IP addresses and execute a command to add the IP
address to a MySQL database and increment a counter. What I'm
specifically doing is counting spam/ham based on IP address. But - if I
can just receive the IP address I'll figure put how to process it.
I have xinentd if that maes things easy.
On the client side I want to in an efficient way be able to pass a
string containing the IP address to the server. Maybe talk to an open port?
So - what's the simplest way to accomplish this?
It seems you would want near-real-time posting of those IP.
Since you said that your 'target' is a MySQL DB, you might want to simply use
SQL to write them directly into the DB from within an acl.
There are examples available in SQL greylisting code - not cut-and-paste usable,
but which you should be able to readily adapt to your needs.
An Exim MACRO from Toleff Fog Heen is illustrative of the 'INSERT' code concept:
GRAYLIST_ADD = INSERT INTO relaytofrom (relay_ip, from_domain) \
VALUES ( '${quote_mysql:$sender_host_address}', \
'${quote_mysql:$sender_address_domain}')
One might add timestamps and an autoincrementing 'serial' field, then a trigger
on the count if MySQL has triggers Else PostgreSQL sells for the same price, and
does have.
Note that 'read' privileges are not enough, so make sure that any other SQL code
is resistant to exploits that could modify the DB improperly.
I suspect this would run faster if you used something simpler than an SQL DB.
|
Thanks Bill,
I have that now - but what I want to do is give other people a simple
way to send me IP addresses in real time. I dont want to give the direct
access to my database and provide them with passwords and query strings.
What I want to do is to have them do something simple like write a
string to a port and I'll handle the processing on my end with something
that listens on that port, gets the string, and runs the MySQL on my end.
I have this new system working for me and I want to make it public. But
to do that I need to make it easy for people to send me their data. I
think I have the server side of it working. Still testing. So what I
need now is an easy way for Exim to open a port and send a string to it.
Right now the best I have is running netcat (nc) but would like
something cleaner.
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/ |
|
| Back to top |
|
 |
Bill Hacker *nix forums Guru
Joined: 27 Jan 2005
Posts: 427
|
Posted: Fri Jul 21, 2006 2:31 am Post subject:
Re: Need a little unix help
|
|
|
Marc Perkel wrote:
| Quote: |
W B Hacker wrote:
Marc Perkel wrote:
Some of you I'm srue are experts at this. I can probably get something
to work but I want to do it right. And it's fairly simple.
What I want to do is create a simple client/server application where
clients running Exim can easily report IP addresses to the server.
The server will accept the IP addresses and execute a command to add the IP
address to a MySQL database and increment a counter. What I'm
specifically doing is counting spam/ham based on IP address. But - if I
can just receive the IP address I'll figure put how to process it.
I have xinentd if that maes things easy.
On the client side I want to in an efficient way be able to pass a
string containing the IP address to the server. Maybe talk to an open port?
So - what's the simplest way to accomplish this?
It seems you would want near-real-time posting of those IP.
Since you said that your 'target' is a MySQL DB, you might want to simply use
SQL to write them directly into the DB from within an acl.
There are examples available in SQL greylisting code - not cut-and-paste usable,
but which you should be able to readily adapt to your needs.
An Exim MACRO from Toleff Fog Heen is illustrative of the 'INSERT' code concept:
GRAYLIST_ADD = INSERT INTO relaytofrom (relay_ip, from_domain) \
VALUES ( '${quote_mysql:$sender_host_address}', \
'${quote_mysql:$sender_address_domain}')
One might add timestamps and an autoincrementing 'serial' field, then a trigger
on the count if MySQL has triggers Else PostgreSQL sells for the same price, and
does have.
Note that 'read' privileges are not enough, so make sure that any other SQL code
is resistant to exploits that could modify the DB improperly.
I suspect this would run faster if you used something simpler than an SQL DB.
Thanks Bill,
I have that now - but what I want to do is give other people a simple
way to send me IP addresses in real time. I dont want to give the direct
access to my database and provide them with passwords and query strings.
What I want to do is to have them do something simple like write a
string to a port and I'll handle the processing on my end with something
that listens on that port, gets the string, and runs the MySQL on my end.
|
OK - 'manual' input, then.
Opening a port might invite a DoS.
A well-crafted form on a Web-page might be OK.
Having them 'forward' the entire message to a dedicated address might be better,
as you could vet it for accuracy, AND get more than just the IP out of it.
| Quote: |
I have this new system working for me and I want to make it public. But
to do that I need to make it easy for people to send me their data.
|
I would be careful with 'public' there.
Not everyone is equally qualified to even ID the correct IP from whence the
problem has emanated - or if it is IP-persistent (think sporadic zombification
of communities of WinBoxen on DHCP) - or if blocking an IP used
once-in-a-great-while does more good than harm.
Look at some of the contentious junk that has found it way into rfc-ignorant.
For example, listing the entire .de .tld WHOIS as broken because it needs proper
CLI tails to craft a whois inquiry when telnet'ed into - despite the fact that
it DOES respond correctly to a proper 'whois' client. Go figure.
This new tool of yours will probably only retain 'credibility' if you are very
strict about what you allow into it.
For the near-term, multiple-acl tests and near-real-time count incrementing on
servers you control, can analyze and tune, is probably where that works best.
A razor-style sharing comes later, IMNSHO.
| Quote: | I
think I have the server side of it working. Still testing. So what I
need now is an easy way for Exim to open a port and send a string to it.
Right now the best I have is running netcat (nc) but would like
something cleaner.
|
Have a look at the ways you can tune 'checkservice', then. A template that reads
counter(s) or files published via http might be useful.
Exim can append more (or LESS) than just entire messages to arbitrary files, not
just to the default logs.
Bill
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/ |
|
| Back to top |
|
 |
Marc Perkel *nix forums Guru
Joined: 27 Jan 2005
Posts: 502
|
Posted: Fri Jul 21, 2006 2:36 am Post subject:
Re: Need a little unix help
|
|
|
W B Hacker wrote:
| Quote: | Marc Perkel wrote:
W B Hacker wrote:
Marc Perkel wrote:
Some of you I'm srue are experts at this. I can probably get something
to work but I want to do it right. And it's fairly simple.
What I want to do is create a simple client/server application where
clients running Exim can easily report IP addresses to the server.
The server will accept the IP addresses and execute a command to add the IP
address to a MySQL database and increment a counter. What I'm
specifically doing is counting spam/ham based on IP address. But - if I
can just receive the IP address I'll figure put how to process it.
I have xinentd if that maes things easy.
On the client side I want to in an efficient way be able to pass a
string containing the IP address to the server. Maybe talk to an open port?
So - what's the simplest way to accomplish this?
It seems you would want near-real-time posting of those IP.
Since you said that your 'target' is a MySQL DB, you might want to simply use
SQL to write them directly into the DB from within an acl.
There are examples available in SQL greylisting code - not cut-and-paste usable,
but which you should be able to readily adapt to your needs.
An Exim MACRO from Toleff Fog Heen is illustrative of the 'INSERT' code concept:
GRAYLIST_ADD = INSERT INTO relaytofrom (relay_ip, from_domain) \
VALUES ( '${quote_mysql:$sender_host_address}', \
'${quote_mysql:$sender_address_domain}')
One might add timestamps and an autoincrementing 'serial' field, then a trigger
on the count if MySQL has triggers Else PostgreSQL sells for the same price, and
does have.
Note that 'read' privileges are not enough, so make sure that any other SQL code
is resistant to exploits that could modify the DB improperly.
I suspect this would run faster if you used something simpler than an SQL DB.
Thanks Bill,
I have that now - but what I want to do is give other people a simple
way to send me IP addresses in real time. I dont want to give the direct
access to my database and provide them with passwords and query strings.
What I want to do is to have them do something simple like write a
string to a port and I'll handle the processing on my end with something
that listens on that port, gets the string, and runs the MySQL on my end.
OK - 'manual' input, then.
Opening a port might invite a DoS.
A well-crafted form on a Web-page might be OK.
Having them 'forward' the entire message to a dedicated address might be better,
as you could vet it for accuracy, AND get more than just the IP out of it.
I have this new system working for me and I want to make it public. But
to do that I need to make it easy for people to send me their data.
I would be careful with 'public' there.
Not everyone is equally qualified to even ID the correct IP from whence the
problem has emanated - or if it is IP-persistent (think sporadic zombification
of communities of WinBoxen on DHCP) - or if blocking an IP used
once-in-a-great-while does more good than harm.
Look at some of the contentious junk that has found it way into rfc-ignorant.
For example, listing the entire .de .tld WHOIS as broken because it needs proper
CLI tails to craft a whois inquiry when telnet'ed into - despite the fact that
it DOES respond correctly to a proper 'whois' client. Go figure.
This new tool of yours will probably only retain 'credibility' if you are very
strict about what you allow into it.
For the near-term, multiple-acl tests and near-real-time count incrementing on
servers you control, can analyze and tune, is probably where that works best.
A razor-style sharing comes later, IMNSHO.
I
think I have the server side of it working. Still testing. So what I
need now is an easy way for Exim to open a port and send a string to it.
Right now the best I have is running netcat (nc) but would like
something cleaner.
Have a look at the ways you can tune 'checkservice', then. A template that reads
counter(s) or files published via http might be useful.
Exim can append more (or LESS) than just entire messages to arbitrary files, not
just to the default logs.
Bill
|
I'm not looking for the load of sending whole messages. Those who will
be feeding it will be hand chosen by me ans will be given a secret host
and port to send it to and it will be set up to only accept from
specific hosts. But - I want to make it so that anyone can read it and
use it to filter spam. If this is sucessful then I hope some big service
will take it over and other people will do it right. I'm good at
starting things because I think outside the box. But once I get it going
then other can improve it and do it right.
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/ |
|
| Back to top |
|
 |
Bill Hacker *nix forums Guru
Joined: 27 Jan 2005
Posts: 427
|
Posted: Fri Jul 21, 2006 4:10 am Post subject:
Re: Need a little unix help
|
|
|
Marc Perkel wrote:
*heavily trimmed*
| Quote: |
Marc Perkel wrote:
What I want to do is create a simple client/server application where
clients running Exim can easily report IP addresses to the server.
The server will accept the IP addresses and execute a command to add the IP
address to a MySQL database and increment a counter. What I'm
specifically doing is counting spam/ham based on IP address.
|
*trimmed* (questions / suggestions on methods)
| Quote: |
... what I want to do is give other people a simple
way to send me IP addresses in real time. I dont want to give the direct
access to my database and provide them with passwords and query strings.
What I want to do is to have them do something simple like write a
string to a port and I'll handle the processing on my end with something
that listens on that port, gets the string, and runs the MySQL on my end.
|
*trimmed* (questions / suggestions on methods)
| Quote: | I
think I have the server side of it working. Still testing. So what I
need now is an easy way for Exim to open a port and send a string to it.
Right now the best I have is running netcat (nc) but would like
something cleaner.
|
*trimmed* (questions / suggestions on methods)
| Quote: |
I'm not looking for the load of sending whole messages. Those who will
be feeding it will be hand chosen by me ans will be given a secret host
and port to send it to and it will be set up to only accept from
specific hosts. But - I want to make it so that anyone can read it and
use it to filter spam. If this is sucessful then I hope some big service
will take it over and other people will do it right. I'm good at
starting things because I think outside the box. But once I get it going
then other can improve it and do it right.
|
Marc,
To the extent that you have a tool that can dynamically, in near-real-time, keep
a "recent bad behaviour by-IP" <brown?>list, yes, I think it IS 'out of the box'
and worth looking into. For one thing, it need not necessarily have to make
off-box callouts. For another, the IP's could be exported to a firewall, further
reducing Exim load.
I'm still interested in that, so please don't give it up.
At the point it becomes less 'near real time', less fequently updated/purged,
less local, has a longer time-to-live, and moves toward yet-another-RBL, I think
it is neither new, nor out-of-the box, and is already well-covered, particularly
if you are basing otherwise 'cheap' IP gonging on the result of otherwise
'expensive' SpamAssassin scoring.
SA already has white/black autolisting tools and there are perhaps a hundred
RBL's already. Typically fewer than 5 are sufficiently accurate, current, AND
broadly useful to justify the bother of checking against. Each sysadmin has his
own favorites.
There still may be new value to your approach at the RBL end of the spectrum,
but it is not clear to me.
A potential application that could benefit is blocking a zombie that connects,
then tries to send a dozen or so messages to our valid domains, but to all or
mostly machine-generated invalid recipients.
By incrementing an acl_c variable instead of an acl_m variable at each recipient
verification failure, then multiplying that value by 'n' seconds for a "delay =
", the rapid ramp-up in delay can cause impatient zombies to abandon the
connection, often by the 2d or 3rd message in each assault.
This approach is 'cheap' only if one can afford to hold open a lot of IP
connections and Exim process children, but it at least avoids an 'expensive' SA
call.
OTOH, it has no 'memory', the process repeats - twice more on average - before
the free-range-rude are off to hit another target - returning anywhere from
hours to days later when the whole dance repeats.
Your approach *could perhaps* 'cache' that errant behavior and put such a caller
into the penalty box right away on each succesive assault.
Permanently blocking the IP is of less value to me, as these are often transient
bad-actors amongst otherwise-OK players.
Thoughts?
Bill
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/ |
|
| Back to top |
|
 |
Marc Perkel *nix forums Guru
Joined: 27 Jan 2005
Posts: 502
|
Posted: Fri Jul 21, 2006 5:52 am Post subject:
Re: Need a little unix help
|
|
|
Here's what I'm doing. I'm using MyDNS which is a DNS server with a
MySQL backend and so it is close to real time. I think I figured out how
to do what I want with the client/server reporting. On the server side
I'm using xinetd to pipe the incoming text into a perl program and then
into MySQL. On the client side will be a small script that Exim will run
using netcat.
What I'm thinking is that the reporting client will send short strings:
ham 1.2.3.4
spam 5.6.7.8
honeypot 9.8.7.6
This strings will be processed on my end any update my database. Some of
it is real time. Some of it is calculaed every 5 minutes to update the
lists. Servers that send only ham make the whitelist, only spam makes
the blacklists, and mixed makes the yellowlist. It's working for me
right now and I'm working on being able to let others read it and a
select few feed data to it. I'll slowly increase the number of people
that can use it and see how it scales up. At some point others will see
how it works and want to do it big scale and do it right.
One important thing to think about is that the idea of the blacklist is
to be really accurate. But it isn't as much to catch spam, which it will
do, but to identify ham servers and eliminate false positives. I think
that this system if used widely enough will have it's biggest impact in
allowing good email to pass through and eliminate false positives for
banks and other commercial sources that never send spam.
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/ |
|
| Back to top |
|
 |
Bill Hacker *nix forums Guru
Joined: 27 Jan 2005
Posts: 427
|
Posted: Fri Jul 21, 2006 6:52 am Post subject:
Re: Need a little unix help
|
|
|
Marc Perkel wrote:
| Quote: | Here's what I'm doing. I'm using MyDNS which is a DNS server with a
MySQL backend and so it is close to real time. I think I figured out how
to do what I want with the client/server reporting. On the server side
I'm using xinetd to pipe the incoming text into a perl program and then
into MySQL. On the client side will be a small script that Exim will run
using netcat.
What I'm thinking is that the reporting client will send short strings:
ham 1.2.3.4
spam 5.6.7.8
honeypot 9.8.7.6
|
Two fields only? Looks ideal for a lite, fast, cdb format...
| Quote: |
This strings will be processed on my end any update my database. Some of
it is real time. Some of it is calculaed every 5 minutes to update the
lists.
|
5 minutes is probably 'near real time' enough for smtp use.
| Quote: | Servers that send only ham make the whitelist, only spam makes
the blacklists, and mixed makes the yellowlist. It's working for me
right now and I'm working on being able to let others read it and a
select few feed data to it. I'll slowly increase the number of people
that can use it and see how it scales up. At some point others will see
how it works and want to do it big scale and do it right.
One important thing to think about is that the idea of the blacklist is
to be really accurate. But it isn't as much to catch spam, which it will
do, but to identify ham servers and eliminate false positives. I think
that this system if used widely enough will have it's biggest impact in
allowing good email to pass through and eliminate false positives for
banks and other commercial sources that never send spam.
|
Blocking based on protocol misbehaviour - which we rely on more than SA scores,
is at least fairly repeatable from any given IP.
When you are using spam scores, OTOH, I can't see it as either repeatable or
simple enough to 'rate' the source IP as much of anything but 'yellow'.
Here are two (of several) problem areas:
1) Insurance, mortgage, brokerage and online banking accounts, utility bills,
airmiles programs, even video-rental and supermarket chains, not to mention
certain more specialized mailing lists - typically send messages (not
necessarily sensitive information) to customers who have signed-up for them and
(mostly) want to see them.
These are nearly always in html, usually graphics-heavy, designed with not-quite
standard Win or Lin Tools, hence usually get an unfavorable spam score -
sometimes enough to need manual whitelisting. Some of the more careful spammers
actually create "cleaner" messages than the local electric company bothers to
do, 'coz they know they must do so.
2) The largest of ISP mail services may get all or nearly all of the protocol
and DNS steps spot-on, yet suffer waves and waves of compromised WinBoxen, not
to mention a high percentage of chronically broken MUA (missing headers, MIME
encoding, etc.).
In our HKG-based 'corporate' environment, we can safely block all of roadrunner
and comcast - but I dare not do that with msn/hotmail, aol, yahoo or gmail.
- Too much risk of blocking new client inquiries that some of our clients rely on.
All of these providers are *way* better behaved than they were 3-4 years ago
when the worst 3 admitted to trafficing 2 *billion* spam messages - that they
knew about - per every 24 hours.
But cleaned-up or not, the very size of their nearly-100% WinWoes customer base
means they will probably *never* be off your 'yellow' list, and may spend a lot
of time on the blacklist.
So - I don't see that one can draw a sufficiently accurate *generalization* of
IP 'goodness' based on spam scores. If one could do so, it would already be a
mainstay of SpamAssassin or similar scanners.
Spam scanning - if a message gets that far - pretty well has to be done one
message at a time, and doesn't necessarily tell you much of lasting value about
the server on the source IP that handed it to you.
OTOH - persistent arrivals from an IP that doesn't resolve, has no PTR or A
record, uses a mismatched or obviousy forged HELO, HELO's as your own box, may
very well benefit from a white/yellow/black list IP lookup before making remote
rDNS / forward/reverse lookup calls.
Our version of that uses lists categorized as white, black, and 'brown' (draw
the obvious inference!) and is manually populated from harvesting the Exim logs,
but seldom has even 100 entries on the largest of the lists.
In fact, if any list became really large, the CPU cycles and fs activity needed
to parse it would probably make the remote lookups 'lighter', even if not as fast.
The rest is too random, IP-wise. Spammers, who have been caught with as many as
40,000 zombified separate-IP WinBoxen under their control, work very hard to
make it so. Essentially all of those will fail a simple rDNS test, but so too,
will NetSol's cluster, home to thousands of SME domain virtual mx.
The only 'easy answer' I have is "closer scrutiny, less magic".
JM2CW,
Bill
--
## List details at http://www.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://www.exim.org/eximwiki/ |
|
| Back to top |
|
 |
Google
|
|
| Back to top |
|
 |
|
|
The time now is Sun Nov 23, 2008 2:03 pm | All times are GMT
|
|
Online Advertising | Fast Loans | Mortgage | Western Union Money Transfer | Internet Advertising
|
|
Copyright © 2004-2005 DeniX Solutions SRL
|
|
|
|
Other DeniX Solutions sites:
Unix/Linux blog |
electronics forum |
medicine forum |
science forum |
|
|
Privacy Policy
|
Powered by phpBB © 2001, 2005 phpBB Group
|
|