posted at: 11:12 | path: | permanent link to this entry
size-20160520 553249 size-20160521 445430 size-20160522 322695 size-20160523 193474 size-20160524 156869 size-20160525 295624 size-20160526 744956This is the amount of bytes in bounced emails. I run a tight ship suspending bouncing users/emails. Spam is very managable. So if nothing else. This shows I have more work now with this than I had in 2012.
size-20121028:250077 total size-20121029:128818 total size-20121030:546180 total size-20121031:179077 totalThe size of october 30th is a fluke as we generated twice the number of outgoing mails that we werw supposed to (two runWTP processes).
size-20090418:307080 total size-20090419:307312 total size-20090420:286988 total size-20090421:276548 totalWe use qmail as our MTA. A terrible choice. Done before my time. I can’t seem to find any reasonable and easy implementable across the board anti spam measures for qmail.
If we had only used postfix I would have had a slew of tricks up my sleeve to limit incomming spam at the MTA level. Aw shucks!
Let us all hope that the economy will strengthen and that the crisis will end soon.
size-20090315:248780 total size-20090316:277280 total size-20090317:257092 total size-20090318:251552 total
size-20071220:154916 total size-20071221:175956 total size-20071222:194584 total size-20071223:206920 total size-20071224:203796 total size-20071225:190368 totalNot quite record level, but not far from it. The lowest this year was:
size-20071008:110360 totalAnd record high was:
size-20070726:1542880 total
Today I combined them in the new script gn (name is short and sweet). So now when I run gn I get “how many fresh bounces today” “email” “consecutive days of bouncing”.
Saving the internet. One script at a time.
posted at: 11:12 | path: | permanent link to this entry
Small update today on the RFC part. See http://bouncegrokker.org/FAQ.txtBeen away for 3 weeks in Thailand wit my family
First suspension!
Strange. Just days after users having confirmed their emails, emails start to bounce again. So today I have suspended the first user since our ISP nextente.no (sucks) shut us down temporarily. The suspended user was someone @planet.nl.
size-20061215:765872 total
Sub 1GB ok!
After sending out emails to all customers of WatcThatPage.com, even historic ones, bounces went through the roof!
size-20061202:1920840 total size-20061203:1876864 total size-20061204:1709428 total size-20061205:1689740 total size-20061206:1690868 total size-20061207:1650308 total size-20061208:1624936 total size-20061209:1507172 total size-20061210:1391948 total size-20061211:1264248 total size-20061212:1118376 total size-20061213:985308 total size-20061214:868548 totalBut now it is coming down to more reasonable levels. As now all users have confirmed that they want to continue as customers, bounces on the update emails are low. But it’l pick up again. I’m sure.
Hopefully the number of active customers will pick up again as well. We’ve lost quite a lot of customers.
Comcast addresses still gives problems. And now they won’t accept emails to get our ISP’s SMTP server anymore. It’s now an URL thingy. It can be found at http://www.comcastsupport.com/rblA starnge twist of fate
Due to complaints our ISP (nextgentel.no sucks!) blocked all of our outgoing emails! Pretty bad for a company like WatcThatPage.com who ha sending out emails to subscribing customers as sole means of income!
No counters @ comcast.net
Seem like comcast.net doesn’t have any kind of counters on their unblock procedures. I just bounced (resent) an old “please unblock” letter. And it went through. We are again able to send emails to comcast. For as long as it lasts.
size-20061115:630564 total size-20061116:692712 total size-20061117:702040 total
Yes, no, yes, no, yes, no!
We got whitelisted by comcast.net after repeated emails. But now we are blacklisted again. Our ISP’s (nextgentel.no sucks) email server even got blacklisted by aol.com. Ssizes are up.
size-20061101:449472 total size-20061102:446236 total size-20061103:462056 total size-20061104:471620 total size-20061105:496088 total size-20061106:531492 total size-20061107:582716 total size-20061108:623364 total size-20061109:640584 total size-20061110:644208 total size-20061111:619048 total size-20061112:617080 total size-20061113:611528 total size-20061114:591336 totalIf only we had the time and resources to set up our own email infrastructure and not be dependent on our ISP’s SMTP server. The akvavit will keep me alive.
A good day for bouncegrokking
A bit hung over from yesterdays excesses today got of to a slow start. But I finished the SORBS letter and the infrastructure to send it out. Sent out a lot of letters and with the work also found a lot of defective/obsolete/non working email addresses that I suspended. Todays level of bounces: 446MB needs to be trimmed down.
posted at: 19:43 | path: | permanent link to this entry
The FAQ
Beta version of SORBS explanation letter finished. Sent it to the rest of the WTP team for proofreading.
The FAQ
I have not worked on the FAQ for some time now. Fixed a typo today. That’s something. It’s at http://bouncegrokker.org/FAQ.txt
comcast.net
We are (again) blacklisted by comcast.net. Or rather our ISP is. nextgentel.no sucks!
New sorbs trouble
Seems like a new wave of sites are blocking us as our ISP (nextgentel.no sucks) is listet in the SORBS list.
secureserver.net
In the abundance of blocklists, our ISP (nextgentel.no sucks) is now blacklisted by a new service: secureserver.net. And when they are blacklisted: So are we!
Comcast plays nice
We are off the comcast.net blacklisted.
If only all ISP’s were as quick to respond!
Yes: Shit happens
We are (again) blacklisted by comcast.net. Or rather our ISP is. I have sent them an email beggin g them to unblock us.
this practice continues.
I hope our ISP won’t get blocked by more blocklists.Record low
Through hard work I have now managed to set a new record:
size-20061011:244284 total size-20061012:235916 total size-20061013:230808 total230MB is the lowest amount of bounce data ever! Hooray for me! When things are as good as this, something bad usually happens. Cross my fingers.
A happy suspended customer
Suspended on of our customers yesterday. And he was happy! (Or rather wented his dismay with BellSouth tech support).
At 06:57 10/11/2006, Jon Ross wrote: >I have now suspended your watchthatpage.com account. Thanks for letting me know. I'm working on setting up my own SMTP/POP3 server, and once that's operational, I'll be able to bypass BellSouth's incompetence. Thanks, Craig
Flatline
The amount of bounced data has leveled out at about 300MB. Not bad.
Allways new reasons for bounces. Never ending struggle.
Metrics of all sorts
The size of bounced emails has gone further down and is still decreasing:
size-20060920:411864 total size-20060921:401028 totalI have thrown in a substantial amount of hours so it is deserved.
The gimmefresh and xtail tricks gives good attack vectors to reduce the bounces.
Through old school unix text and data massaging I get all sorts of metrics.
What email addresses do I see most. Which addresses are (pretty sure) errors that gives reason to suspend (do not nsuspend for transient errors). Which domain generates most bounces etc. The jotr tool can give me what are trend changes in all of the above data.
Through all these metrics I can monitor trends and change. And it gives me several different attack vectors to reduce bounces. Which email genrates mos bounces. Who are fresh bounces. Which part of the “battle” is going weel and vice versa.
Clearly being on a roll with success is motivating!On a good roll!
The size of bounced emails keeps going down:
size-20060912:720868 total size-20060913:655756 total size-20060914:579360 total size-20060915:522024 totalTrue, I keep tidying up and suspending customers with emails that bounce, but I have to admit I am getting the reduction without too much hard work. The reason why the pool of bounced emails is not beeing reduced at a faster rate is much due to the setup of the mail server (SMTP) software qmail on our outboun mail server. It does some truly stupid things. I did not set up the server. If it had been me, it would have run postfix. Postfix rules! Made a new snippet three days ago to just show me who were the fresh bounces. Called it gimmefresh. Old school shell script:
!/bin/sh
# gimmefresh -> Who are the fresh bounces of today (and today only)
cd /home/mailer || exit
egrep -h “^X-WTP-TID: `date +%Y%m%d`” mbox-watchthatpage-user-“`date +%Y%m%d`”* | awk ‘{print $NF}’ | sort | uniq -c | sort -nr
Someone has come to their senses
The size of bounced emails has gone down a lot during the week end:
size-20060908:1040264 total size-20060909:951396 total size-20060910:846816 total size-20060911:766492 totalAnd It is not me! I havent done any work on the bounces since friday. Seems like comcast.net has come to their senses and now allow emails to the through! I’m not sure yet if this is really the reason. More debugging needed. But it looks like comcast.net now is A OK. May be that sbcglobal.net has come around as well. This bodes well! Maybe even pacbell.net, prodigy.net, swbell.net and snet.net as well. If this is the case then I am really happy. I have been working on getting them to adjust their filters for more than a year! On a sidenote. Todays tool is xtail.pl. Really nifty to monitor rotating logfiles without having to restart the tail all the time. Like this nifty command line:
xtail.pl . | grep “^X-WTP-TID: `date +%Y%m%d`”
Steady work helps
Used time just suspending the real bounces. About email addresses 20-30 each day. It helps. But manually watching and removing is slow. Gotten the amount of bouncemails for last period down to 1GB. So it helps.
Putting in the time
Doing the manual work of removing bouncing email messages. Due to the many SORBS bounces (due to our incompetent ISP (nextgentel.no)) I can’t run the automated processes to suspend en masse. Too many false positives.
And false positives are time consuming.
Take full controll
I can see no other way. As our ISP (nextgentel.no sucks) is unable, unwilling and incompetent in getting of the SORBS list. WatchThatPage.com needs to establis it’s own production net. Not in an ADSL pool.
Our ISP still listed in SORBS.
Our ISP (nextgentel.no) is still listed in the sorbs blacklist.
Too depressed
With the bounces due to our ISP’s sorbs listing I havent done much to weed out “bouncers” today.
We’ve been in contact with nextgentel.no tech support
The support person was utterly clueless of course. Uttering things like it’s sorbs.net’s fault. Clueless.
posted at: 20:36 | path: | permanent link to this entry
Blocked by sorbs!
The ISP of watchthatpage.com managed to get listed in the sorbs blocklist.
As many ISPs an IS/IT departments block IP addresses based on the sorbs.net
listings, the number of bounces for watchthatpage.com will skyrocket.
WTP’s ISP is NextGentel.no and I really don’t think they do a good enough job with their mail systems.
posted at: 16:19 | path: | permanent link to this entry
randgen
As you will soon(?) notice the utility randgen, currently in version 0.2.8 is used in a lot of the tools & uilities. All that randgen does is to generate a random string. Originally I made i to generate good and semi random passwords, but in these snippets i use it to generate unguessable filenames for temporary storage. With an 8 characters long string (and wothout special chars) it gives 248155780267521 (or 2.5x10^14) variations. Extremly har to guess for a tmp-file snatcher.
So if you want to use my snippets and scripts you willø need to0 install randgen. The source code is there. And it is GPL and free for all. If you have a problem with Free Software and the Gnu Public License, bouncegrokker.org is not for you.
posted at: 20:03 | path: | permanent link to this entry
New snippet, jnbt, added.
Added a small snippet to the collection of analysis tools today.
The letters nbt -> New Bigger Than.
It’s meant to be a way to detect newcommers to the bounce list files.
Newcommers with a high frequency, prefereably. Detecting bouncers early
ios a boon. With all the email queues on different relays and servers, it
is not unusal that one single bouncemail keeps generating bounces every
day for almost a month!
posted at: 20:03 | path: | permanent link to this entry
Made a new piece of software today. Combining earlier pieces an a small program written in c. The snippet is called jotr (Jon On The Rise). The ‘J’ is just to make it easily recogniseable for me. I hope to prefix all the hacks, snippets and programs I use with ‘bg’ before releasing them to the public.
Thje ‘jotr’ program makes it easy for me to see which email address the has the biggest increase in bounces. Makes it easier for me to stop sending mail to an email address that bounces early. Early detection is good!
posted at: 19:57 | path: | permanent link to this entry
