The Bat! Mailing Lists
By Leif Gregory
 

(click HERE to get the same page in Russian)

Contents

What is a mailing list?

There are basically two types of mailing lists. The first is a distribution type, and the second is a discussion type. The distribution type is basically a list where you are providing information in one direction. Typical lists of this type are announcements, newsletters and well, SPAM. These lists are very simple to set up, as about the only thing you have to do is to create an address group with all of the recipients in it. A discussion list is quite a bit more complex to set up. Basically, people send posts (e-mail) to a single e-mail address. From there the post is processed and then sent to everyone in an address book group. The complexity lies in setting up the Reply To:, From:, POP3 accounts etc.

General setup for discussion mailing lists

The easiest way to run a list is to have a separate POP3 account, and as a major plus a forwarding e-mail address (This is a really nice option to have. If you have a forwarding address set up, the e-mail address that subscribers will Un/Subscribe and send posts to will never change. So if you change POP3 servers or give the list over to someone else because you've decided you don't want to run it anymore, you will not impact your subscribers.) To find one of these types of services, just type either "free +email" or "free +POP3" in your favorite search engine. You can set up a discussion list using your existing (main) POP3 account, but it takes quite a bit more work, which I will not cover in this how-to. It will however, generally follow the same guidelines as a stand alone POP3, but your filtering will be a little more complex.

I am going to define some variables so that the below how-to will be much clearer. Please substitute your actual information for that which I have provided.

 

Procedure for setting up the list using a separate POP3 account

First click here to download the IDL mailing list account that I have created. You will be "importing" this to The Bat! Imported is in quotes, because there is no actual provision under The Bat! to import an account. I'll explain how to get this account installed into The Bat! in just a minute. I have done about 90% of the work for you, all you have to do is replace the "IDL" information to suit your discussion list, which we will go through step by step.

How to "import" the IDL account into The Bat!

  1. Go to where IDL.ZIP was saved when you downloaded it.
  2. Double click the IDL.ZIP file.
  3. Click the "Extract" button.
  4. In the dialog box that comes up do these two things: The "Extract To" field should be the directory that your accounts are stored in under The Bat! In my case it is: C:\Program Files\The Bat!\MAIL. The second thing is to check the "Use Folder Names" checkbox if it isn't already checked.
  5. If The Bat! isn't already running, start it.
  6. Click "Account", select "New".
  7. In the "Account Name" field, enter "IDL".
  8. Click the "Next" button.
  9. Put your name in the "Your Full Name" field.
  10. In the "E-mail Address" field put the forwarding e-mail address you set up for your list. If you didn't set up a forwarding e-mail address (You really should use one of these), then put the POP3 e-mail address that you will be using for the list.
  11. The "Organization" field can be left blank or put in the name of your discussion list.
  12. Click the "Next" button.
  13. In the "SMTP Server" field, put whatever SMTP server you will be using. I have mine set up for my regular ISP SMTP server address.
  14. In the "POP3 server" field, put the same address as you used in step 10.
  15. Click the "Next" button.
  16. In the "Username" and "Password" fields, put whatever your username/login is for the POP3 account you are using in step 14.
  17. The two checkboxes are up to you, but I have both unchecked. If the POP3 account you are using requires APOP, then you need to check this box. For the normal operation of the discussion list, you don't need to keep copies of messages on the server.
  18. Click the "Next" button.
  19. Check the "Do you want to check the account settings now" radio button and click the "Finish" button.

Ok, now that you have "imported" the IDL account, we need to finish setting it up for your discussion list.

How to set up the account for your discussion list.

  1. Under the "General" sheet, verify that all the information is correct.
  2. Under the "Transport" sheet, verify that all the information is correct.
  3. For the first few weeks of your list's "life", you should check the "Deferred" option in the "Delivery" options on the "Transport" sheet. The reason for this, is that you want to make sure that everything is correct in the outgoing messages to be sent to your subscribers before they get sent.
  4. Under the "Mail Management" sheet, you shouldn't have to change anything.
  5. Under the "Options" sheet, set whatever options you prefer to use. You may want to leave the "Empty trash folder on exit" unchecked for the first few weeks of list "life" also. Sometimes you make a mistake in mail handling, and may accidentally delete a message.
  6. Under the "Files and Directories" sheet, you may have to modify the "Home Directory" depending on whether you specified a different "Extract To" path in step 4 of the "How to "import" the IDL account into The Bat!".
  7. Click the "+" in front of the "Templates" sheet.
  8. Select "New Message". We are now going to set this template up. Hopefully you have given some thought as to what you would like your messages to look like when you post to your own discussion list. The merit to having the below boilerplate:

    --
    ***************************************************
    Report problems to: <mailto:ldgregory@biogate.com>
    Check the IDL FAQ: <http://www.idl.com/FAQ>

    To unsubscribe, click here:
    <mailto:list@idl.net?subject=Unsubscribe_IDL>
    ***************************************************

    Is that your subscribers will be reminded each time you post a message to your own discussion list of how to report problems, unsubscribe, and where your discussion list web site or FAQ is. You can make this contain as much or as little information as you want, but I would suggest keeping it short.

    Take very careful notice of the two dashes right above the boilerplate. Please note that it is '-- ' (dash dash space). The trailing space is extremely important. What this is for, is that TB cuts off any text following the '-- ' when you reply to a message. How this works in your favor is like this.

    You send a message to the list, at the bottom of this message is your boilerplate. Now if you don't have the '-- ', then when someone replies to your message, the boilerplate is quoted. This basically helps you to keep the posts looking nice and clean.
  9. This next block is your macros. Let me show them to you and then I'll explain what they are doing.

    %TO="IDL Members <list@idl.net>"
    %BCC="IDL <list>"
    %RETURNPATH=""
    %RETURNPATH="ldgregory@biogate.com"

    The Bat! requires that you have a valid e-mail address in the TO field. Putting the discussion lists e-mail address in here serves two purposes. The first is that it looks better than putting in your e-mail address. The second and most important, is that when the message is sent, you will receive it again.

    This sounds stupid, but it is necessary if you wish to verify that your message went out without actually subscribing yourself to your own list, and it allows you to keep the message in its final form just like all the other posts that come from your subscribers. This is useful if you want to create a digest version (which I'll get to later in this article), or if you want to place a searchable message archive on a Web site.

    The BCC field must contain your address book group that has all of your subscribers in it.. This is very important. If you put the address book group with all of your subscribers in the TO field, then everyone who is subscribed to your discussion list will see everyone else's e-mail address who is also subscribed to the discussion list. This is considered very bad netiquette.

    The RETURNPATH is very important in that it provides an e-mail address for bounced messages to come back to instead of your subscribers. Trust me, you have to have it in there, I learned the hard way with my list. Why are there two of them? Well, there's an undocumented feature in The Bat! that allows you to run a macro with a NULL input. What this does is to clear the field so that the second occurrence of the macro can fill it without worrying about what was stored in it previously. This will become much more easily understood when we get to the filtering section (filter four).
  10. Select the "Reply" template. Again, you'll want include the boilerplate to remind the subscribers when you reply to a subscribers post. Here are the macros:

    %TO""
    %TO="%OFROMNAME <list@idl.net>"
    %BCC="IDL <list>"
    %RETURNPATH=""
    %RETURNPATH="ldgregory@biogate.com"

    Here's where the NULL value macro becomes a little easier to understand. Let's say "Billy Bob" <bbob@somewhere.com> sends a post. If I want to reply to his message before it gets sent to the list, and send my reply to the list, then I need to change his TO info so that it matches with what we are going to do in the filtering section to all of the messages posted. Namely change the TO, and REPLY-TO information. Again, just hang on a little while and I'll explain this fully in the filters section.
  11. This pretty much concludes the account setup. You shouldn't have to use the forward template, and you can put whatever cookies in the cookies template that you want to use.
     
  12. One last thing about the account settings. There are five folders (other than the default ones) included the IDL account. Here's what they are for.

    ***** Note: If you don't see the below folders, select the IDL account, then hold down the CTRL-ALT-SHFT-L keys simultaneously. This will force TB to search for any missing folders in that account.

    SPAM - This is the folder I move SPAM to. I'm pretty aggressive in tracking down spammers. If you are interested, I wrote an article for my weekly e-zine entitled "To Catch A Spammer".

    Moderated - I've already explained this one, but it is where the messages in their final form (as they were distributed to your subscribers) will reside after the "Move moderated incoming messages" filter processes it.

    Failures - This folder will be used to catch bounce messages. Believe me, you will get them. Remember the RETURN-PATH macro? This is what its main purpose is for. Please ensure that the e-mail address you use in RETURN-PATH is different than the one your subscribers use to post and Un/Subscribe to. This will keep the bounce messages from possibly being missed by the filters in your IDL account and getting sent out to your subscribers.

    IDL Subscribers - This is the folder that I move the Subscribe requests to, once I have verified that they have been added to the address book. This provides three functions. One is that at a glance, I can tell how many people are subscribed to the discussion list. Two, I have a record of subscribes. This way if someone later on says that I am spamming them, I can prove that they (or at least someone pretending to be them) did indeed subscribe, and three, if someone did not use the correct string to subscribe i.e. they sent a message with the words "Subscribe IDL " in the body of the message as opposed to the subject, you can just copy their e-mail address to the clipboard, right click on the "IDL Subscribers" folder, select "Create a new message" which will create a new message with the "Subscription receipt" in it. All you have to do is paste the subscribers e-mail address into the TO field and send it.

    *** NOTE: I have used text files for inclusion into the templates for both "IDL Subscribes" (both filter and folder), and "IDL Unsubscribes" (both filter and folder) that contain the actual text or the Un/Subscribe. This way, if you make changes, you can do it in one place and it updates both the filter and folder versions. If you specified a different "Extract To" path in step 4 of the "How to "import" the IDL account into The Bat!", then you will need to change the directory path in all the templates (both folder and filter) for IDL Un/Subscribes.

    IDL Unsubscribers - Serves the same function as the "IDL Subscribers", but is for unsubscribers. If a subscriber messes up the unsubscription process i.e. puts the words "Unsubscribe IDL" in the body as opposed to the subject, then you can cut and paste their address into the "Create a new message" option when you right click the "Unsubscribe IDL" folder.

The Filtering Sub-system

This is the area that will make or break your mailing list. You will need at least four of the fiver filters that I will be explaining for the basic operation.

  1. While the IDL account is highlighted, click "Account" and select "Sorting office/filters"
  2. Click the "+" in front of "Incoming messages"


Filter One - Housekeeping

This filter is a housekeeping filter. Because we are changing the TO header in the "Post" filter (See filter four) to "<list@idl.net>" this message is going to come back to you through the normal post e-mail address. You've already processed this message, so you don't want to send it back out to the subscribers again. This is one other reason we are using the RETURN-PATH macro. It is our filter string, whose job it is to move already processed (sent to the list subscribers) messages to the "Moderated" folder (which by the way is where you should make your replies to messages that subscribers have posted.) If you specified a different "Extract To" path in step 4 of the "How to "import" the IDL account into The Bat!", then you will need to change the directory path in this filter to reflect the actual location of the "Moderated" folder.

What we are actually doing with this filter is searching the kludges for the string [RETURN-PATH: ldgregory@biogate.com]. This is something we have modified in filter four to solve two problems: The first is bounce messages get sent to the designated address, and two we can use it as our search string, because it will only appear in messages we have already processed.

Of course you could just send these messages to the trashcan, but if you are going to participate in your own discussion list, it is easier to reply to this message than cutting and pasting all their info during the first time it comes through. Also, if you decide later on to build a searchable message-base on a Web site, you will have the message in its final, post filter-processed form.

 

Filter Two - Subscribes

The Second filter is the subscription filter. You need to determine what keywords you want to use for your filter string. The easiest, and most logical keywords would be something like "Subscribe" and "IDL". Because not all e-mail clients support the "%20" symbol when resolving a mailto type URL, and also because some do not support actual space characters, I like to use the underscore character between the keywords i.e. "Subscribe_IDL". This is a good way to ensure that whatever e-mail client your subscribers may be using, your URL will work. The added benefit of using the underscore character is that it is almost totally invisible in a URL, because they are almost always underlined. An example of this URL that you would put on a Web pages or in the sig portion of your e-mail messages would look something like this:

To subscribe to the IDL mailing list, click here:
<mailto:list@idl.com?subject=Subscribe_IDL>

If you click on this link, The Bat! will create a new message with the subject line filled in with "Subscribe_IDL", now all the prospective subscriber has to do it click send.

After you have highlighted the filter named "Susbcribe_IDL" do this:

  1. Change the filter name in the "Name" field to whatever you wish to call it.
  2. Change the filter string in the field "Strings" to whatever you will use as your subscribe filter string.
  3. Click the "Alternatives" tab.

    What we are going to do here is to try and idiot proof your list a little. Believe me, no matter how hard you try, you will always have people trying to subscribe/unsubscribe from your list using everything but the correct keywords.

    The alternatives tab is for "ORing" your filter strings. For more information on this, check the TB help file under "Mail Filtering (Sorting Office)" "Signal Strings".

    On this sheet, you will want to change all the different variations of the subscribe signal string to match your list needs.
  4. Click the "Actions" tab.
  5. Check the "Send Auto-Reply" checkbox.
  6. Click the "Template" button.

    Here's where you will tell the new subscriber about the list, what to expect, what e-mail address to use to post messages, some rules (concerning flaming, beating a dead horse etc), where to report problems etc. Take some time to think out what you want to put in here. You can use or modify the sample template to fit your needs.

    If you read the entire sample, you would have noticed at the bottom there was a macro. It was like this:

    %SUBJECT="IDL Discussion List Subscription Receipt."

    Because this is a new message created by the filter, you should specify a subject to be placed in the SUBJECT field. The TO field will have whatever e-mail address is specified in the subscribers REPLY-TO information. Here's something important to remember, this message will contain in the FROM and REPLY-TO fields whatever information you have specified in the account that calls this filter (In this case it's the IDL account settings). This is why you should set up a separate account in The Bat! when running a discussion list. You can specify that your FROM information say something like "IDL Discussion List Moderator <list@idl.net>" and your REPLY-TO info "Leif Gregory <list@idl.net>"
  7. Also on the "Actions" sheet, you'll see that I have checked the "Add the sender's address to the address book" option. This option allows new subscribers to be automatically added to your address book group for the list. Make sure you change the name of the address book group listed to the one you are using for the list.

    ***** Note: If you check under the unsubscribe filter (filter three), you'll notice that the option to automatically remove an unsubscriber's address is not enabled. Unfortunately, TB does not currently check to see if the address really exists in the group before deleting it. What this means is if someone trys to unsubscribe from your list, but tries to do it under the wrong e-mail address, then TB will happily comply, but in reality does nothing other than to send them the unsubscription receipt saying they were successfully unsubscribe. Of course this person will probably send you some nasty e-mail when they continue receiving posts to the list. For now, it is better to do it manually.

    The way it is set up now, the unsubscribe receipt will still be automatically generated, but you can intervene before it is sent out if the address proved to be incorrect when you manually try to delete them from the address book group.

 

Filter Three - Unsubscribes

The third filter is your unsubscribe filter. Eventually, someone will want to unsubscribe, so set up this filter similar to the subscribe filter. Of course, you will want to make changes so that it reflects an unsubscribe command. I've included a sample template for you to modify or use as your see fit.

Filter Four - Posts

The fourth filter is the subscriber post filter. This is the real meat of your filtering system, because it allows subscribers to post messages to the rest of the subscribers. This filter will be set up similar to the Un/Subscribe filters, with the following differences:

Under the "Rule" tab, your first filter string should be: [list@idl.net], select the "Recipient" location, and presence is "Yes". This is a SPAM measure. Most times when you receive SPAM, the recipient list has been suppressed, meaning your e-mail address is not displayed along with all the other unlucky recipients of the SPAM. So by specifying "list@idl.net" you will be eliminating a huge majority of SPAM from making it to your subscribers.

The second filter string should be: [Subscribe_IDL], select the "Subject" location, and presence is "No". This will ensure that Subscribe requests don't get sent to the rest of your subscribers.

The third filter string should be: [Unsubscribe_IDL], select the "Subject" location, and presence is "No". This will ensure that unsubscribe requests also don't get sent.

Under the "Actions" tab scroll down till you see "Create a message for", put your name and e-mail address here like this: "Leif Gregory"<list@idl.net> Now click the template button. Here is what the stuff in the template means:

The %TEXT is going to cut and paste the posters text into this message. You don't want to use the %QUOTE macro, because that's not what you want.

The three blank macros "TO, REPLYTO, and FROM" are an undocumented feature in The Bat! What happens when you don't specify any text between the quotes is that it blanks out the field (A NULL value.) Remember when you put your name and e-mail address in the field under the action "Create a message for"? Well, if you don't blank out this field, that info will appear here in addition to the info placed there by the second occurrence of "TO" in the above macro. This is a bad thing. The REPLYTO and FROM fields will contain the information you specified in the IDL account settings, so we want to blank these out.

Ok, now on to the second occurrences of TO, REPLYTO and FROM. When a subscriber posts a message, and the other subscribers receive it, you want the TO, FROM and REPLYTO information to reflect the person who posted the message. i.e. say Billy Bob <bbob@user.com> posts a message to the list. When a subscriber receives the message, you want the TO field to say "IDL Discussion List <list@idl.net>" (unless they were replying to someone's message, then you would want it to have the repliees address here.), you want the REPLYTO field to say "Billy Bob <list@idl.net> (This is extremely important! The REPLYTO header is the address that is used when someone replies to a message. We want the replies to come back to the list, so that they will get sent to all the other subscribers too. If the replyer wants to send a private message to only Billy Bob, then they should cut and paste the address in the FROM field), which brings up what you want displayed in the FROM field. You want it to say "Billy Bob <bbob@user.com>" (You want the subscribers real e-mail address here.) With that out of the way, let's dissect the macros.

%TO="%OTONAME <list@idl.net>"    Remember when I said that you wanted the TO field to display "IDL Discussion List" unless the post is a reply? Here's what happens in this macro. %OTONAME takes all the text except the actual e-mail address i.e. "IDL Discussion List", or in the case of a reply "Billy Bob" and puts it into the TO field. We are then specifying that the e-mail address be <list@idl.net>. So the new TO address will be "IDL Discussion List <list@idl.net>" or in the case of a reply "Billy Bob <list@idl.net>" .

%ReplyTo=%OFROMNAME <list@idl.net> When a subscriber replies to a message, this is the actual e-mail address it will use. We are taking the original posters name and changing the e-mail address so that the reply gets sent to the list instead of the original poster personally.

%From="%OFROMNAME <%OFROMADDR>" Here we are using all of the original posters real information. Their real name and real e-mail address.

%BCC="IDL <list>" This is your address book group entry. Of course we do not want to show everyone on the list who else is also subscribed to the list, so we put it in the Blind Carbon Copy (BCC) field. This also help to prevent spamming of your subscribers.

 

Filter Five - Move sent messages to trash

This is the filter can do without if you don't want it. However, I would highly recommend it, otherwise the sent posts will accumulate in your "sent" mail folder, and you would have to move them to the trash manually.

 

Setting up the digest version

Right now, this is one area when The Bat! is kind of weak. I have asked RIT Labs to try and work on this area just a little more. I would have included this section in the filters section, but I didn't feel that it has the proper capability to really be used as a digest generator. There is an option on the "Actions" tab for "Export message to file". The problem is that you can either include all of the kludges (really messy looking), or none of the kludges (difficult to reply to people or to see who wrote the message and it's subject.) I asked RIT Labs if they could allow you to select which kludges to include in the exported message. I think that these kludges are necessary (DATE, FROM, REPLYTO, TO and SUBJECT,). If you wish to experiment around with the digest generator anyway, do this:

  1. In the IDL filters, click the "+" in front of "Incoming messages".
  2. Select "Move moderated incoming messages".
  3. Select the "Actions" tab.
  4. Scroll down till you see "Export message to file".
  5. Put a checkmark in the box.
  6. Click the "Browse" button to the right of the pathname field.
  7. I have already selected the "TBDigest.TXT" file. I'd suggest keeping a copy of all the digests you create for future use (on a web site etc).
  8. If you specified a different "Extract To" path in step 4 of the "How to "import" the IDL account into The Bat!", then you will need to change the directory path to reflect your directories.
  9. Put a checkmark in the "Append to existing file" radio button.
  10. Exit out of the filters dialog window.
  11. To send a digest to someone (You'll have to do this manually, as there is no way currently to define to TB when to send these digests out.) right click on the "Digest" folder, select "Create a new message". You should be able to just click send from here.
  12. Remember to empty out the "TBDigest.TXT" after sending it, because then you'll just append more messages to the end.
  13. Also remember to set up an address book group called "IDL Digest" (Or change it to whatever.)


Closing remarks and how to contact me

Ok, we're done. You have all of the stuff necessary to start your own discussion list. If you have any additional questions or find any faults in this how-to, please e-mail me at:

"Leif Gregory <ldgregory@pcwize.com>"

I will be glad to answer any questions you may have.

 

Troubleshooting

No matter how hard you try, there is always something that didn't get covered. Here are the answers to some common problems people using this how-to have come up against.

The RETURN-PATH doesn't seem to be working / posts that have already been processed and sent out to the list are being treated like new posts when they come back through the system.

Most likely the SMTP or POP server you are using is removing the RETURN-PATH statement from the header. Some are set up this way, and you might be able to get the administrator of that server to change that. If not, then you will most likely have to find a different SMTP or POP server.

To determine which is actually the culprit, create a new message from the IDL list, remove the addresses from the TO and BCC fields (you might have to enable the viewing of these fields first by clicking "View" in the message editor window and putting a check mark next to them.) Once you have cleared out these addresses, put my address <ldgregory@pcwize.com> in the TO field and send it to me. Of course, please type something in telling me why you are sending the message to me. I'll check the headers, and if it is missing, then the culprit is your SMTP server. I will then send you a reply that has a RETURN-PATH statement in it as well, you check the headers to see if it is there. If not, then your POP server is deleting them as well.

 

I've got a problem with a person who used to be subscribed, but I unsubscribed them because they either didn't follow the list rules or were causing problems. Even though they are unsubscribed, they can still post to the list.

Unfortunately, there is no way to have TB check to see if a post is coming from a person in the subscriber address book group. The only thing I can tell you is to put a filter in the incoming filter section to move their messages to the trash. Make sure you put that filter at the top of the list using the "move up" button. TB checks messages against filters in the order they are listed. Therefore, if you don't put it above at least filter four, then their messages will still get processed.