Spam Filters, Email Filters: How do they work?

There are many questions about SPAM filters and how they work, or in some cases, don’t work.  You think that you have blocked something and a couple of days later it’s back, Why?  There are a lot of questions around SPAM filters and how they work, what triggers the blocking? Then the bigger problem is how do the SPAMMERS adjust to get around the filters? This can get confusing, but I’ll explain the process as best I can without getting too technical. Filtering is such an essential aspect of any security strategy, but it is only one aspect or layer of a much bigger plan. However, it is also one of the most end-user facing components of the strategy, which can make it one of the most misunderstood and frustrating parts of the security strategy for end-users.

What is the scope of this problem? According to Trend Micro, there are more than 400 billion spam-based emails sent every day. OK, so you get a dozen a day in your inbox or even fifty a day, you think that is terrible try turning off your SPAM filters, the number of messages would stagger you into submission. However, in the world of security, this is one area that company’s do pretty good at managing because it is a visible threat, something you see, and something you have to deal with once it is in your inbox. Then there is always the struggle between convenience and security, the underlying question of the productivity of employees, and finding the balance between security and productivity.  The education of employees is key to determining what only resistance to understanding the risk and exposure of not having proper filtering in place, in comparison to what is simply inconvenient to the employees' work processes.

The common question is, what triggers a message to be blocked?  There are a lot of reasons messages get blocked, but most of it is based on a scoring algorithm that is based on several industry standards and company configurable parameters.

• Content Filters; this is one area where the words and the combination of words matter. The filters are looking for profanity, sexual references, terms used in combination that could be considered offensive or profiling.

• Header Information; The header information includes but is not limited to the sender's email address, is it valid? There are many vital triggers such as numeric characters in the email address, or numeric characters in the return address. Additional triggers may be the sender's address is different than the return address, or there are active links to websites in the header or message body. References to foreign countries or IP addresses that are external IP’s could also trigger blocking. All these can trigger the message to be blocked or quarantined.

• Blacklists have the sender ever been or are currently on a blacklist due to being reported as spammers.

• Rules-based triggers, these are rules that are configured by your company based on specific corporate standards and expectations. These could be filtering items such as adult material or limiting the type of executable files or even limiting the file size and active links and embedded macros within email messages. Some companies may filter emails from Social media or other file sharing applications.

Content filtering what does that mean? According to Barracuda, content filtering is a process that screens and excludes web pages or emails that are deemed objectionable. Content filtering is often used in conjunction with firewalls to add a layer of security to corporate networks. Content filtering can also perform policy enforcement or to maintain corporate ethical standards. Content filtering works by analyzing content patterns, which could be text strings, words used together, or objects embedded in pictures. According to Barracuda, some of the scenarios that would be blocked include:

• Adult material or material that has been reported through the reporting process as pornography, this is an obvious one due to sexual harassment laws and demeaning workplace laws.

• SPAM sites, this one is a little more difficult because what is SPAM to some is gold to others; however, this would be websites, IP addresses, or domain names that have been reported or labeled as a location that is spending out SPAM messages. When it comes to the spread of Malware, Ransomware, or other viruses, SPAMMING is a significant contributor to how viruses spread. However, due to the increasing use of online resources, shopping sites, and other sites by employees, the expectation that SPAM is going to go down is simply not realistic.

• Hate sites or sites that have violent content could even include emails that refer to these sites. Organizations will block these sites for employee safety, and no company wants to be involved with organizations that promote hate or violence.

• Social Networking Sites can be blocked due to corporate standards because they can distract employees and reduce productivity. However, most content filtering products can open Social Media sites up for specific individuals due to job roles such as a marketing person.

Blacklist, what are they, and how do they work, but more importantly, how do you get on them?  Blacklists are used by Internet service providers to identify and block IP addresses, websites, and email addresses that are distributing harmful information. There are different types of blacklists, private blacklists these tend to be based on some very restrictive guidelines, based on SPAM reports, and you may not know you are on one of these individual lists until you start getting bounce-back messages.  A public blacklist is something that can be searched to determine if you are on the blacklist or not. Public blacklists are typically blocking an IP address or a domain name.  Blacklists can often create some end-user confusion because people think that just because the individual has whitelisted an IP address or domain name, that email should come through.  However, many times these IP addresses or domain names are on the master list distributed by vendors such as Barracuda, Webroot, SORBS, or SPAMHAUS. When those lists get updated, the IP address or domain can be blocked again.

Getting put on a blacklist is kind of like being placed in the penalty box in hockey. If you think about blacklisting like a regular mail process, you have receivers' address and a return address. Your IP address or domain name is how Internet service providers track all emails where it is going and where it came from based on IP address and domain name. During the email sending process, the IP address is compared to a list of know bad actors on the Internet, and if that IP address is on the list, then it triggers your filtering to block that email message. Whitelisting is a process that users can go through to allow individual emails to come through the filtering. The method of Whitelisting is to tell the filtering system that you know this sender and trust them. However, this process does not always work as expected for users because someone else may be reporting them as SPAM, and when the master lists synchronize again, then they could be blocked again. The process of blacklisting and whitelisting can be very frustrating to users, this cat and mouse game can continue because managing these lists is an automated process, and it is not based on the individual, and that can be hard for individuals to understand.

There are some things you can do to keep you from being blacklisted.  According to SparkPost.com, here are some basic rules of thumb that could keep you and your company off the blacklist.

• Don’t buy an email list; I get these email list solicitations all the time, especially when I register for a conference. However, people on these lists will at a higher percentage rate mark your email as SPAM, which could result in you or your company being added to a blacklist.

• Don’t string readers along with vague content; this is one of the items that filters are looking for; make sure that your subject line is clear and pointed.

• Don’t send attachments; unfortunately, this is also a critical matrix that filters are looking for; however, this may only get you blocked but not blacklisted. Utilizing collaborative sites such as ShareFile or Teams can and are better at managing attachments anyway.

• Use a legitimate address, email addresses with random letters or numbers are key watch indicators for SPAM filters, they may be cute, but it could get you blacklisted.

• Be careful with punctuation, exclamation marks, or words commonly used by Spammers such as Free, Win, Opportunity will often trigger filters to block emails.

• Limit the use of BCC; large bcc lists will trigger blocking and could add you to blacklists.

• Don’t use too many images; you need to have a balance of images and text not to trigger the filtering. Images can have embedded code that is viruses, so emails with only photos are likely to be blocked.

• Don’t use all caps in the subject line; there are other ways to emphasize something such as bold or underlining.

• Have a secondary email address for online shopping, social media, or other subscription-based products or services. Separating personal and business is not only a reasonable security precaution to protect your company, but it will also put the focus of the filters on another email address verse your corporate address.

Attachments are one of the biggest frustrations for users when they get blocked or quarantined; the reasons are simple according to F-Secure, as referenced in a Forbes Magazine article, 85% of attachments have malicious code or the potential for containing malicious code. This is the biggest reason that most systems and filtering devices are set to quarantine emails with attachments. Or companies are now encouraging the use of file-sharing programs that allow for the user to attach an inactive link to a document inside of Microsoft Teams or Citrix ShareFile as two examples.  Some of the most dangerous file attachments are.DOC, which is Microsoft Word, XLS, which is Microsoft Excel, PDF.ZIP these are some of the most recognized and dangerous file types, but they are the most commonly used in the business world.

These common file types are the most used by hackers because the hope is that you will recognize the file type, and you will not correctly check the email to ensure that it is legitimate. Checking things like the email address of the sender, making sure that the email address is spelled correctly, the name in the header is the same as the return sender address, checking for proper punctuation and spelling.  Doing all this before opening any attachment is very important to protect yourself from Malware, Ransomware, and Phishing attacks. According to Forbes magazine, there are an estimated 60 billion legitimate emails per day. These are translating to a click-through rate of almost 15%, which equals millions of click-throughs per day. These click-throughs are exposing your company to hundreds of viruses and malware within legitimate emails, so when you consider that nearly 400 billion emails per day are sent, the risk is very high. However, SPAM and email filters regardless of how frustrating to end-users they are without them; the exposure rate is exceptionally high.

Whitelists are another common function of email and SPAM filters, and there are, as you can imagine, a lot of factors that go into the whitelist process. Whitelisting does increase risk when you think about that 91% of email attacks start with a phishing or Spoofing of emails. Phishing or Spoofing is email processes that specifically target individuals within your organization to enlist a response or confirmation to that email. Whitelisting an individual or domain can increase the convenience of receiving emails, but it can also increase the risk by exposing you to upstream malware and other viruses. If you are going to whitelist, be careful, make sure that you want to whitelist the sender or domain, then keep in mind the filtering is trying to tell you something. The technology thinks there is a problem, so check it before you simply whitelist because you don’t want to go through the validation process.