In 2006, U.S. consumers received about 800 million text messages that they identified as “spam.” In 2007, we estimated the total was around 1.1 billion. Our estimate for 2008 is 1.5 billion.

At first sight, these numbers sound large, but remember that this is over an entire year and that there are some 200 million active SMS users in the nation. In other words, the problem is negligible, and nothing like the size of the email spam problem. The typical user receives “a few” SMS spam messages per year. Obviously some unlucky people receive considerably more, but many receive none at all. Compare the 2007 estimate with CTIA’s 2007 numbers for legitimate U.S. text messaging: Spam is about one-third of 1% (0.3%) of the total messages received.

Mobile service providers such as Sprint and AT&T Wireless are highly motivated to keep it this way, for obvious customer satisfaction reasons. The good news is that cell phone networks aren’t completely “open” like email is, so it’s much more difficult for a sender to anonymously send spam. (Note that we’re being careful to estimate the number of spammy messages received--this is the figure after the carriers have thwarted other attempts to spam. We can’t accurately estimate how many unsuccessful attempts there are, but it’s a substantial number--probably at least another 1.5 billion in the United States.)

Very few of these spam messages are sent from a real handset in the conventional way. Mainly they are injected via an SMSC (Short Message Service Center) or email/Web gateway, possibly from overseas.

For the most part, the cost to a consumer of receiving a spammy SMS message is only theoretical. Many U.S. users pay a monthly charge, which entitles them to send or receive a “bucket” of messages. Even for those who pay per individual message, the average number of messages per consumer is small--one or two a week. (The North American mobile market is unusual in this respect. The vast majority of countries’ wireless providers do not charge for receiving messages or calls. For example, consumers in the U.K., France, and Germany can receive as many text messages as they like, without charge and without fear of exceeding a monthly allowance.)

… Richi Jennings

Updated July 8: the price changed slightly between when we were briefed and the announcement. 

On July 7 8, Microsoft formally announced its new Online hosted services. These are “in the cloud”, or software-as-a-service (SaaS) implementations of Exchange and SharePoint (not to be confused with Exchange Hosted Services, which is the hosted email security service formerly from Frontbridge).

The key new piece of information announced was the prices. As David predicted a typical small or medium business would pay $20 per user per month for the combination of Exchange, OCS, LiveMeeting, and SharePoint Online. However, Microsoft announced the price would actually be $15.

$15 is too expensive. Here’s two reasons why:

First, compare that price with Google Apps at $50/year ($4.17/month). At one fifth third the price, the combination of white-label Gmail, Google Calendar, Google Sites, and Google Talk may not provide 100% feature equivalence — but in most cases it will be more than good enough. Don’t forget that Google offers 25GB of email storage at that price, versus Microsoft’s 1GB, which is paltry by comparison. Some organizations may even find the free version of Google Apps is sufficient for their needs, assuming they can live with the lack of a service-level agreement.

Second, Microsoft doesn’t seem to have learned from the mistakes of others. Over the past ten years, we’ve seen vendor after vendor try to offer hosted Exchange — many of them backed by substantial Microsoft resources — but few have survived. Again, the problem is one of cost. Although the vendors would make a coherent, well-argued case that an organization should migrate to its hosted service, few IT managers believed it would save them money.

These vendors would tell potential purchasers that they could provide the service for less money than it was currently costing to run it in-house, but when it came time to actually quote for the service, most IT managers simply didn’t believe it cost them that much.

For fans of Economics 101, the hosted providers were charging more than the market would bear. Looks like Microsoft is making the same mistake. It’s a pity: Exchange 2007 is much more suited to offering the required economies of scale than previous versions.

… Richi Jennings

On July 7 8, Microsoft formally announced its new Online hosted services. These are “in the cloud”, or software-as-a-service (SaaS) implementations of Exchange and SharePoint (not to be confused with Exchange Hosted Services, which is the hosted email security service formerly from Frontbridge).

Microsoft first announced this more than a year ago, and has been offering it in beta form for several months.

The services run in Microsoft’s own datacenters, on shared hardware — or dedicated hardware for larger customers.

We’ve seen a demo of the tools to migrate users from an in-house Exchange network to the service. It looks comprehensive. The most useful aspect is that a customer can choose a subset of their users to move to the service, retaining other users on the in-house system.

Naturally, the service allows customers to synchronize their Active Directory (AD) forest between their in-house AD servers and the ones in the cloud.

Of course, this puts Microsoft into direct competition with their partners who are already offering hosted Exchange/Sharepoint — often using market development funds from Microsoft itself. However, this does at least validate the market. Microsoft will also allow partners to resell the Online services, with some attractive affiliate kickbacks.

… Richi Jennings

We recently talked to Bizanga, maker of email boundary appliances for service providers. Bizanga’s Intelligent Message Processor (IMP) products are based on hardware from IBM, HP, and Sun, and used by organizations such as Cox Communications. IMP lets service providers slot in their preferred options for spam control, malware control, encryption, archiving, certification, and authentication.

Bizanga has an interesting twist on email hygiene. Like Cisco/IronPort’s offering, IMP uses a custom operating system — as opposed to the usual Linux or BSD Unix. Unlike IronPort, it offers customers a wide choice of third-party engines, from vendors such as Cloudmark, Commtouch, Goodmail, Kaspersky, McAfee, Zimbra, etc.

Bizanga says its new version 3.5 scales to support one million consumer users. That implies it can cope with about 100 million incoming SMTP connections per day, peaking at several thousand per second. Impressive.

(Of course, Bizanga’s Intelligent Message Processor is not to be confused with the ARPANET’s first routers, which were known as Interface Message Processors.)

… Richi Jennings

Expanding on: Steve Kille’s recent bulletin about IP reputation. SPF isn’t exactly a “reputation mechanism” — although it can be used to help identify the sender, in order to make improved reputation-based decisions.

SPF, DKIM, and other “sender authentication” schemes help a receiving MTA decide if it knows which domain sent a message. For example, SPF can tell if the sending IP address 1.2.3.4 is authorized to send mail claiming to be from example.com and DKIM can tell if the incoming message was signed by example.com’s private key.

If the receiving MTA knows the sending domain, it doesn’t need to rely on the reputation of the sending IP address, which can be a blunt instrument. It allows domains themselves to have reputations. It’s especially useful for whitelisting known-good domains, so that mail from them doesn’t fall victim to the false-positive problem.

(Another common way of describing SPF et al is: mechanisms to detect forgeries, which amounts to the same thing but in a different context.)

… Richi Jennings

We were recently briefed by Abaca, a vendor of spam control technology, on its proprietary ReceiverNet spam control algorithm. Abaca makes impressive claims for the accuracy and performance of ReceiverNet. The underlying algorithms have been explained to us under NDA, and they are very interesting. In essence — and at the risk of over-simplification — it relies on understanding the relationships between senders and recipients.

The ReceiverNet algorithm seems to be a good way of automatically generating on-the-fly, per-user whitelists and blacklists, with minimal time delay. The technology employs a more rigorous, statistical approach to this, which we found impressive. It helps to differentiate between purely-spammy senders, and those that appear to send both spam and ham email (for example, where a good sender shares an IP address with a bad sender). This should help prevent false positives.

ReceiverNet also keeps track of what proportions of good and bad email a recipient usually gets. This effectively turns every user into a fuzzy spamtrap. This aggregated data is used to weight the spam/ham decision.

Of course, this idea won’t work so well in the case when little or no data is known about the sender — the “zero-hour” problem. However, ReceiverNet also does statistical analysis of the content of known spam and ham messages. This allows it to compare known content with the content of unknown messages.

If it’s still not clear whether the message is spam or ham, ReceiverNet adds it to a separate inbox of “uncertain” messages. It’s good to see that Abaca doesn’t simply dump the such messages into a huge quarantine — it’s doing what I have often suggested spam control should do: prioritize the quarantine, to show the most likely false positives first.

Of course, no shiny, new spam control technique is perfect. Over the years we’ve seen many new techniques promise much and deliver little. But we like Abaca’s approach, it’s very promising.

… Richi Jennings, with Nick Shelness

Whether or not you or I believe BorderWare’s amazing claim that it filters 98% of spam using reputation alone, it’s clear that reputation is increasingly important.

No surprise there, but what’s the implication on legitimate email users?

As more and more spam filtering relies on your reputation as an email sender, your reputation gets more and more important. Lest we forget, most spam is sent by malware-infected zombies, some of which could be on your network.

That’s why outbound spam filtering is increasingly important. It’s not just about being a good ‘net citizen--you need it to protect your reputation.

If you don’t keep a lid on spam exiting your network, your reputation will be trashed. In crude terms, your outbound IP addresses will be blacklisted, meaning your ability to send email to your legitimate business contacts will be severely limited.

If a few of your users are unwittingly sending spam, then all of your users will have serious trouble sending legitimate email.

Of course, an outbound spam filter can’t, by definition, use sender reputation. It has to rely primarily on content filtering. Those who claim that reputation is the be-all-and-end-all of spam filtering are missing an important point.

… Richi Jennings, with thanks to Proofpoint’s Andrew Lochart and David Stanley

BorderWare is making a very interesting claim. It seems to be blocking an enormous proportion of its customers’ inbound spam simply using IP reputation.

While most anti-spam vendors these days talk about blocking roughly 75% of the spam using IP reputation (basically a fancy word for DNSBLs), BorderWare’s quoted statistic is 98.3%. Wow, that’s a lot, especially considering that the law of diminishing returns almost certainly applies.

This is a good thing because the more spam one can identify and block by reputation, the less spam content one has to examine using techniques such as Bayesian analysis, which are computationally expensive.

But how does the company achieve such a high figure? By slashing the time taken for new entries to be added to its centralized reputation database (”Borderware Security Network,” soon to be rebranded as “Reputation Authority”).

These days, new zombie spam sources don’t hang around to be detected. They get sending as soon and as fast as they can--botmasters have realized that a fresh, undetected spam source is far more effective than an old, known source. Minutes count; in fact in the spameconomy, time is money.

… Richi Jennings

I moderated a webinar earlier this week. It was intended to be a press-only event, to support Abaca’s recent launch of ReceiverNet. Inevitably with these things, a few non-press register, but that’s perfectly OK.

The thing that really surprised me was that non-press outnumbered the press folks two-to-one. There also seems precious little spam-related on publications’ editorial calendars.

Doesn’t the mainstream media care about spam any more? Certainly their readers do, as evidenced by the continuing churn in the spam filtering marketplace.

Any thoughts? Leave a comment; we’d love to hear what you think.

… Richi Jennings

Trend Micro takes an unusual approach with its hosted/managed/in-the-cloud email security service. Rather than trying to do everything, it sticks to what a service is good at.

Trend is applying the 80/20 rule. It promotes a hybrid approach, with the hosted service implementing only a first level of spam filtering based on reputation. This filters roughly 80% of the inbound spam. The remaining email is passed on to spam filtering appliances on the customers’ premises, to deal with the other 20%.

The on-premise appliance can therefore be more easily customized to conform to local policy. When it comes to filtering spam using content, it’s best to have an understanding of the types of legitimate content that the organization sends and receives. The obvious example is medical organizations, which may well expect to receive email about a certain blue pill whose name begins with V.

Of course, organization-specific customization can be done in the cloud. There’s nothing intrinsic about it that forces it to be on-premise, but this split in responsibilities seems to have merit.

… Richi Jennings

We recently talked to Voltage Security, which announced something called “Connected VSN.”

VSN is the Voltage Security Network, and it isn’t new. It’s a hosted service that implements the key management for Voltage-style identity-based encryption (IBE). The idea is that instead of on-premise key management, you centralize the key generation in the cloud. This is similar to the architecture used by Identum (now part of Trend Micro). It’s the “Connected” part that’s new.

There’s a class of customer that wants to do outbound encryption at the gateway--possibly driven by local policy--but doesn’t want to provide the decryption service to nonlocal users. This type of hybrid architecture is what Connected VSN is for.

The sender has an on-premise Voltage appliance that manages keys and performs outbound encryption. Recipients then use the VSN service hosted by Voltage to decrypt the message.

… Richi Jennings

IronKey isn’t just another encrypted USB flash-drive-key-stick-thingy. For a start, the company makes a big thing of its claim that IronKey is the only such device designed from the get-go to be secure (as opposed to a flash drive that’s had security “bolted-on,” presumably).

It’s an interesting claim, but of arguable merit. However, there are other aspects that are worth talking about:

• This key will self-destruct. If you try to disassemble it, or if you enter the wrong password too many times, the IronKey doesn’t just wipe itself, it destroys the flash memory, making it worthless.
• It’s not just a device, but also a service. If you register the device on IronKey’s Web site, the company offers password recovery/escrow and access to IronKey’s own Tor anonimizing network (i.e., a private network, not the usual public one).
• It also acts as a 2FA device. A firmware update will add the necessary logic to make it act as a VeriSign VIP device, for two-factor authentication. An “enterprise” version of the device will also have similar support for RSA SecurID.

Shipping now for Windows XP and Vista. Mac and Linux support are “nearly ready.”

… Richi Jennings