This is a technical post where we discuss the use of the Postal SMTP Benchmarking tool to determine the performance of your email servers. For our examples, we’re targeting our Managed Email Tunnel but the information should also be of use for any plain mail server.

Let’s start by crediting Russel Coker for giving us Postal, a great tool for benchmarking mail servers. While technically part of a suite designed to measure the performance of a mail system from end to end, this post will introduce the use of this tool to measure the speed at which email messages can be accepted from your email software by your MTA.

Postal composes and sends random email messages using configurable parameters. Each message submission is logged and counted, and a minute by minute running total is produced. This allows you to see the actual number of messages along with any errors that might occur during message submission.

Benchmark preparation

Since in this post we’ll only look at measuring the speed at which a Managed Email Tunnel can accept your email flow, we need to make sure that whatever messages we’ll send, won’t escape the email system. Leaking the amount of email involving in just a few minutes of testing can cause operational issues and is generally considered abusive.

We’ve already taken care of this for you. All Managed Email Tunnel instances come pre-configured so that email sent from benchmark@esmtp.email to discard@tld.invalid is accepted, signed and queued just as regular email. Later, all test emails will be silently discarded without consequences to any third parties. This allows for a very simple benchmarking setup.

Our recommended setup involves creating a directory where we’ll keep configuration files and the test result data for later analysis. The key parts is to tell Postal how to address the emails it will be generating. In our Linux test server we would type the following commands:

mkdir -p postal/logs
cd postal
echo discard@tld.invalid > recipients
echo benchmark@esmtp.email > senders

Now, within the postal directory, we have all we need to test.

Executing the test

More than 85% of our customers’ messages are between 25K and 35K in size so we’ll pick these thresholds for our messages. Assuming that your Managed Email Tunnel tunnel is yours.tunnel.esmtp.email, then the following command could be used for the test:

/usr/sbin/postal -M 25 -m 35 -t 10 -c 500 \
  -z logs/test-`date +%Y%m%d`             \
  -a -f senders yours.tunnel.esmtp.email recipients

The example above will create 10 concurrent threads. Each will keep one SMTP connection open and send up to 500 messages. After sending the random number of messages, the SMTP connection is closed and a new SMTP connection is created. Each minute, on the minute, Postal will output the activity counters that report the number of messages accepted, the number of kilobytes sent, the number of connections and the number of errors.

To stop the test, press Ctrl+C in your keyboard and wait for the next counter. Then hit Ctrl+\ to force Postal to finish.

For the test results depicted in the table below, messages were sent from a server located around 70 ms away from the Managed Email Tunnel. This roughly matches a scenario where the originating mail server is located in Europe, interacting with a tunnel instance based in the US East Coast.

While these results aren’t bad by any stretch, they can be much better when the originating mail server is closer to the tunnel instance. In this case, they are 6 ms apart and these are the results.

As the tests run, you can check your Managed Email Tunnel monitoring console and see performance indicators, which is a helpful feedback mechanism to further tune your tunnel’s performance.

All things considered, these numbers show that a nearby instance of Managed Email Tunnel can accept 900,000 messages per hour.

Recommended parameters

There’s a wide variety of email submission programs in use that are compatible with our Managed Email Tunnel. Keeping that variety in mind, these are general settings that you can use as a starting point for tuning your own delivery. Rather than try and list products our customers are using, we’ve grouped them based on the library they use for the actual message delivery.

The restriction in the maximum number of messages per connection is usually memory-related, as many commonly used email generators will require progressively larger amounts of memory and may fail mid-campaign. Because of this, we advise our clients to find sending parameters that work well and don’t optimize unnecessarily.