Chapter 7: The Mail Queue Up Main page Chapter: Suggested Reading 

8 Command-Line Log Analysis Tools

With InterWorx 4.9, we introduced some new log analytics tools, hereafter referred to as qmail-tools. They’re a set of bash scripts that provide a more human-friendly readout for qmail’s logs. They convert the weird base64 timestamp formatting to something actually legible, but also allow you to break down the logs in almost any way you want to.
A brief word of caution before we begin. These logs operate only on the send level. If the mail you’re looking for never touched send, but died at the smtp level, then it won’t show up using these tools. With the power and flexibility of these tools you can do anything from find spammers to provide assurances to your users that their e-mails are in fact getting through to recipients.

8.1 Path Setup

The new qmail log analytics scripts are in ~iworx/lib/logtools/bin/. It’s probably easier to use them if you execute the following command first to drop them into your command path:
export PATH=$PATH:/home/interworx/lib/logtools/bin
That’ll let you access the tools regardless of what directory you’re in, though they do specifically refer to the qmail log files and won’t be able to analyze any other logfiles.

8.2 Selectors

There are two “selector” commands, each of which lets you select a range of time to examine the logs in. You MUST pipe the selector commands into an operator command to get anything useful - Running the tools by themselves just gets you a help page. Worth noting is that both tools use either epoch seconds, the UNIX date command, or a “human-friendly date stamp”, “fuzzy date”, sometimes called an “hstamp”. Hstamps come with a caveat: the user is not permitted to “mix” fuzzy dates. You cannot, for example, specify “3 days 12 hours 4 minutes”. You would need to do the math on that to get “5044 minutes”.
It’s worth noting that if you exceed the time frame allotted by qmail’s logs, it will give you an error that looks like this:
Warning: We only have logs dating back to <date of earliest log>.

8.2.1 qmail-last-x

The “qmail-last-x” command selects the logs for a time period starting however long ago until now. Hence “last x”, where x is any specified period.
  1. [root@host]# qmail-last-x “3 hours” |
  2. [root@host]# qmail-last-x “5 minutes” |
  3. [root@host]# qmail-last-x “6 days” |

8.2.2 qmail-start-end

The “qmail-start-end” command lets you give two timestamps in which to gather logs, to the nearest hour’s precision. The last two examples will both give you data from November 4, 2011 20:00:00 EST to November 11, 2011 15:00 EST.
  1. [root@host]# qmail-start-end ‘hstamp ’5 hours ago’‘ ‘hstamp ’3 hours ago’‘ |
  2. [root@host]# qmail-start-end ‘date +%s -d"2011-11-04 20:13"‘ ‘date +%s -d"2011-11-11 15:38"‘ |
  3. [root@host]# qmail-start-end 1320438841 1321043679 |

8.3 Operators

So what does someone pipe these amazing, flexible selection commands into? Why, the search operators, of course. Search operators let you specify what you’re looking for within the specified time period. There are two different search operators, the “Show” and “Top” commands, and a special class of operator called an “X-Filter”.

8.3.1 Show

By default, the “Show” search operators don’t display the mail sender. The -detail version of each script, however, DOES show senders.
show-all Displays all successes, deferrals, and failures in the given time period.
  1. [root@host]# qmail-start-end ‘date +%s -d"2011-11-08 02:13"‘ ‘date +%s -d"2011-11-10 14:38"‘ | show-all
  2. [root@host]# qmail-last-x ’12 hours’ | show-all-detail
show-success Displays all successfully delivered mail in the given time period.
show-failures Displays all failed transfers in the given time period.
show-deferrals Displays all deferred mails in the given time period.

8.3.2 Top

The important things to know about the “Top” search operators is that each is able to be sorted by field, and each comes in a “-local” and “-remote” flavor which can distinguish between locally-made and remote deliveries.
top-recipients Lists the users who have received the most mail in the specified time period.
top-senders Lists the users who have sent the most mail in the specified time period.
top-deferrals Displays a list of the users who have had the most deferrals.
top-failures Shows the users who have had the most failed deliveries.
top-user-stats Displays a combination of all statistics for all users.
top-domain-stats Displays a combination of all statistics for all domains.

8.4 X-Filters

X-Filters are inserted between the selector and operator in a log search and act like any other filter, permitting a narrower search.
xrcpt The “xrcpt” filter, when passed with a recipient e-mail address, narrows a search to only include that recipient address.
  • [root@host]# qmail-last-x ’6 days ago’ | xrcpt i.get.lots.of.spam@example.tld | show-all
xsend When paired with a sending e-mail address, “xsend” restricts searches to only contain that address.
  • [root@host]# qmail-start-end ‘hstamp ’20 hours ago’‘ ‘hstamp ’3 minutes ago’‘ | xsend mail-is-broken@example.tld | show-failures-detail
xuser Pass a UNIX username to “xuser” and it will look for records only on domains owned by that user.
  • [root@host]# qmail-start-end ‘hstamp ’1 week ago’‘ ‘hstamp ’3 seconds ago’‘ | xuser mailgag1 | show-deferrals
xwebmail “xwebmail” requires no arguments - It merely shows you all webmail usage.
  • [root@host]# qmail-last-x ’12 hours ago’ | xwebmail | show-all
 Chapter 7: The Mail Queue Up Main page Chapter: Suggested Reading 

(C) 2017 by InterWorx LLC