p9cxRGhc

1Here are some useful things to know for managing an Exim 4 server. This assumes a prior working knowledge of SMTP, MTAs, and a UNIX shell prompt.
2
3Message-IDs and spool files
4The message-IDs that Exim uses to refer to messages in its queue are mixed-case alpha-numeric, and take the form of: XXXXXX-YYYYYY-ZZ. Most commands related to managing the queue and logging use these message-ids.
5
6There are three -- count 'em, THREE -- files for each message in the spool directory. If you're dealing with these files by hand, instead of using the appropriate exim commands as detailed below, make sure you get them all, and don't leave Exim with remnants of messages in the queue. I used to mess directly with these files when I first started running Exim machines, but thanks to the utilities described below, I haven't needed to do that in many months.
7
8Files in /var/spool/exim/msglog contain logging information for each message and are named the same as the message-id.
9
10Files in /var/spool/exim/input are named after the message-id, plus a suffix denoting whether it is the envelope header (-H) or message data (-D).
11
12These directories may contain further hashed subdirectories to deal with larger mail queues, so don't expect everything to always appear directly in the top /var/spool/exim/input or /var/spool/exim/msglog directories; any searches or greps will need to be recursive. See if there is a proper way to do what you're doing before working directly on the spool files.
13
14Basic information
15Print a count of the messages in the queue:
16
17root@localhost# exim -bpc
18Print a listing of the messages in the queue (time queued, size, message-id, sender, recipient):
19
20root@localhost# exim -bp
21Print a summary of messages in the queue (count, volume, oldest, newest, domain, and totals):
22
23root@localhost# exim -bp | exiqsumm
24Print what Exim is doing right now:
25
26root@localhost# exiwhat
27Test how exim will route a given address:
28
29root@localhost# exim -bt alias@localdomain.com
30user@thishost.com
31    <-- alias@localdomain.com
32  router = localuser, transport = local_delivery
33root@localhost# exim -bt user@thishost.com
34user@thishost.com
35  router = localuser, transport = local_delivery
36root@localhost# exim -bt user@remotehost.com
37  router = lookuphost, transport = remote_smtp
38  host mail.remotehost.com [1.2.3.4] MX=0
39Run a pretend SMTP transaction from the command line, as if it were coming from the given IP address. This will display Exim's checks, ACLs, and filters as they are applied. The message will NOT actually be delivered.
40
41root@localhost# exim -bh 192.168.11.22
42Display all of Exim's configuration settings:
43
44root@localhost# exim -bP
45Searching the queue with exiqgrep
46Exim includes a utility that is quite nice for grepping through the queue, called exiqgrep. Learn it. Know it. Live it. If you're not using this, and if you're not familiar with the various flags it uses, you're probably doing things the hard way, like piping `exim -bp` into awk, grep, cut, or `wc -l`. Don't make life harder than it already is.
47
48First, various flags that control what messages are matched. These can be combined to come up with a very particular search.
49
50Use -f to search the queue for messages from a specific sender:
51
52root@localhost# exiqgrep -f [luser]@domain
53Use -r to search the queue for messages for a specific recipient/domain:
54
55root@localhost# exiqgrep -r [luser]@domain
56Use -o to print messages older than the specified number of seconds. For example, messages older than 1 day:
57
58root@localhost# exiqgrep -o 86400 [...]
59Use -y to print messages that are younger than the specified number of seconds. For example, messages less than an hour old:
60
61root@localhost# exiqgrep -y 3600 [...]
62Use -s to match the size of a message with a regex. For example, 700-799 bytes:
63
64root@localhost# exiqgrep -s '^7..$' [...]
65Use -z to match only frozen messages, or -x to match only unfrozen messages.
66
67There are also a few flags that control the display of the output.
68
69Use -i to print just the message-id as a result of one of the above two searches:
70
71root@localhost# exiqgrep -i [ -r | -f ] ...
72Use -c to print a count of messages matching one of the above searches:
73
74root@localhost# exiqgrep -c ...
75Print just the message-id of the entire queue:
76
77root@localhost# exiqgrep -i
78Managing the queue
79The main exim binary (/usr/sbin/exim) is used with various flags to make things happen to messages in the queue. Most of these require one or more message-IDs to be specified in the command line, which is where `exiqgrep -i` as described above really comes in handy.
80
81Start a queue run:
82
83root@localhost# exim -q -v
84Start a queue run for just local deliveries:
85
86root@localhost# exim -ql -v
87Remove a message from the queue:
88
89root@localhost# exim -Mrm <message-id> [ <message-id> ... ]
90Freeze a message:
91
92root@localhost# exim -Mf <message-id> [ <message-id> ... ]
93Thaw a message:
94
95root@localhost# exim -Mt <message-id> [ <message-id> ... ]
96Deliver a message, whether it's frozen or not, whether the retry time has been reached or not:
97
98root@localhost# exim -M <message-id> [ <message-id> ... ]
99Deliver a message, but only if the retry time has been reached:
100
101root@localhost# exim -Mc <message-id> [ <message-id> ... ]
102Force a message to fail and bounce as "cancelled by administrator":
103
104root@localhost# exim -Mg <message-id> [ <message-id> ... ]
105Remove all frozen messages:
106
107root@localhost# exiqgrep -z -i | xargs exim -Mrm
108Remove all messages older than five days (86400 * 5 = 432000 seconds):
109
110root@localhost# exiqgrep -o 432000 -i | xargs exim -Mrm
111Freeze all queued mail from a given sender:
112
113root@localhost# exiqgrep -i -f luser@example.tld | xargs exim -Mf
114View a message's headers:
115
116root@localhost# exim -Mvh <message-id>
117View a message's body:
118
119root@localhost# exim -Mvb <message-id>
120View a message's logs:
121
122root@localhost# exim -Mvl <message-id>
123Add a recipient to a message:
124
125root@localhost# exim -Mar <message-id> <address> [ <address> ... ]
126Edit the sender of a message:
127
128root@localhost# exim -Mes <message-id> <address>
129Access control
130Exim allows you to apply access control lists at various points of the SMTP transaction by specifying an ACL to use and defining its conditions in exim.conf. You could start with the HELO string.
131
132# Specify the ACL to use after HELO
133acl_smtp_helo = check_helo
134
135# Conditions for the check_helo ACL:
136check_helo:
137
138    deny message = Gave HELO/EHLO as "friend"
139    log_message = HELO/EHLO friend
140    condition = ${if eq {$sender_helo_name}{friend} {yes}{no}}
141
142    deny message = Gave HELO/EHLO as our IP address
143    log_message = HELO/EHLO our IP address
144    condition = ${if eq {$sender_helo_name}{$interface_address} {yes}{no}}
145
146    accept
147NOTE: Pursue HELO checking at your own peril. The HELO is fairly unimportant in the grand scheme of SMTP these days, so don't put too much faith in whatever it contains. Some spam might seem to use a telltale HELO string, but you might be surprised at how many legitimate messages start off with a questionable HELO as well. Anyway, it's just as easy for a spammer to send a proper HELO than it is to send HELO im.a.spammer, so consider yourself lucky if you're able to stop much spam this way.
148
149Next, you can perform a check on the sender address or remote host. This shows how to do that after the RCPT TO command; if you reject here, as opposed to rejecting after the MAIL FROM, you'll have better data to log, such as who the message was intended for.
150
151# Specify the ACL to use after RCPT TO
152acl_smtp_rcpt = check_recipient
153
154# Conditions for the check_recipient ACL
155check_recipient:
156
157    # [...]
158
159    drop hosts = /etc/exim_reject_hosts
160    drop senders = /etc/exim_reject_senders
161
162    # [ Probably a whole lot more... ]
163This example uses two plain text files as blacklists. Add appropriate entries to these files - hostnames/IP addresses to /etc/exim_reject_hosts, addresses to /etc/exim_reject_senders, one entry per line.
164
165It is also possible to perform content scanning using a regex against the body of a message, though obviously this can cause Exim to use more CPU than it otherwise would need to, especially on large messages.
166
167# Specify the ACL to use after DATA
168acl_smtp_data = check_message
169
170# Conditions for the check_messages ACL
171check_message:
172
173    deny message = "Sorry, Charlie: $regex_match_string"
174    regex = ^Subject:: .*Lower your self-esteem by becoming a sysadmin
175
176    accept
177Fix SMTP-Auth for Pine
178If pine can't use SMTP authentication on an Exim host and just returns an "unable to authenticate" message without even asking for a password, add the following line to exim.conf:
179
180  begin authenticators
181
182  fixed_plain:
183  driver = plaintext
184  public_name = PLAIN
185  server_condition = "${perl{checkuserpass}{$1}{$2}{$3}}"
186  server_set_id = $2
187>  server_prompts = :
188This was a problem on CPanel Exim builds awhile ago, but they seem to have added this line to their current stock configuration.
189
190Log the subject line
191This is one of the most useful configuration tweaks I've ever found for Exim. Add this to exim.conf, and you can log the subject lines of messages that pass through your server. This is great for troubleshooting, and for getting a very rough idea of what messages may be spam.
192
193log_selector = +subject
194Reducing or increasing what is logged.
195
196Disable identd lookups
197Frankly, I don't think identd has been useful for a long time, if ever. Identd relies on the connecting host to confirm the identity (system UID) of the remote user who owns the process that is making the network connection. This may be of some use in the world of shell accounts and IRC users, but it really has no place on a high-volume SMTP server, where the UID is often simply "mail" or whatever the remote MTA runs as, which is useless to know. It's overhead, and results in nothing but delays while the identd query is refused or times out. You can stop your Exim server from making these queries by setting the timeout to zero seconds in exim.conf:
198
199rfc1413_query_timeout = 0s
200Disable Attachment Blocking
201To disable the executable-attachment blocking that many Cpanel servers do by default but don't provide any controls for on a per-domain basis, add the following block to the beginning of the /etc/antivirus.exim file:
202
203if $header_to: matches "example\.com|example2\.com"
204then
205  finish
206endif
207It is probably possible to use a separate file to list these domains, but I haven't had to do this enough times to warrant setting such a thing up.
208
209Searching the logs with exigrep
210The exigrep utility (not to be confused with exiqgrep) is used to search an exim log for a string or pattern. It will print all log entries with the same internal message-id as those that matched the pattern, which is very handy since any message will take up at least three lines in the log. exigrep will search the entire content of a log entry, not just particular fields.
211
212One can search for messages sent from a particular IP address:
213
214root@localhost# exigrep '<= .* \[12.34.56.78\] ' /path/to/exim_log
215Search for messages sent to a particular IP address:
216
217root@localhost# exigrep '=> .* \[12.34.56.78\]' /path/to/exim_log
218This example searches for outgoing messages, which have the "=>" symbol, sent to "user@domain.tld". The pipe to grep for the "<=" symbol will match only the lines with information on the sender - the From address, the sender's IP address, the message size, the message ID, and the subject line if you have enabled logging the subject. The purpose of doing such a search is that the desired information is not on the same log line as the string being searched for.
219
220root@localhost# exigrep '=> .*user@domain.tld' /path/to/exim_log | fgrep '<='
221Generate and display Exim stats from a logfile:
222
223root@localhost# eximstats /path/to/exim_mainlog
224Same as above, with less verbose output:
225
226root@localhost# eximstats -ne -nr -nt /path/to/exim_mainlog
227Same as above, for one particular day:
228
229root@localhost# fgrep YYYY-MM-DD /path/to/exim_mainlog | eximstats
230Bonus!
231To delete all queued messages containing a certain string in the body:
232
233root@localhost# grep -lr 'a certain string' /var/spool/exim/input/ | \
234                sed -e 's/^.*\/\([a-zA-Z0-9-]*\)-[DH]$/\1/g' | xargs exim -Mrm
235Note that the above only delves into /var/spool/exim in order to grep for queue files with the given string, and that's just because exiqgrep doesn't have a feature to grep the actual bodies of messages. If you are deleting these files directly, YOU ARE DOING IT WRONG! Use the appropriate exim command to properly deal with the queue.
236
237If you have to feed many, many message-ids (such as the output of an `exiqgrep -i` command that returns a lot of matches) to an exim command, you may exhaust the limit of your shell's command line arguments. In that case, pipe the listing of message-ids into xargs to run only a limited number of them at once. For example, to remove thousands of messages sent from joe@example.com:
238
239root@localhost# exiqgrep -i -f '<joe@example.com>' | xargs exim -Mrm
240Speaking of "DOING IT WRONG" -- Attention, CPanel forum readers
241I get a number of hits to this page from a link in this post at the CPanel forums. The question is:
242
243Due to spamming, spoofing from fields, etc., etc., etc., I am finding it necessary to spend more time to clear the exim queue from time to time. [...] what command would I use to delete the queue
244The answer is: Just turn exim off, because your customers are better off knowing that email simply isn't running on your server, than having their queued messages deleted without notice.
245
246Or, figure out what is happening. The examples given in that post pay no regard to the legitimacy of any message, they simply delete everything, making the presumption that if a message is in the queue, it's junk. That is total fallacy. There are a number of reasons legitimate mail can end up in the queue. Maybe your backups or CPanel's "upcp" process are running, and your load average is high -- exim goes into a queue-only mode at a certain threshold, where it stops trying to deliver messages as they come in and just queues them until the load goes back down. Or, maybe it's an outgoing message, and the DNS lookup failed, or the connection to the domain's MX failed, or maybe the remote MX is busy or greylisting you with a 4xx deferral. These are all temporary failures, not permanent ones, and the whole point of having temporary failures in SMTP and a mail queue in your MTA is to be able to try again after awhile.
247
248Exim already purges messages from the queue after the period of time specified in exim.conf. If you have this value set appropriately, there is absolutely no point in removing everything from your queue every day with a cron job. You will lose legitimate mail, and the sender and recipient will never know if or why it happened. Do not do this!
249
250If you regularly have a large number of messages in your queue, find out why they are there. If they are outbound messages, see who is sending them, where they're addressed to, and why they aren't getting there. If they are inbound messages, find out why they aren't getting delivered to your user's account. If you need to delete some, use exiqgrep to pick out just the ones that should be deleted.
251
252Reload the configuration
253After making changes to exim.conf, you need to give the main exim pid a SIGHUP to re-exec it and have the configuration re-read. Sure, you could stop and start the service, but that's overkill and causes a few seconds of unnecessary downtime. Just do this:
254
255root@localhost# kill -HUP `cat /var/spool/exim/exim-daemon.pid`
256You should then see something resembling the following in exim_mainlog:
257
258pid 1079: SIGHUP received: re-exec daemon
259exim 4.52 daemon started: pid=1079, -q1h, listening for SMTP on port 25 (IPv4)
260Read The Fucking Manual
261The Exim Home Page
262
263Documentation For Exim
264
265The Exim Specification - Version 4.5x
266
267Exim command line arguments