· 7 years ago · Jan 04, 2019, 01:14 PM
1From comp.lang.lisp Thu Oct 26 16:06:28 1995
2Newsgroups: comp.lang.lisp
3Path: edcogsci!jeff
4From: jeff@cogsci.ed.ac.uk (Jeff Dalton)
5Subject: Revised Pitfall list
6Message-ID: <DH17Jy.GrH@cogsci.ed.ac.uk>
7Organization: Centre for Cognitive Science, Edinburgh, UK
8Date: Thu, 26 Oct 1995 01:16:43 GMT
9Lines: 466
10
11I've added a number of new items and revised others to make
12the presentation clearer or to fix mistakes. The result is
13a bit long, and I'm considering ways to make it easier to deal
14with. Suggestions are welcome.
15
16----------------------------------------------------------------------
17Common Lisp Pitfalls
18
19Last updated: Thu Oct 26 00:56:33 1995 by Jeff Dalton
20
21This is a list of "pitfalls": little traps that can catch even
22experienced programmers. They often involve somewhat counterintuitive
23aspects of Common Lisp that tend to be revealed only by a careful
24reading of CLtL (or of the ANSI standard). However, pitfalls do not
25necessarily represent defects in the language.
26
27The list was written by Jeff Dalton <J.Dalton@ed.ac.uk>. Please
28send suggestions and corrections. Even the best Lisp books can
29fall into pitfalls or get some of these issues wrong. That makes
30it more important for lists like this to be correct, and for that
31I need your help.
32
33Another pitfall list, containing parts of this one, can be found in
34the "Frequently Asked Questions" list of Comp.lang.lisp.
35
36Contents:
37 * Assorted pitfalls
38 * Sort pitfalls
39 * Function vs eq
40 * Iteration vs closures
41 * Limits
42 * Definitions and declarations
43 * Packages
44 * CLOS
45 * Acknowledgments
46
47All references to CLtL are to the 2nd edition.
48
49Assorted pitfalls.
50
51 * The result of many non-destructive functions such as REMOVE
52 and UNION can share structure with an argument; so you can't
53 rely on them to make a completely new list.
54
55 * APPEND copies all of its arguments _except the last_.
56 CONCATENATE copies all of its (sequence) arguments.
57
58 * SORT is (usually) destructive.
59
60 So, for instance, (SORT (REMOVE ...) ...) may not be safe.
61
62 * Destructive functions that you think would modify CDRs might
63 modify CARs instead. (Eg, NREVERSE.)
64
65 * The value of a &REST parameter might not be a newly constructed
66 list. (Remember that the function might be called using APPLY,
67 and so an existing list might be available.) Therefore it is not
68 safe to modify the list, and the following is _not_ equivalent to
69 the LIST function: (lambda (&rest x) x). [See CLtL p 78, which is
70 not as clear as it might be.]
71
72 * Many of the functions that treat lists as sets don't guarantee
73 the order of items in the result: UNION, INTERSECTION, SET-DIFFERENCE,
74 SET-EXCLUSIVE-OR, and their destructive "N" versions.
75
76 * REMOVE- and DELETE-DUPLICATES keep the _later_ (in the sequence)
77 of two matching items. To keep the earlier items, use :FROM-END T.
78 Remembering that :FROM-END exists may make it easier to remember
79 the default behavior.
80
81 * Array elements might not be initialized to NIL. Eg,
82
83 (make-array 10) => #(0 0 0 0 0 0 0 0 0 0)
84
85 Use (make-array 10 :initial-element nil).
86
87 * READ-FROM-STRING has some optional arguments before the
88 keyword parameters. If you want to supply some keyword
89 arguments, you have to give all of the optional ones too.
90
91 Other functions with this property: WRITE-STRING, WRITE-LINE,
92 PARSE-NAMESTRING.
93
94 * EQUAL compares both vectors and structs with EQ. EQUALP descends
95 vectors and structs but has other properties (such as ignoring
96 character case) that may make it inappropriate. EQUALP does not
97 descend instances of STANDARD-CLASSes.
98
99 * EQ may return false for numbers and characters even when they seem
100 to be provably the same. The aim is to allow a greater range of
101 optimizations, especially in compiled code, by not requiring that
102 numbers and characters behave like proper objects-with-identity.
103 CLtL p 104 gives an extreme example: (let ((x 5)) (eq x x)) might
104 return false.
105
106 * Some Common lisp operators use EQ, rather than the usual EQL,
107 in a way that cannot be overridden: CATCH, GET, GET-PROPERTIES,
108 GETF, REMF, REMPROP, and THROW. See table 5-11 on p 5-57 of the
109 standard.
110
111 * The function LIST-LENGTH is not a faster, list-specific version
112 of the sequence function LENGTH. It is list-specific, but it's
113 slower than LENGTH because it can handle circular lists.
114
115 * (FORMAT T ...) interprets T as *STANDARD-OUTPUT*
116 All other I/O functions interpret T as *TERMINAL-IO*.
117
118 * COERCE will not perform all of the conversions that are available
119 in the language. There are good reasons for that, but some cases
120 may be surprising. For instance, COERCE will convert a single-character
121 string to a character, but it will not convert a character to a
122 single-character string. For that you need STRING.
123
124 * The DIRECTORY function returns the TRUENAME of each item in the
125 result, which can be slow. If you don't need the truenames, on
126 some systems it's faster to run "/bin/ls" and read its standard
127 output (if your Lisp supports this).
128
129 * The following trick for intercepting all unwinds is not portable
130 and might not be allowed at all:
131
132 (tagbody (unwind-protect (do-some-stuff)
133 ;; Intercept all unwinds, ha ha!
134 (go where-I-say))
135 where-I-say
136 ;; We always get here.
137 (do-some-more-stuff))
138
139 Informally: once an unwind to a certain point has begun, it must
140 always go at least that far. [See CLtL pages 189-192.] Similar
141 tricks using THROW or RETURN-FROM suffer the same fate. I used
142 GO above because I think that makes the intention clearer.
143
144 * Remember that there are "potential numbers" and that they are
145 reserved tokens [CLtL pages 516-520]. Normally, only odd things
146 such as 1.2.3 or 3^4/5 are "potnums", but when *READ-BASE* is
147 greater than 10, many more tokens are effected. (For real fun,
148 set your read base to 36.)
149
150SORT pitfalls.
151
152 It may seem odd to have a whole section devoted to SORT, but I've
153 seen so many erroneous SORT calls that I've decided it's necessary.
154
155 * As mentioned above, SORT is (usually) destructive and so such
156 combinations as (SORT (REMOVE ...) ...) may not do as you expect
157 (if what you expect is that REMOVE will produce a new list).
158
159 * SORT may not return equal elements in the same order in different
160 CL implementations, because different sorting algorithms are used.
161 To ensure that you get the same result in all implementations,
162 when that's what you want, use STABLE-SORT (or write your own).
163
164 * The comparison predicate given to SORT is treated as a strict
165 "less than" and so should return NIL when its 1st argument is
166 considered greater than _or equal to_ its 2nd.
167
168 * If the comparison predicate (strictly speaking, the combination
169 of the predicate and the key function) does not consistently express
170 a total order on the items being sorted, then the items "will be
171 scrambled in some unpredictable way" [CLtL p 408].
172
173 It's easy to go wrong when writing "lexicographic" multi-step
174 comparisons and end up with a predicate that says both A < B
175 and B < A for some A and B. For example:
176
177 (defun fg-lessp (a b)
178 (or (< (f a) (f b)) ;first compare f values
179 (< (g a) (g b)))) ;then compare g values
180
181 Fg-lessp does the right thing when (f a) is less than (f b):
182 it returns true -- and when (f a) is equal to (f b): it goes
183 on to compare g values. But it also compares g values when
184 (f a) is _greater than_ (f b), when what it should do is return
185 false.
186
187 Also make sure the predicate is _transitive_. For instance,
188 if you're comparing objects of different types and you decide
189 (for example) that numbers are < strings and strings are < symbols,
190 make sure the predicate says numbers are < symbols too.
191
192 * CLtL suggests that using the :KEY argument to SORT may be more
193 efficient than calling the key function inside the comparison
194 predicate (p 408). However, it may well be less efficient.
195 Implementations may not take advantage of the separate :KEY
196 to extract the keys only once; and the key function might be
197 compiled in-line when it appears inside the predicate.
198
199Function vs eq.
200
201 This issue will not affect many programs, but it can come up when
202 writing certain things that would seem natural in Scheme. The next
203 section discusses some related issues that are more likely to matter.
204 [For the Scheme account of these issues, see the R4RS section 4.1.1,
205 Lambda expressions, (especially the final sentence) and section 6.2
206 under EQV?.]
207
208 * A FUNCTION special form may construct a new function object each
209 time it's evaluated, and therefore even (flet ((f ...)) (eq #'f #'f))
210 can return false. The same is true with LABELS instead of FLET
211 and for FUNCTION used with the name of a global function.
212 However, function objects are proper objects-with-identity, and
213 (let ((f #'(lambda ...))) (eq f f)) will always return true.
214
215 It could be argued that the LET and FLET examples above should
216 be equivalent, that the difference is only syntactic. Scheme
217 programmers, and others, may well _expect_ them to be equivalent
218 (I know I did). But they (we) would be wrong.
219
220 Whether SYMBOL-FUNCTION can also construct a new object each time
221 it's called is not clear. Moreover, I haven't find explicit permission
222 in the standard for the behavior of FUNCTION described above. The
223 only explicit justification I've ever seen cited is CLtL p 118:
224
225 ... a perfectly valid implementation might simply cause every
226 distinct evaluation of a FUNCTION form to produce a new closure
227 object not EQ to any other.
228
229 In context, it's far from clear that this is _meant_ to give such
230 general permission. All of the examples in the section involve
231 FUNCTION of a LAMBDA-expression and an issue [see the 2nd item
232 in the next Pitfall section below] that would also arise for Scheme.
233 However, e-mail discussion within X3J13 showed substantial support
234 for the view that FUNCTION should be allowed to always create a new
235 object.
236
237Iteration vs closures.
238
239 * DO and DO* update the iteration variables by assignment; DOLIST and
240 DOTIMES are allowed to use assignment (rather than a new binding).
241 (All CLtL says of DOLIST and DOTIMES is that the variable "is
242 bound" which has been taken as _not_ implying that there will be
243 separate bindings for each iteration.)
244
245 Consequently, if you make closures over an iteration variable
246 in separate iterations, they may nonetheless be closures over
247 the same binding and hence will all find that the variable has
248 the same value -- whatever value the variable was given last. Eg,
249
250 (let ((fns '()))
251 (do ((x 1 (1+ x)))
252 ((= x 3))
253 (push #'(lambda () x)
254 fns))
255 (mapcar #'funcall (reverse fns)))
256
257 returns (3 3), not (1 2). However, it would return (1 2) if
258 #'(lambda () x) were replaced by (let ((x x)) #'(lambda () x)).
259
260 * A related, but distinct, question is whether a loop such as the
261 one above creates two different (ie, not EQ) function objects from
262 #'(lambda () x) or only one. On this second question, see CLtL
263 pages 117-118. For the example that returns (3 3) above, where
264 -- perhaps contrary to the author's intention -- the functions
265 would be "behaviorally identical with respect to invocation",
266 implementations are allowed to create only a single function object.
267
268 This rule appears to apply only to "distinct evaluations of the
269 same function form" [CLtL p 118], not to behaviorally identical
270 functions in general. However, it applies even if the functions
271 are closures over different bindings.
272
273Limits.
274
275 You should be aware that certain limits exist, even if you think
276 your code won't have to take them into account.
277
278 * The array-total-size-limit may be as small as 1024.
279
280 * Such things as (apply #'append list-of-lists) to flatten a list
281 of lists may run afoul of call-arguments-limit, which can be as
282 small as 50. In GCL 1.1 (to pick an implementation I have handy),
283 it's 64; so fairly low values can occur.
284
285 Alternatives include:
286
287 (reduce #'append list-of-lists :from-end t)
288
289 (mapcan #'copy-list list-of-lists)
290
291Definitions and declarations.
292
293 * (DEFVAR var init) assigns to the variable only if it does not
294 already have a value. So if you edit a DEFVAR in a file and
295 reload the file only to find that the value has not changed,
296 this is the reason. (Cf DEFPARAMETER.)
297
298 * DEFCONSTANT has several potentially unexpected properties:
299
300 - Once a name has been declared constant, it cannot be used a
301 the name of a local variable (lexical or special) or function
302 parameter. Really. See page 87 of CLtL II.
303
304 - A DEFCONSTANT cannot be re-evaluated (eg, by reloading the
305 file in which it appears) unless the new value is EQL to the
306 old one.
307
308 Note that the EQL rule makes it difficult to use anything
309 other than numbers, symbols, and characters as constants,
310 especially given the "or both" in the next subitem.
311
312 - When compiling (DEFCONSTANT name form) in a file, the form
313 may be evaluated at compile-time, load-time, or both.
314
315 (You might think it would be evaluated at compile-time and
316 the _value_ used to obtain the object at load-time, but it
317 doesn't have to work that way.)
318
319 * Internal DEFUNs define global functions, not local ones.
320 Scheme programmers, in particular, may expect otherwise.
321 When the outermost form is not a DEFUN (or, presumably, a
322 DEFMETHOD or DEFGENERIC), some implementations won't compile
323 it (or the function definitions it contains).
324
325 * CLtL says that type declarations apply to names rather than to
326 particular variables (bindings). Consequently, they can cross
327 lexical scopes. Eg:
328
329 (let ((x 1))
330 (declare (fixnum x))
331 (let ((x ...))
332 ;; This x must be a fixnum too!
333 ...))
334
335 There is some doubt that this is what Common Lisp was meant to do,
336 and (so far as I can tell) the ANSI standard disagrees with CLtL;
337 but that is what CLtL II says on page 219. On my reading of the
338 standard, the fixnum declaration would apply only to the outer X,
339 which is what lexical scope ought to imply. But given that CLtL
340 explicitly says otherwise, it's possible that some implementations
341 have been misled.
342
343 * You often have to declare the result type to get the most
344 efficient arithmetic. Eg,
345
346 (the fixnum (+ (the fixnum e1) (the fixnum e2)))
347
348 rather than
349
350 (+ (the fixnum e1) (the fixnum e2))
351
352 * When calling a function on more than two arguments, remember that
353 there can be intermediate results. For instance, when adding three
354 fixnums, it's not enough to say only that the _final_ result
355 is a fixnum. You'll have to break the computation down into
356 2-argument calls.
357
358 * Declaring the iteration variable of a DOTIMES to have type FIXNUM
359 does not guarantee that fixnum arithmetic will be used. That is,
360 implementations that use fixnum-specific arithmetic in the presence
361 of appropriate declarations may not think _this_ declaration is
362 sufficient. It may help to declare that the limit is also a
363 fixnum, or you may have to write out the loop as a DO and add
364 appropriate declarations for each operation involved. (This calls
365 for a macro.)
366
367 * Remember that implementations often use type declarations for
368 optimization rather than for checking. Adding type declarations
369 can _reduce_ the number of checks. Different implementations do
370 different things, and their behavior may be affected by OPTIMIZE
371 declarations.
372
373 * PROCLAIM vs DECLAIM: compilers are not required to process top-level
374 calls to PROCLAIM at compile time [CLtL p 223]. This is supposedly a
375 "clarification", rather than a change in the language. To ensure an
376 effect at compile time, use DECLAIM or put EVAL-WHEN around PROCLAIM.
377 You'll probably want (EVAL-WHEN (COMPILE LOAD EVAL) (PROCLAIM ...)),
378 or whatever the new-style equivalent is [see CLtL pages 88-94].
379
380 * Compile-time side-effects of defining macros (DEFVAR, DEFUN, DECLAIM,
381 etc.) may or may not be visible in the interpreter or in later calls
382 to COMPILE or COMPILE-FILE [CLtL p 689]. That's one reason why
383 files are often loaded right after being compiled, when a sequence
384 of files is being compiled using COMPILE-FILE. But that practice
385 has its own dangers since it might, for instance, (re)define
386 functions. That's one reason why you're sometimes advised to put
387 macros and some other kinds of definitions in separate files.
388
389 * INLINE and NOTINLINE [see CLtL, pages 229-230]
390
391 - You should place an INLINE proclamation _before_ the definition
392 of a function you want to have compiled inline.
393
394 - If you want a function to be inline only locally, when there's
395 an INLINE declaration, but not everywhere, you still need an
396 INLINE proclamation. CLtL p 230 recommends:
397
398 (declaim (inline gnards))
399 (defun gnards ...)
400 (declaim (notinline gnards))
401
402 - You need a NOTINLINE declaration to make sure SETF of SYMBOL-
403 FUNCTION will affect calls between functions defined in the
404 same file.
405
406 - Implementations aren't required to compile anything inline
407 no matter what declarations you use.
408
409Packages
410
411 * During the standardization of Common Lisp, the language was changed
412 in a number of ways that affect package operations. Many existing
413 programs will break, if the new rules are strictly applied. However,
414 some implementations have not fully adopted the new rules, and some
415 implementations still support old-style programs in one way or another.
416 This item briefly describes some of the changes.
417
418 - IN-PACKAGE no longer creates the package if it does not already
419 exist. Moreover, IN-PACKAGE is now a macro rather than a function.
420
421 - In general, only macros and special forms have compile-time effects.
422 That's one reason why IN-PACKAGE is now a macro. However, other
423 package operations such as IMPORT, EXPORT, and USE-PACKAGE are
424 still functions. To get the desired effects at compile time,
425 use EVAL-WHEN or put the function calls in a file that you
426 tell the compiler to load. Use DEFPACKAGE when you can.
427
428 - Instead of the LISP and USER packages, there are packages called
429 COMMON-LISP and COMMON-LISP-USER.
430
431 - There are a number of restrictions on how "conforming programs"
432 can use symbols in the COMMON-LISP package. See CLtL pages 259-261
433 or section 11.1.2.1.2, pages 11-5 and 11-6 of the ANSI standard.
434
435 * (EXPORT NIL) does not export NIL. You have to use (EXPORT '(NIL)).
436
437 * If you have a package named P that exports a symbol named P, you
438 won't (in another package) be able to say (USE-PACKAGE 'P). Instead
439 of 'P you'll have to write "P" or maybe #:P. ["Why?" is left as
440 an exercise for the reader.]
441
442 * The best order for package operations is _not_ that of the "Put in
443 seven extremely random user interface commands" mnemonic [CLtL p 280].
444 Instead, it's the order used by DEFPACKAGE [p 271]. The change is to
445 put EXPORT last. Of course, it's often better to just use DEFPACKAGE.
446
447CLOS
448
449 * Shared (i.e., class-allocated) slots of standard classes are not
450 per-(sub)class. That is, if a class C has a slot S with :ALLOCATION
451 :CLASS, that one slot is used by all instances of C or of subclasses
452 of C, except for subclasses where the slot has been "shadowed"
453 [See CLtL p 779]. To get a per-class slot, you have to explicitly
454 define it for each class.
455
456 * Don't forget, with :AROUND methods, that other :AROUND methods
457 might execute around them. (E.g., an :AROUND method that
458 recorded run time might not be the outermost one.) A similar
459 point applies to :BEFORE and :AFTER methods.
460
461 * Don't forget that a method specialized to class C can be called
462 on instances of subclasses of C, or that such a subclass may have
463 ancestors that have no other relation to C.
464
465Acknowledgments
466
467 Andrew Philpot <philpot@isi.edu> for pointing out that CLtL p 408
468 limits the consequences of giving SORT a predicate that does not
469 consistently represent a total order.
470
471 Thorsten Schnier <thorsten@Rincewindarch.su.edu.au> for reminding
472 me that (the ...) does not imply a type check.
473
474End
475
476----------------------------------------------------------------------