Common Lisp and ASDF System Testing

Question

I am wondering how one runs tests using ASDF. Suppose we have the following system definition. What command would I run to run the tests?

I have tried running things like (asdf:compile-system cl-mongo) and (asdf:load-system cl-mongo) but it does not become apparent how to test. If it helps, the code comes from this repository.

Should I somehow be compiling and loading the test system as well?

   (in-package #:cl-user)

(defpackage #:cl-mongo-system (:use #:cl #:asdf))

(in-package #:cl-mongo-system)

(asdf:defsystem #:cl-mongo
  :name   "cl-mongo"
  :author "Fons Haffmans; [email protected]"
  :version "0.7"
  :licence "MIT"
  :description "lisp system to interact with mongodb, a non-sql db"
  :depends-on (:uuid
           :babel
           :bordeaux-threads
           :documentation-template
           :lisp-unit
           :parenscript
           :split-sequence
           :usocket)
  :serial t
  :components 
 ((:module "src"
    :serial t
    :components ((:file "packages")
         (:file "octets")
         (:file "pair")
         (:file "encode-float")
         (:file "bson-oid")
         (:file "bson-binary")
         (:file "bson-time")
         (:file "bson-regex")
         (:file "bson-code")
         (:file "bson")
         (:file "bson-decode")
         (:file "bson-array")
         (:file "document")
         (:file "mongo-syntax")
         (:file "java-script")
         (:file "bson-encode-container")
         (:file "protocol")
         (:file "mongo")
         (:file "db")
         (:file "mem")
         (:file "do-query")
         (:file "doc")
         (:file "map-reduce")
         (:file "shell")))
   (:static-file "README.md")
   (:static-file "COPYING")))

(asdf:defsystem #:cl-mongo-test
  :name   "cl-mongo"
  :author "Fons Haffmans; [email protected]"
  :version "0.7"
  :licence "MIT"
  :description "testing cl-mongo"
  :depends-on (:cl-mongo)
  :serial t
  :components 
  ((:module "test"
    :serial t
    :components ((:file "package")
         (:file "test-utils")
         (:file "regression")))))

As requested, I am including the error I receive when I run (asdf: compile-system 'cl-mongo-test)

COMPILE-FILE-ERROR while
compiling #<CL-SOURCE-FILE "cl-mongo-test" "test" "regression">
   [Condition of type UIOP/LISP-BUILD:COMPILE-FILE-ERROR]

Restarts:
 0: [RETRY] Retry compiling #<CL-SOURCE-FILE "cl-mongo-test" "test" "regression">.
 1: [ACCEPT] Continue, treating compiling #<CL-SOURCE-FILE "cl-mongo-test" "test" "regression"> as having been successful.
 2: [RETRY] Retry ASDF operation.
 3: [CLEAR-CONFIGURATION-AND-RETRY] Retry ASDF operation after resetting the configuration.
 4: [RETRY] Retry SLIME REPL evaluation request.
 5: [*ABORT] Return to SLIME's top level.
 --more--

Backtrace:
  0: (UIOP/LISP-BUILD:CHECK-LISP-COMPILE-RESULTS NIL T T "~/asdf-action::format-action/" ((#<ASDF/LISP-ACTION:COMPILE-OP > . #<ASDF/LISP-ACTION:CL-SOURCE-FILE "cl-mongo-test" "test" "regression">)))
  1: ((SB-PCL::EMF ASDF/ACTION:PERFORM) #<unavailable argument> #<unavailable argument> #<ASDF/LISP-ACTION:COMPILE-OP > #<ASDF/LISP-ACTION:CL-SOURCE-FILE "cl-mongo-test" "test" "regression">)
  2: ((:METHOD ASDF/ACTION:PERFORM-WITH-RESTARTS :AROUND (T T)) #<ASDF/LISP-ACTION:COMPILE-OP > #<ASDF/LISP-ACTION:CL-SOURCE-FILE "cl-mongo-test" "test" "regression">) [fast-method]
  3: ((:METHOD ASDF/PLAN:PERFORM-PLAN (LIST)) ((#1=#<ASDF/LISP-ACTION:COMPILE-OP > . #2=#<ASDF/LISP-ACTION:CL-SOURCE-FILE #3="cl-mongo" #4="src" "shell">) (#5=#<ASDF/LISP-ACTION:LOAD-OP > . #2#) (#1# . #6=..
  4: ((FLET SB-C::WITH-IT :IN SB-C::%WITH-COMPILATION-UNIT))
  5: ((:METHOD ASDF/PLAN:PERFORM-PLAN :AROUND (T)) ((#1=#<ASDF/LISP-ACTION:COMPILE-OP > . #2=#<ASDF/LISP-ACTION:CL-SOURCE-FILE #3="cl-mongo" #4="src" "shell">) (#5=#<ASDF/LISP-ACTION:LOAD-OP > . #2#) (#1# ..
  6: ((FLET SB-C::WITH-IT :IN SB-C::%WITH-COMPILATION-UNIT))
  7: ((:METHOD ASDF/PLAN:PERFORM-PLAN :AROUND (T)) #<ASDF/PLAN:SEQUENTIAL-PLAN {10049ED323}>) [fast-method]
  8: ((:METHOD ASDF/OPERATE:OPERATE (ASDF/OPERATION:OPERATION ASDF/COMPONENT:COMPONENT)) #<ASDF/LISP-ACTION:COMPILE-OP > #<ASDF/SYSTEM:SYSTEM "cl-mongo-test">) [fast-method]
  9: ((SB-PCL::EMF ASDF/OPERATE:OPERATE) #<unused argument> #<unused argument> #<ASDF/LISP-ACTION:COMPILE-OP > #<ASDF/SYSTEM:SYSTEM "cl-mongo-test">)
 10: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE))
 11: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (T T)) #<ASDF/LISP-ACTION:COMPILE-OP > #<ASDF/SYSTEM:SYSTEM "cl-mongo-test">) [fast-method]
 12: ((SB-PCL::EMF ASDF/OPERATE:OPERATE) #<unused argument> #<unused argument> ASDF/LISP-ACTION:COMPILE-OP CL-MONGO-TEST)
 13: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE))
 14: (ASDF/CACHE:CALL-WITH-ASDF-CACHE #<CLOSURE (LAMBDA NIL :IN ASDF/OPERATE:OPERATE) {10049CF6AB}> :OVERRIDE NIL :KEY NIL)
 15: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (T T)) ASDF/LISP-ACTION:COMPILE-OP CL-MONGO-TEST) [fast-method]
 16: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (T T)) ASDF/LISP-ACTION:COMPILE-OP CL-MONGO-TEST) [fast-method]
 17: (ASDF/OPERATE:COMPILE-SYSTEM CL-MONGO-TEST)
 18: (SB-INT:SIMPLE-EVAL-IN-LEXENV (ASDF/OPERATE:COMPILE-SYSTEM (QUOTE CL-MONGO-TEST)) #<NULL-LEXENV>)
 19: (EVAL (ASDF/OPERATE:COMPILE-SYSTEM (QUOTE CL-MONGO-TEST)))
 --more--

Answer 1

When I create a new project with cl-project the asdf definition contains a :in-order-to ((test-op …)) clause:

(asdf:defsystem :foo
   :author ""
    …
   :in-order-to ((test-op (test-op :foo.test))))

and my foo.test.asdf:

(asdf:defsystem :foo.test
  :author ""
   …
  :perform (test-op (op system)
             (funcall (read-from-string "prove:run")
                      (system-relative-pathname :foo.test "test/"))))

To run tests, we can execute asdf:test-system or (in this case, using prove prove:run in your REPL.

• https://lispcookbook.github.io/cl-cookbook/testing.html

Answer 2

See the asdf best practices document: https://gitlab.common-lisp.net/asdf/asdf/blob/master/doc/best_practices.md