How to do image-based development in Common Lisp?

Question

I am new to Common Lisp. This is how I develop programs in other languages, and also how I now develop programs in Common Lisp:

1. Open a text editor (e.g. vim or emacs) to create/edit a text file.
2. Write source code into the text file. (If unsure about the behavior of a snippet of code, and an REPL is available, then evaluate the snippet in the REPL, verify that the snippet evaluates as expected, and then go back to writing more code.)
3. Save the text file.
4. Ask the compiler/interpreter to load and run the source code in the text file. (e.g. sbcl --script myprog.lisp)
5. Go to step 1 if needed.

This is the conventional write-compile-run development cycle for most programming languages. However, in the lisp world, I hear things like "interactive development" and "image-based development", and I feel that I am missing out on an important feature of Common Lisp. How do I do "image-based development" instead of "write-compile-run development"?

Can someone provide a step-by-step example of "image-based development" similar to how I described "write-compile-run development" above?

(Note: I am using SBCL)

Answer 1

In typical Common Lisp implementations the runtime, the compiler, parts of the development environment and the program you are developing reside in the same program and share the same object space. The compiler is always available while you develop the program and the program can be incrementally developed. The development tools have access to all objects and can inspect their state. One can also undefine/remove, replace, enhance functionality from the running program.

Thus:

• don't restart the program you are developing. Stay connected and update it. Even days, weeks, or months - if possible.
• write code in such a way that the program can be replicated and built from scratch if necessary. Build it from time to time and fix any build problems.
• once you use our program and there is an error -> fix the error within the program, while being able to inspect the full error state
• creating a running program is either loading all code into a plain Lisp all the time or saving an executable image with the loaded code/data

Fixes to program bugs can also shipped to the user as compiled Lisp files, which gets loaded into the delivered program and update the code then.

Answer 2

Let's say that you are using SBCL with Emacs and SLIME (e. g. through Portacle).

1. Open Emacs
2. Start SLIME (M-x slime) — this starts a “plain” Lisp process in the background and connects the editor functions provided by slime to it; then gives you a REPL that is also connected into this process (image)
3. Open a text file (e. g. foo.lisp)
4. Type some code
5. Press C-c C-k to compile the file and load it into the running Lisp process
6. Switch to the REPL, try it out
7. Switch to the Lisp file (step 4).

This is just very basic usage. Further things to do/learn

• You can also compile and load just a single toplevel form (C-c C-c)
• Learn about packages
• Learn about systems (ASDF)
• Learn how to use Quicklisp to get the libraries you want
• Learn how to access inline documentation from the REPL

Note that you never need to unload your program, you just modify it, even when downloading and loading new libraries. This makes the feedback cycle instantaneous in most cases. You also never need to switch away from the IDE (Emacs).