--- title: "Building a REPL OS in Common Lisp, Part 5: Persistence" date: "2026-06-06" description: "Part 5 of a series where we build a custom operating system inside a Lisp REPL. We add filesystem persistence using S-expressions and enable loading code from virtual files." tags: ["lisp"] series: "Building a REPL OS in Common Lisp" part: 5 language: "en" draft: false --- *Part 5 of a series where we build a custom operating system that runs inside a Lisp REPL.* --- ## The Problem Our virtual filesystem works, but it's ephemeral. Quit the REPL, and everything is gone. Every OS needs persistence: a way to save state and restore it later. In this new chapter we'll add the ability to save the filesystem to disk and load it back on startup. We'll also add `tree` and `find` commands to visualize and search the filesystem. ## Serialization We need to convert our tree of nodes to something we can write to a file. Most languages require you to choose a format (JSON, XML, YAML) and use a library to parse and generate it. Lisp has a superpower: S-expressions. The same syntax we use for code is also a data format. And Lisp has built-in functions to read and write it: - `print` writes any Lisp object to a stream in a format that can be read back - `read` parses that format back into a Lisp object No library needed. No schema definition. No version compatibility headaches. The same reader that parses your source code parses your data files. ```lisp (defvar *fs-path* (merge-pathnames ".repl-fs.dat" (asdf:system-source-directory :repl))) (defun node-to-sexp (node) "Convert a node tree to an S-expression for serialization." (list :name (node-name node) :type (node-type node) :contents (node-contents node) :children (mapcar #'node-to-sexp (node-children node)))) (defun sexp-to-node (sexp) "Convert an S-expression back to a node tree." (make-node :name (getf sexp :name) :type (getf sexp :type) :contents (getf sexp :contents) :children (mapcar #'sexp-to-node (getf sexp :children)))) ``` Simple. `node-to-sexp` recursively converts the tree to nested property lists. `sexp-to-node` does the reverse. The data file lives in the project directory. ## Save and Load ```lisp (defun save-fs () "Save the virtual filesystem to disk." (with-open-file (out *fs-path* :direction :output :if-exists :supersede) (print (node-to-sexp *fs*) out)) (format t "~%Filesystem saved to ~A" *fs-path*)) (defun load-fs () "Load the virtual filesystem from disk." (if (probe-file *fs-path*) (progn (with-open-file (in *fs-path*) (setf *fs* (sexp-to-node (read in)))) (format t "~%Filesystem loaded from ~A" *fs-path*)) (format t "~%No saved filesystem found. Starting fresh."))) ``` `save-fs` writes the tree to `.repl-fs.dat`. `load-fs` reads it back if the file exists, or starts fresh if not. Here's what the file looks like after creating `/home` with a `notes.txt` file: ```lisp (:NAME "/" :TYPE :DIRECTORY :CONTENTS "" :CHILDREN ((:NAME "home" :TYPE :DIRECTORY :CONTENTS "" :CHILDREN ((:NAME "notes.txt" :TYPE :FILE :CONTENTS "Remember to add mounts in v6!" :CHILDREN NIL))))) ``` It's just a nested property list: the same S-expression syntax we use everywhere in Lisp. Perks of Lisp. This is the beauty of `print`/`read` as a serialization pair: - `print` outputs data in a format `read` can parse - `read` returns exactly what `print` wrote - They're inverses of each other, built into the language No JSON library, no schema, no versioning issues. If you can build it from lists, symbols, strings, and numbers, you can serialize it. This is one of Lisp's quiet superpowers. ## Executing Code from Files We can store data in files. But what about code? Imagine we want a `tree` command to visualize the filesystem. We could hardcode it in `repl.lisp`, but there's a more interesting approach: store the command definition in a file and load it. This is how real operating systems work. Commands live in `/bin` or `/usr/bin`. They're just files. We can do the same. First, we need a way to execute code from a file: ```lisp (register-command (make-command :name 'load-file :description "Load and execute a Lisp file: (load-file \"/bin/tree.lisp\")" :function (lambda (path) (let ((node (find-node path))) (if (and node (eq (node-type node) :file)) (progn (with-input-from-string (in (node-contents node)) (loop for form = (read in nil :eof) until (eq form :eof) do (dispatch form))) (format t "~%Loaded ~A" path)) (format t "~%No such file: ~A" path)))))) ``` This reads all forms from a file and passes each to `dispatch`, the same function the REPL uses. So a file can contain commands, Lisp code, or new command definitions. ## Creating User Commands Let's create a `/bin` directory and a simple command: ``` λ (mkdir "/bin") Created /bin λ (write "/bin/hello.lisp" "(register-command (make-command :name 'hello :description \"Say hello\" :function (lambda () (format t \"~%Hello from user space!\"))))") Created and wrote to /bin/hello.lisp λ (load-file "/bin/hello.lisp") Loaded /bin/hello.lisp λ (hello) Hello from user space! ``` The command we just created works. This is self-hosting: the OS can extend itself. There's a practical limitation: the REPL reads a complete form when you press Enter, so you can't easily write multi-line code. Simple one-liners work, but complex commands like `tree` would be tedious to type. I leave to the reader the pleasure of experimenting with it. The ideal solution would be to have a nice text editor (perhaps in the future). For now, the real solution is to edit files externally, which is exactly why we need mounts (Part 6). For now, we include `tree` and `find` as built-ins. ## Built-in Commands: Tree and Find Since complex commands are hard to type at the REPL, we include `tree` and `find` as built-ins: ```lisp (register-command (make-command :name 'tree :description "Show filesystem tree" :function (lambda () (labels ((print-tree (node depth) (format t "~%~A~A~A" (make-string (* depth 2) :initial-element #\Space) (node-name node) (if (eq (node-type node) :directory) "/" "")) (dolist (child (node-children node)) (print-tree child (1+ depth))))) (print-tree *fs* 0))))) ``` Remember DFS? [Depth-first search](https://en.wikipedia.org/wiki/Depth-first_search): visit a node, then recurse into each child before backtracking. We print the current node, recurse into children with increased depth. The depth controls indentation. We use `labels` to define a local recursive function. Why not `defun`? - `print-tree` is only used inside this command - No need to pollute the global namespace - The function is defined right where it's used, making the code self-contained `labels` is like `let` for functions. It creates local bindings that can reference themselves (for recursion) or each other. ```lisp (register-command (make-command :name 'find :description "Find files by name: (find \"notes\")" :function (lambda (pattern) (let ((results '())) (labels ((dfs (node path) (let ((current-path (if (string= path "") (node-name node) (format nil "~A/~A" path (node-name node))))) (when (search pattern (node-name node)) (push current-path results)) (dolist (child (node-children node)) (dfs child current-path))))) (dfs *fs* "")) (if results (dolist (path (nreverse results)) (format t "~% ~A" path)) (format t "~%No matches for ~A" pattern)))))) ``` Also DFS. Why DFS and not BFS ([breadth-first search](https://en.wikipedia.org/wiki/Breadth-first_search))? - DFS explores one branch completely before moving to the next - BFS explores all nodes at depth 1, then depth 2, etc. - DFS is simpler to implement with recursion - For "find all matches," both work. DFS just finds them in a different order BFS would require a queue to track nodes to visit. DFS uses the call stack naturally. For our use case, DFS is simpler and works fine. A few things to note: - We use `search` (a built-in) to check if `pattern` is a substring of the node name - Results are collected with `push`, which builds the list in reverse order - We use `nreverse` at the end to restore the original order (root-to-leaf) - The path is built up as we descend, so each result has its full path ## Auto-Save and Auto-Load The REPL now loads the filesystem on startup: ```lisp (defun my-repl () "Start the REPL. Type (help) for commands, (quit) to exit." (load-fs) (format t "~%Welcome to REPL OS v5") (format t "~%Type (help) for commands, or any Lisp expression.~%") (catch 'repl-exit (loop (print-prompt) (let ((form (read))) (record-history form) (handler-case (dispatch form) (error (e) (format t "~%ERROR: ~A" e))))))) ``` And `quit` auto-saves: ```lisp (register-command (make-command :name 'quit :description "Exit the REPL" :function (lambda () (save-fs) (format t "~%Goodbye.~%") (throw 'repl-exit nil)))) ``` ## Running It ```lisp (repl:my-repl) ``` ``` No saved filesystem found. Starting fresh. Welcome to REPL OS v5 Type (help) for commands, or any Lisp expression. λ (mkdir "/bin") Created /bin λ (write "/bin/greet.lisp" "(register-command (make-command :name 'greet :description \"Greet someone\" :function (lambda (name) (format t \"~%Hello, ~A!\" name))))") Created and wrote to /bin/greet.lisp λ (load-file "/bin/greet.lisp") Loaded /bin/greet.lisp λ (greet "world") Hello, world! λ (tree) // bin/ greet.lisp λ (quit) Filesystem saved to /path/to/repl/.repl-fs.dat Goodbye. ``` Start again: ``` Filesystem loaded from /path/to/repl/.repl-fs.dat Welcome to REPL OS v5 Type (help) for commands, or any Lisp expression. λ (greet "world") ERROR: Undefined function: GREET λ (load-file "/bin/greet.lisp") Loaded /bin/greet.lisp λ (greet "world") Hello, world! ``` The filesystem persisted, but the command didn't. It was only in memory. We need to reload it each session. An init file could solve this, or we could build auto-loading into the REPL. ## What We Built 1. **Serialization**: convert node tree to S-expressions and back 2. **Persistence**: save to disk, load on startup 3. **Auto-save/load**: filesystem survives restarts 4. **load-file**: execute Lisp code stored in virtual files 5. **User space**: commands can be created and stored in `/bin` This is the beginning of self-hosting. The OS can now extend itself. Users can write their own commands, store them in the filesystem, and load them when needed. We showed `tree` and `find` as examples of what user commands look like. (In the actual code, they're built-in for convenience, but they could live in `/bin`.) They use DFS (depth-first search) to traverse the filesystem tree. You could write more: `grep` to search file contents, `cp` to copy files, `mv` to rename them. ## What's Missing We can create and run code, but we can't access the real filesystem. The virtual filesystem is an island. We can't read the source code of the REPL itself. In Part 6, we'll add **mounts**: the ability to map virtual paths to real directories. `(mount "/sys" "/path/to/repl/")` will let us read and write actual files. Then we can truly modify the OS from within. --- *Next: Part 6: Mounts*