The Library Problem

Today, I got a new task which involved scraping some web pages. Obviously, as part of this I needed a library to download the pages in question, so I asked around¹ and was pointed to Drakma. It's Weitz-ware, so you can count on it to be reliable and well-documented, so I downloaded it.

Trying to load it did not go so well. I discovered that I needed a few dependencies. Some I had already, but I still had to go back and download Chunga (oddly, Wietz said to get 0.5.0 or higher, even though it's only up to 0.4.3, and he's the one who wrote Chunga²), and usocket, and a newer puri, and a newer cl-base64, and CL+SSL. CL+SSL was especially fun, because it's one of those libraries that says to pull it from cvs rather than use the tarball. Alas, the cvs command they provided utterly failed to work for me, so I went with the tarball anyway.

Once I had all those unzipped into my libraries folder, I tried again. FAILURE. It turns out that these dependencies had dependencies of their own. CFFI seemed to be called for. So I went to the CFFI download page, got it, and found that it had three more dependencies of its own.

Babel and trivial-features were easy enough downloads, but then I got to Alexandria. Alexandria has not yet made a real release. You have to pull it from the repository. But it's even better than that, because the repository is darcs, a version control system that pretty much nobody uses. In my case, I had to first install darcs just so I would be able to get this dependency that was 3 or 4 levels removed from the library I was actually interested in.

All told, it looks like I downloaded at least 9 additional libraries, one of which involved installing special software I have no need for. So Common Lisp definitely does have a library problem, but it's not the one a lot of people think it is. I've always been able to find a library that provides me with the functionality I need. The library problem lies in the fact that I need to download 20 other libraries, often through their versioning systems, in order to get that one to work.

Happily, this is a problem I'm in a good position to help with, since I'm already distributing a ready-to-use lisp system. As you may have noticed, Cusp comes with a number of popular lisp libraries already installed. The "Load Package" dialog even gives descriptions of what some of them are for, thanks to Sergey's work. Obviously, though, there are a lot more libraries I still need to include. It's a wonder I put up with this sort of thing when I was a newb.

So, I think packaging libraries will be a higher priority for the next release. And if somebody could make an asdf-install for sbcl that will work on windows, they will have my hearty thanks.


¹s/around/one guy
²Don't laugh, though. His documentation is still way better than yours.