Saturday, November 15, 2008

Language Vulnerabilities

Cruel Fate has lately had me doing a great deal of work in ColdFusion. Comparing this with my UCW/Clsql experience has really driven home to me why there are so many html and sql injection vulnerabilities out there on the web. ColdFusion strongly encourages you to just spit whatever was passed to you back out to the user, and practically punishes you for safely constructing sql strings. In UCW and Clsql, on the other hand, you had to actively try in order to open yourself up to an injection attack on either front.

This got me thinking about how other languages tend to lend themselves to certain vulnerabilties. We're all aware of buffer overflow exploits; these are due almost entirely to the fact that C/C++ force you to waste time managing memory. Such things are virtually non-existent in languages like Lisp or even Java.

So what kinds of vulnerabilities does Lisp usage tend to open you up to? The only thing I can think of are the potential hazards involved in using read. As we had to keep in mind when developing Paragent, you absolutely must set *read-eval* to nil wherever you use read. Otherwise, you are at great risk from the #. macro, which will essentially allow an attacker to (easily) run any code they want on your machine. Happily, it's easy enough to avoid once you're aware of it.

Beyond that, I'm at a loss. Can anybody else think of some security issues that are particular to Common Lisp?

Friday, September 12, 2008

Making an Executable

Ostia Antica, harbor city of ancient Rome Not infrequently, some poor lisp newb will ask how to make an executable in lisp. The answer of course, is "what do you mean by 'executable?'" If the newb is lucky, somebody will eventually take break from pretending not to know exactly what he means and point him to save-lisp-and-die. But, they will hasten to add, making an executable is really a silly thing to do, so why bother?

Fresh from the mind of Sergey Kolos, Cusp contributor extraordinaire and the man who recently brought us integrated unit testing comes this wild innovation: a menu item called "Create Exe." That's right. In the next version, all Cusp users will have to do is right-click on their project and select "Create Exe." That, my friends, is what user-friendliness looks like.

So buck up, newbs. With luck, the endless discussions on what the meaning of the word "is" is will soon be replaced by the simple advice to "use Cusp."

I wouldn't hold my breath, though.

Monday, August 25, 2008

Making a small Common Lisp Project (in Cusp)

In honor of my recent addition to Planet Lisp, and as a special thank you to Xach, I figured I'd counter-blog his old post on how to make a lisp project (in Slime). You can go read that if you want. Here's the Cusp way:

Make the project. Go to New > Lisp Project. Enter the name of your project. You're done.

Write the code. You'll notice that a file named "main.lisp" has been opened for you. Put your lisp code there. You can compile the definitions as you write them (with Alt+C), but that's mostly a vestigial holdover from Slime. Cusp is smart. Save your file and it will automatically compile any new or changed top-level forms. (It'll even undefine functions that you've deleted.)

Later on... If you close Cusp, when you come back you'll need to reload the project. To do this, right-click on the .asd file in the project, and choose "Load Project." You may also need to change the package in the REPL to match the one for this project.

So what is all this stuff?

You've got all you need to start hacking, but you probably want to know what's going on behind the scenes. The New Project wizard creates a project folder with three files in it: main.lisp, defpackage.lisp, and [project name].asd. Here's what they do:

main.lisp is the main code file. Put the guts of your program here. You can, of course, create additional files as your project grows. More on that in a moment.

defpackage.lisp defines the package for this lisp project, which is basically the lisp equivalent of a namespace. You could pollute cl-user with your project internals, but that's just asking for trouble down the line. You'll note that main.lisp started out with (in-package :[project-name]). You'll want to make sure any new files you add start that way, too.

If you want to import symbols from another package into your project (so you can say split rather than cl-ppcre:split) put the name of that package into the :use section. This is similar to using in C++ or C#, or import in Java. If your project turns into a library to be used by other projects, you would put any exported symbols into the :export section. Notice that by default, you are already importing :cl.

[project-name].asd is best thought of as your project file. This tells Lisp what libraries to load up for your project, and what files are part of your project. You'll notice that defpackage and main are already listed in the :components section. If you add more files to your project, make sure to tack them on to the end of that list. Files will be loaded in the order they are listed.

Dependencies are also handled by the asd file. If, as in the earlier example, you want to use cl-ppcre, make sure to add it to the :depends-on section.

Asd files can get more complicated if you need, but this should be enough to take care of you in 95% of all cases. Happy hacking!

Monday, August 18, 2008

Unit Tests (Alpha)

Some of the other developers are currently working on integrating unit testing into Cusp. It's a project I'm not personally involved in, but I'm rather excited about. Up until now, the bulk of Cusp development has consisted of putting existing Slime functionality into a more sane and usable UI. With this one, we'll be breaking entirely new ground.

Things are still in the very early stages, so if you want to contribute code, ideas, or just see how the current plans are shaping up, you can check out the thread.

In which I mention that I like macros

Lest this blog be limited only to complaining about Lisp's shortcomings, allow me to take moment to sing the praises of macros, one of the greatest language features that's probably not even possible in other syntaxes.

Recently, I had to write a program to scrape a bunch of data off of a few thousand web pages. My general pattern was to skip forward until I found a certain string, then skip forward a bit more and grab whatever was between certain tags. It was also important to leave the cursor at the end of wherever I had searched, so that the next bit of data I scraped out would start looking at that point. This particular mutation made a function not terribly well suited to the task.

This came out to roughly ten lines of code per field searched for. But it was trivial to put those into a macro, and turn all my scrapes into something along the lines of (after-and-between phone "Phone:" "<b>" "<br").

The macro facility is one I especially miss when I'm programming Cusp (in Java, naturally). In the SwankInterface class, I have several dozen functions which all start with identical code to register a callback, and all end by calling a function to send a string to Swank. It's only 2 lines per function, but now that I'm used to Lisp, those lines annoy me. It's wasteful, and what if I have to change them someday?

Just one of the ways in which Lisp ruins other languages for you. CLOS makes things even worse.

Tuesday, August 12, 2008

The Library Problem

Today, I got a new task which involved scraping some web pages. Obviously, as part of this I needed a library to download the pages in question, so I asked around1 and was pointed to Drakma. It's Weitz-ware, so you can count on it to be reliable and well-documented, so I downloaded it.

Trying to load it did not go so well. I discovered that I needed a few dependencies. Some I had already, but I still had to go back and download Chunga (oddly, Wietz said to get 0.5.0 or higher, even though it's only up to 0.4.3, and he's the one who wrote Chunga2), and usocket, and a newer puri, and a newer cl-base64, and CL+SSL. CL+SSL was especially fun, because it's one of those libraries that says to pull it from cvs rather than use the tarball. Alas, the cvs command they provided utterly failed to work for me, so I went with the tarball anyway.

Once I had all those unzipped into my libraries folder, I tried again. FAILURE. It turns out that these dependencies had dependencies of their own. CFFI seemed to be called for. So I went to the CFFI download page, got it, and found that it had three more dependencies of its own.

Babel and trivial-features were easy enough downloads, but then I got to Alexandria. Alexandria has not yet made a real release. You have to pull it from the repository. But it's even better than that, because the repository is darcs, a version control system that pretty much nobody uses. In my case, I had to first install darcs just so I would be able to get this dependency that was 3 or 4 levels removed from the library I was actually interested in.

All told, it looks like I downloaded at least 9 additional libraries, one of which involved installing special software I have no need for. So Common Lisp definitely does have a library problem, but it's not the one a lot of people think it is. I've always been able to find a library that provides me with the functionality I need. The library problem lies in the fact that I need to download 20 other libraries, often through their versioning systems, in order to get that one to work.

Happily, this is a problem I'm in a good position to help with, since I'm already distributing a ready-to-use lisp system. As you may have noticed, Cusp comes with a number of popular lisp libraries already installed. The "Load Package" dialog even gives descriptions of what some of them are for, thanks to Sergey's work. Obviously, though, there are a lot more libraries I still need to include. It's a wonder I put up with this sort of thing when I was a newb.

So, I think packaging libraries will be a higher priority for the next release. And if somebody could make an asdf-install for sbcl that will work on windows, they will have my hearty thanks.

1s/around/one guy
2Don't laugh, though. His documentation is still way better than yours.

Sunday, June 29, 2008

Stupid Eclipse Tricks

Overall, Eclipse is great platform to program on, and making a plugin is an experience I would recommend to any coder because it will get you acquainted with a large, complex and fairly well-architectured system.

Of course, nothing is perfect, and Eclipse has faults just like the rest of us. Living in Java-land, it tends to suffer from what is termed ravioli code, where code is scattered throughout hundreds of tiny, tiny objects. Often, to do what you want to do, you need to define one class which then configured by another class which itself may be generated by some factory object. Often, I find the complex tasks are made very easy, but it can be maddening to track down the documentation for tiny little changes I want to make.

And occasionally, poor design does manage to slip in. Take the argument list tooltip (one of the 3 essential editor components). This handy guy gives you function signatures and a little descriptive documentation. In Cusp, The signature is formatted in bold to make it quicker for your eye to pick out. Displaying the tooltip to the user with the required information was trivial. Making part of it bold, that was the hard part.

Here's a quick overview of the architecture. To get your editor to behave the way you want it to, you subclass TextSourceViewerConfiguration, which then informs the editor how it should colorize syntax, how to indent, how to automplete, etc (most of these configurations involve returning another object that does the actual work). Lest you think this is an unnecessary abstraction, it turned out to be very handy when I wanted to make the input area of the REPL behave like the editor, even though they are different beasts.

In there, we tell the editor about Arglist Assistance as follows:

ca = new ContentAssistant();
ca.setContentAssistProcessor(new ArglistAssistProcessor(editor),

IPreferenceStore ps = LispPlugin.getDefault().getPreferenceStore();
if( ps.getBoolean(PreferenceConstants.ARGLIST_BELOW)){
} else {


public IContentAssistant getContentAssistant(ISourceViewer sourceViewer) {
ca.setInformationControlCreator(new LispTextHoverControlCreator());
return ca;

As you may or may not have noticed, ArglistAssistProcessor is the class that actually figures out what information to display. It parses the document to find out what function is being called (or if the cursor is even in a place where a tooltip makes sense), then talks to Swank to get the information, and then returns that in a format Eclipse understands. The process may sound a little complex when you first run into it, but it is well-covered in the tutorials, so it's fairly easy to get to this point.

But how do I format that text for easier reading? Here, the documentation failed me. And so I'm writing up the answer as a service to the rest of you who may want to do this.

Part of the interface for IContentAssistProcessor is getContextInformationValidator(), which returns an IContextInformationValidator, the object that determines when it is time get rid of the tooltip. It turns out that to format the tooltip, you need to make this object also implement IContextInformationValidator.

This is bad design for so many reasons. Nothing in the documentation indicates that this is the place to do the formatting, and it makes no logical sense for a validator to also (sometimes) act as a formatter. It is also rather contrary to the usual Java and Eclipse way to take an object designed for a completely separate task, and, if it happens to implement another interface, use it for that as well. I'm pretty sure this was slapped on with little thought.

But now it's documented. If you're wondering how to format your tooltips in your own editor, now you know.