Coming up Short

Today I want to talk about two functions that are suprisingly complicated, and seriously misnamed. Those functions are the humble read and write functions.

At first glance, read and write seem to be incredibly simple. read takes data from the disk and puts it in a buffer. write does exactly the opposite. However, if you look closer, it's a lot more complicated than that.

Return codes

The first thing to realize is that read and write may handle fewer bytes than requested. So really, they should have been called try_read and try_write, or similar. Unfortunately, these "short reads" and "short writes" happen only very rarely, which means that this mistake is unlikely to show up in testing. It's a beginner's trap!

It's important to remember than a return code of 0 from read means that end-of-file has been reached. On the other hand, a return code of 0 from write is just another short write, and should be handled like all short writes.

Signals

Then there's the behavior in the presence of signals. If a thread is in the middle of a read or a write when a signal arrives, the operation may return with an error code of -1 and an errno of EINTR, meaning no bytes were read or written. It also may return one of the infamous short reads or writes.

In a sense, EINTR is just a way for read to say "I read 0 bytes." Remember, read can't return 0 in this scenario because that means end-of-file. I think there's been at least one case in the Linux kernel of read returning EINTR when no signal had, in fact, been delivered.

I guess the idea behind EINTR is that you can respond more quickly to signals if they cause your read or write operation to return a little early. The problem with that philosophy is that it assumes that you are using signals as a form of inter-process communication, which would be a really bad idea. Someday, I'll probably write an entire blog post about exactly why that's such a bad idea.

Dealing with It

So how do you deal with this insanity? The simplest way is to not deal with it at all, and use fwrite, fread, and friends. These functions will not return short reads or writes, or EINTR. As another bonus, you get caching in userspace. This means that you will be making fewer system calls to the kernel, which generally improves performance.

If someone gives you a file descriptor, you can still use the higher-level functions. Just call fdopen to get a FILE* associated with that file descriptor. Then you can use fread and fwrite on it. But be careful not to mix calls to the high-level and low-level functions. If you do, you will get inconsistent results because the high-level functions are buffered and the low-level ones are not.

If you do need to call the "raw" read and write system calls, you'll need to write a wrapper around them that handles short reads and writes, as well as EINTR. This is surprisingly difficult to get right, but very important to do if you're going to use this interface.

Java Edition

You might think that you could leave all this insanity behind when using a different programming language. Well, at least in the case of Java, you can't. InputStream#read can perform partial reads, the same as the old-fashioned read system call. OutputStream#write doesn't seem to have the same problem; however, some other write APIs, like WritableByteChannel, do.

APIs that return short reads can be more efficient. It's great that we have these APIs. However, if you're going to create an API like that, don't call it read! Call it something like readUpTo or tryRead. That way, at least people will know what they're getting into.

The worst API of all, the all-time stupidest, has to be InputStream#skip. The skip operation moves forward "up to N bytes" in an InputStream. These guys really found the worst features of the POSIX API and blindly copied them. Short reads are at least somewhat useful because it may be more efficient to read data in smaller chunks. You may also get improved interactive performance by interleaving I/O and processing. Of what use are short skips? Since you don't get back the data (it was skipped), there's nothing you can do but call skip again until it does what you want.

When I asked a few Java programmers what skip did, none of them knew about short skips. And it's no surprise: the API is stupid. There's no conceivable reason you would want a short skip, and nothing in the function name to suggest that that it could happen to you. And since short skips only happen every now and then, debugging them is difficult.

Conclusion

Reading and writing may seem like the simplest of operations. But sometimes appearances can be deceiving. Keep on your toes, read the documentation carefully, and don't come up short.