svn: No newline at end of file

If you've used Subversion for any significant length of time, then you've probably run into the following message (or one just like it), near the end of a "svn diff" command.

\ No newline at end of file

Of course, you would have realised that this is because your editor has added a newline character to the end of the file in question, where the original had no such newline character.

For me, this happens most often when patching source code that I've imported from elsewhere, or when editing files under Linux that were originally created under Windows (most of my project contain both Linux and Windows components).

I experience this problem (a slight overstatement perhaps) when editing with either vi or nano (haven't bothered testing any other editors). And while I could probably tell those editors to not add the final newline character somehow, the default behavior is actually considered to be a good thing (indeed various C compilers will actually complain if the final newline character is not present).

And of course, this "no newline at end of file" message is not normally a problem - it just adds noise to svn diff outputs - both now, and in the future. And that's something I dislike quite strongly.

So, how to fix it? Well, the easiest way I've come up with, is a very simple head command to strip the very last byte of the file. It looks like this:

head -c-1 file.ext > file.tmp && mv file.tmp file.ext

As always, you should use commands like this (ie ones that overwrite existing files) with extreme caution. But this one should be fairly safe - as long as you don't run out of disk space ;)

The first part of the command simply strips off the last character from the file.ext file, and writes the output to file.tmp. The the second half of the command moves the file.tmp file back to file.ext, overwriting the original in the process. Because the two commands are separated by the && operator, the second command (mv) will not be executed unless the first command (head) completes successfully. But if you want to be extra cautious, then you can run the two commands separately, like this:

head -c-1 file.ext > file.tmp
(you can manually check the file.tmp output file here)
mv file.tmp file.ext

I hope you find this small tip useful, I know I do :)

Update

Just a couple of more thoughts... first off, if you're dealing with files that have DOS-style line encodings (ie lines end in "\r\n" instead of simple "\n"), then you simple need to run the command twice to cut off both characters, or change the "-c-1" to "-c-2" instead. For example:

head -c-2 file.ext > file.tmp && mv file.tmp file.ext

Also, if you want to process a group of files, then the obvious thing to do is use xargs (such a great utility!). But, of course, xargs commands can get a bit tricky when you want to combine commands (in this case using the && operator. The usual way to handle this (or at least the way I like to) is to parse the grouped commands to a single bash -c command like this:

ls -1 | xargs -I{} bash -c 'head -c-1 "{}" > "{}.tmp" && mv "{}.tmp" "{}"'

Note, this command will process all files in the current directory (so be very careful), but obviously you can modify the beginning ls command to limit the files that will be processed. For example, to just process all C# source files, you could run this:

ls -1 *.cs | xargs -I{} bash -c 'head -c-1 "{}" > "{}.tmp" && mv "{}.tmp" "{}"'

Well, that's it for now ;)

Trackback URL for this post:

http://colby.id.au/trackback/95

"as long as you don't run out of disk space"

Hey there,
just found your blog through a post on qt-interest from you and read this article. I had to chuckle when I was reading "But this one should be fairly safe - as long as you don't run out of disk space ;)" ;)
That's because something that stupid happened to me, just that it was not the disk space that was at it's limit, but the filesystem's maximum file-size.
Long story short: I was in the process of re-installing a linux-server and backed up the subversion repository's dump gzip-compressed to a NAS. When restoring this backup, I totally forgot that gunzip is kinda "in-place" instead of outputting to stdout, like normal unix-programs do. gunzip, however, unzips the content and deletes the original file. In my sad case, it tried to unzip the content, which was larger than 2GB, which the filesystem on the NAS could not handle. It then stopped half-way, but SOMEHOW deleted the original file! It even took me a few seconds to realize that, because I did something like "gunzip /mnt/nas/subversion.dump.gz > ~/subversion.dump" and wondered why subversion.dump was 0 bytes. Then I realized that subversion.dump.gz was missing and subversion.dump was 2GB and something, while it should've been bigger...
Fortunately I had a backup from the morning of that day, so "only" 1 day of commits was lost. If I hadn't had this backup, my co-workers would've killed me. ;-)

Post new comment