Not so Humble

Does the fact that Linux users are contributing almost twice as much for the Humble Indie Bundle as Windows users prove that we’re twice as generous as they are?

Yes. Yes it does.

Comments Off

4,096 ought to be enough for anybody

Other than pointing out that the fault lies primarily with how Adobe’s programmers don’t know how to do anything efficiently on non-Windows platforms except crash the browser, I can personally verify that today’s xkcd is 100% accurate. Including the mouse-over text, and the fact that the kernel patch mentioned in the comic indeed exists.

Urandom fun fact

In turns out that if you’re wiping an external 1 TB hard disk using pseudorandom garbage, the process is CPU-bound and not I/O-bound:

$ sudo dd if=/dev/urandom of=/dev/sdb bs=4K
dd: writing `/dev/sdb': No space left on device
244190647+0 records in
244190646+0 records out
1000204886016 bytes (1.0 TB) copied, 136230 s, 7.3 MB/s

For those of you who have trouble dividing by 3600 in your head, 136,230 seconds works out to about 37.8 hours, with the CPU pegged at 100%. (Well, 50% since it’s a dual-core system, but whatever.)

My guess is that the process of actually encrypting the disk (once I initialize a file system on it) will take even longer, assuming that AES-256 encryption is slower than whatever PRNG algorithm Linux uses to drive /dev/urandom is.

Edit: Actually, that’s extremely incorrect. Encrypting the partition doesn’t actually write anything to it other than some kind of header identifying it as an encrypted partition. Yes, that means that almost all sectors are initially garbage if you try to decrypt them, but with a brand-new partition all sectors are initially garbage anyway. Creating the file system itself doesn’t try to read anything, either; it just writes to the sectors that will make up its index. And once you’ve mounted the file system, you’ll never try to read uninitialized sectors anyway, since there aren’t any files there.

In other words, the only O(n) step when creating an encrypted disk is wiping its previous contents; everything else is O(1) or O(log n) at worst. So why wipe with pseudorandom garbage instead of all zeros, which would be much faster? It’s (hopefully) computationally infeasible to distinguish uninitialized sectors (which look like random garbage because they are random garbage) from encrypted sectors (which look like random garbage because they’re encrypted with a strong algorithm). Not being able to tell where the data even is on the disk makes an attacker’s job more difficult.

Thanks to strong encryption, an attacker now has to either throw a lot of CPU power at the problem or use alternative means for recovering the data.

FLummoxing CLeverness

You wouldn’t think it’d be so difficult to rip a few DVDs. Well, it isn’t, unless you’re trying to do it in a way that’s far too clever for your own good.

A refresher for those of you who haven’t committed my personal computing resources to memory: I have two computers, holly and kryten. (No points for guessing the naming scheme.) holly is my eight-year-old former desktop computer, now relegated to running MythTV. kryten is my four-year-old tablet that I use for pretty much everything else.

Ripping and encoding DVDs is a CPU-intensive process, so naturally I wanted to do the job on kryten, whose wimpy 1.7 GHz Pentium-M is still quite a bit better than holly’s mere 933 MHz Pentium-III. The problem is, kryten doesn’t really have a DVD drive per se. I have a DVD drive that connects via the PCMCIA slot, but it’s slow (as in, too slow to play a DVD real-time) and sometimes hangs the Linux kernel when you eject the card.

My clever plan to get around this was to read the DVDs on holly, copy the raw data over to kryten via the local network, and do the actual encoding from there. Pretty much any program on Linux that reads from a DVD is just as happy to read from a directory where you copied a DVD’s files to, after all.

For the first disc, this worked just fine. For the second one, not so much. For some reason no program could read the VOB files on the DVD without projectile vomiting error messages to the system logs. (I wound up with 600+ MB of error messages in each of three system log files when all was said and done — log files that are typically around 50 KB for a week’s worth of data. So yeah.)

Since the VOBs are where the actual video content of the DVD is stored, this was a bit of a problem. I ran into the same problem with the third disc I wanted to rip. Thinking it might be a flaky DVD drive or driver, I tried rebooting holly (soft and hard), but no change. First disc fine, second and third not. Each disc played perfectly in my set-top DVD player, so the discs themselves were clearly undamaged. kryten’s external DVD drive wasn’t any more successful.

My best guess as to what was going on was that the discs in question had deliberately bad sectors to serve as copy protection to prevent someone from doing exactly what I was trying to do. But if that were the case, you’d think that all three discs would have had it, since they were all part of the same three-disc set. It doesn’t make much sense for only some of the discs to have copy protection. But if it wasn’t deliberate, it’s awfully suspicious that only the VOB files on the two discs were affected.

I wound up ripping the two problematic discs the easy way, doing everything on holly and having MEncoder read from /dev/dvd directly instead of mounting the UDF filesystem and reading that. Lo and behold, that worked, albeit taking longer to encode because of the slower processor. Whatever method programs use to read from the DVD directly instead of via the filesystem apparently never tries to access the sectors causing the errors.

Given that holly is effectively headless (given the “quality” of ATI’s Linux drivers, I didn’t want to switch from MythTV to the console if I could help it), this posed the question of how to figure out which titles on the DVD were the ones I wanted to rip, and not previews or special features or copyright notices or anything. The solution? Playing each title one by one in MPlayer, running in AAlib mode to render the video in glorious ASCII art though the SSH session from kryten to holly. Converting uncompressed DVD-quality video to ASCII art is a lot more bandwidth-friendly than sending it across the network directly, and while it is hardly the way I’d want to watch a video, it’s good enough to recognize the opening scene is what you’re looking for.

But finally, I now have the videos shrunk down into a format my portable media player is happy with. All it took was to stop trying to be so clever about how I was doing it, and thus avoiding whatever the underlying problem was.

Golf with an elf

I came across this page about making the smallest possible executable binary file using Linux’s ELF executable format. Supposedly with judicious use of absolutely evil hacks, you can get the file down to a scant 45 bytes — which is the size of the ELF header.

Or at least, you could on older kernels. On kryten’s slightly dusty 2.6.18 kernel, the two smallest (and evilest) versions of the program crash, but the antepenultimate one, the 64 byte one, works fine. Which is still quite impressive.

Note that I said 45 (or maybe 64) bytes is the smallest executable binary program you can execute successfully. If you switch over to a shell script, you can get quite a bit shorter. In fact, an empty file will execute successfully:

$ touch empty
$ chmod +x empty
$ ls -l empty
-rwxr-xr-x 1 paul paul 0 2008-03-17 22:32 empty
$ ./empty
$ echo $?

It’s the same basic functionality as /bin/true, which on kryten, weighs in at 22,120 bytes.