It’s a long weekend (thanks to African Unity Day) and I’m relaxing in sleepy Ho… it’s nice to get away from the constant hustle and bustle of Accra. Anyway, I’ve had a lot of hard disk trouble lately. First I ran out of space, then my system hard disk died. As if that wasn’t enough, the next day, my spare hard disk died too… making my life doubly miserable. You see, I hadn’t backed up my data… I was out of space after all and the dvd shop is out of my way – I kept putting it off. So I had to raid my younger brother’s piggy bank for a new hard disk. I’d like to think that “I was not attached to those hard disks” but it really grinds my gears the way they both failed in rapid succession. Now if I had my way, somebody at Seagate would be in a lot of pain right now. How is it that the world’s largest hard disk manufacturer has so many defective products on shelves? For what it’s worth, I’m never buying a Seagate hard drive again… even though Barracuda is such a cool name. Western Digital hard drives are – in my experience – much more reliable. But they really should do something about the name “Caviar.”

I now have a lot more hard disk space and a fresh install of Ubuntu 9.04. Morale of the story… back up your data and don’t buy Seagate !

There’s no nicer way to say it… I’m running out of disk space. I have three options: buy a larger hard drive, delete some files to free up space, or compress some of the data. Buying a larger hard drive is the best option in the long term but “in the long term, we’re all dead” and deleting files is painful for me… I’m a serial pack rat. So I decided to explore compression as a way out of my disk space headaches. First, I had to find the most efficient compression algorithm, a task I soon found out is not easy. I read several blogs and websites and everybody had something good to say about their favorite algorithm. But one thing was clear, the GZIP, BZIP2 and LZMA compression algorithms were leading the pack. To satisfy my own curiosity and determine for myself which was the most efficient, I decided to run some benchmarks. To be honest, I’ve been hearing some good things about the LZMA compression algorithm so I was hoping it would live up to the hype.

These benchmarks were conducted on a 2.53 GHz processor with 2GB RAM and a 5400 RPM Seagate Barracuda IDE hard disk. I also throttled the algorithms for maximum compression.

Version information:
gzip 1.3.12
bzip2 1.0.5
LZMA 4.32.0beta3
LZMA SDK 4.43

For starters, I threw an empty 1GiB file with nothing in it but binary zeros at them.

$ dd if=/dev/zero of=test.zero -bs=1024M -count=1
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 187.978 s, 5.7 MB/s

Now the fun starts.

GZIP
$ /usr/bin/time -f “%U seconds CPU %P” gzip -c9 test.zero > test.gz
12.36 seconds CPU 99%

BZIP2
$ /usr/bin/time -f “%U seconds CPU %P” bzip2 -c9 test.zero > test.bz2
32.07 seconds CPU 98%

LZMA
$ /usr/bin/time -f “%U seconds CPU %P” lzma -c9 test.zero > test.lzma
873.79 seconds CPU 96%

So what kind of compression ratios are we talking about here?

$ ls -lh test.zero*
-rw-r–r– 1 kafui kafui  1.0G 2009-03-25 12:01 test.zero
-rw-r–r– 1 kafui kafui 1018K 2009-03-25 12:51 test.gz
-rw-r–r– 1 kafui kafui  148K 2009-03-25 13:10 test.lzma
-rw-r–r– 1 kafui kafui   785 2009-03-25 12:52 test.bz2

GZIP squeezed 1 gigabyte into about 1 megabyte in about 12 seconds… nice. LZMA’s compression ratio was very impressive; it squeezed 1 gigabyte into 148 kilobytes BUT in 873.79 seconds… that’s almost 15 minutes. BZIP2 was absolutely cool… 1Gib down to 785 bytes in 32 seconds! The clear winner here however is BZIP2. It has the highest compression ratio with acceptable time requirements. Now on to tests with real data.

For the next test, I decided to compress the contents of my  /opt folder. To simplify things, I created a tar archive of the folder first.

$ sudo tar -cf opt.tar /opt
[sudo] password for kafui:
tar: Removing leading `/’ from member names
tar: Removing leading `/’ from hard link targets

$ ls -lh opt.tar
-rw-r–r– 1 root root 120M 2009-03-25 15:48 opt.tar

So we’re working with 120MB of data. On to the tests:

GZIP
$ /usr/bin/time -f “%U seconds CPU %P” gzip -c9 opt.tar > opt.tar.gz
19.42 seconds CPU 89%

BZIP2
$ /usr/bin/time -f “%U seconds CPU %P” bzip2 -c9 opt.tar > opt.tar.bz2
30.76 seconds CPU 93%

LZMA
/usr/bin/time -f “%U seconds CPU %P” lzma -c9 opt.tar > opt.tar.lzma
132.21 seconds CPU 92%

$ ls -lh opt.tar*
-rw-r–r– 1 kafui kafui 120M 2009-03-25 15:48 opt.tar
-rw-r–r– 1 kafui kafui 39M 2009-03-25 15:56 opt.tar.gz
-rw-r–r– 1 kafui kafui 36M 2009-03-25 16:09 opt.tar.bz2
-rw-r–r– 1 kafui kafui 25M 2009-03-25 16:16 opt.tar.lzma

Once again, GZIP was the fastest and got 120MB down to 39MB in 19.42 seconds. BZIP2 reduced 120MB to 36MB but took 11.34 seconds longer than GZIP. LZMA delivered the best compression with 25MB but took 132.21 seconds. It appears there are trade-offs with each compression method. GZIP is fast but its compression ratio is the lowest of the three. LZMA (depending on the data) delivers the most efficient compression ratio but takes too much time to do so. BZIP2 strikes a balance between efficient compression and speed… it’s way faster than LZMA and can actually deliver better compression. LZMA just does not live up to the hype.

Unfortunately, these benchmarks were of no use to me because about 140GiB of my data is made up of AVIs, PNGs and JPEGs. These formats are already compressed so there isn’t much room for further compression. But for what it’s worth, I gave the algorithms a spin anyway.

$ ls -lh The.Big.Bang.Theory.S01E10.avi
-rwxrwxrwx 1 kafui kafui 175M 2008-04-18 20:14 The.Big.Bang.Theory.S01E10.avi

GZIP
$ /usr/bin/time -f “%U seconds CPU %P” gzip -c9 The.Big.Bang.Theory.S01E10.avi > The.Big.Bang.Theory.S01E10.avi.gz
10.94 seconds CPU 78%

BZIP2
$ /usr/bin/time -f “%U seconds CPU %P” bzip2 -c9 The.Big.Bang.Theory.S01E10.avi > The.Big.Bang.Theory.S01E10.avi.bz2
55.15 seconds CPU 94%

LZMA
$ /usr/bin/time -f “%U seconds CPU %P” lzma -c9 The.Big.Bang.Theory.S01E10.avi > The.Big.Bang.Theory.S01E10.avi.lzma
138.74 seconds CPU 93%

$ ls -lh The.Big.Bang.Theory.S01E10.avi*
-rwxr-xr-x 1 kafui kafui 175M 2009-03-25 16:34 The.Big.Bang.Theory.S01E10.avi
-rw-r–r– 1 kafui kafui 173M 2009-03-25 16:35 The.Big.Bang.Theory.S01E10.avi.gz
-rw-r–r– 1 kafui kafui 173M 2009-03-25 16:39 The.Big.Bang.Theory.S01E10.avi.bz2
-rw-r–r– 1 kafui kafui 174M 2009-03-25 16:43 The.Big.Bang.Theory.S01E10.avi.lzma

GZIP and BZIP both got the 175MB episode of The Big Bang Theory down to 173MB; BZIP2 of course took 44.12 seconds longer. And LZMA got it down by only 1MB but in 138.74 seconds. As you can see, it doesn’t make much sense for me to compress my videos and pictures… not with those compression ratios. So it seems I’ll just have to cough up the cedis for a new hard drive.

Just a few months ago, I was not even using a graphical environment; videos, music, surfing the internet, instant messaging… all from the terminal. But my philosophy on software has been undergoing subtle changes ever since the I got a faster computer. The thing is, I now default to graphical applications for most tasks. Where aptitude, mplayer, mpd, ncmpc, rtorrent, finch and elinks ruled supreme, synaptic, amarok, smplayer, deluge, pidgin and firefox have the upper hand. So this morning, I decided to find a GUI hardware information program to replace lspci, lshw and dmidecode… not really, I just needed a graphical frontend to these tools.  It took me about 15 minutes to go through the top three: Hardinfo, Sysinfo and Lshw-gtk. Hardinfo was the most impressive of the lot. In addition to hardware information, it can perform benchmark tests and let you compare the results with that of others. My lean, mean and ridiculously affordable box did quite well in the comparison tests. Sysinfo was a little stingy on information but it’s quite capable. Lshw-gtk, as the name implies, is really just a graphical frontend to lshw and threw up some detailed information about my motherboard and CPU but very little else. I’m keeping only hardinfo for the long term however, the others don’t quite live up to expectations. Anyways, that is only my opinion… I’ll let the screenshots do the rest of the talking:

CPU-Z Benchmark

Blowfish Benchmark

Fibinacci Benchmark

MD5 Benchmark

Ray Tracing Benchmark

Last week, I resized one of my partitions to create some swap space. It’s a long painful story that I will not bore you with but essentially, I got cocky with my 2GB RAM and refused to allocate swap space… ubuntu punished me by activating compcache which then caused random lockups of the system. Anyway, I was coming up to the end of an install cycle anyway so I backed up the system with remastersys (that is one story, I’ll have to blog about later), resized one of my partitions to create swap space and out of habit, did a sudo fdisk -l… it told me my partition entries were not in the proper order (a side effect of the resizing). It wasn’t a big problem, just an irritation really but an irritation nevertheless. I decided to fix it before the install:

sudo fdisk /dev/sda
press x to enter expert mode
press f to fix partition order
press w to write the partition table changes to disk

Yes, I’l take a Heineken.

One of my neighbours came to me a couple of days ago with a problem… he’d lost his CV to a windows virus and had no backup. So I said I’d try to recover the file for him. A quick google led me to Foremost. I chose foremost because it’s small, fast and easy to use. Since this was a file recovery operation, I decided to minimize the risk of the deleted file being overwritten by creating a disk image of the pen drive. A quick

dd if=/dev/sdb of=recover.iso

did the trick.

Then it was a simple matter of

sudo apt-get install foremost

to install foremost and then…

foremost -t doc recover.iso

The ‘-t doc’  parameter tells foremost to look for only Microsoft Word files. About a minute later, the thing was done and the files were back. For what it’s worth, if you accidentally delete a file, it’s a very bad idea to continue using the computer. Shut down the pc or disconnect the drive immediately and look for a linux rescue cd.

I chose to stay up and install openSUSE rather than crash the crazy new year parties in Accra (or go to church as all of my neighbours did). It could have been such a great distro. Later in the day, the good and the bad about openSUSE.

The past month has been hectic for me. I’ve been all over the country – Tamale, Kumasi, Ho, Winneba…. During one of my brief stopovers in Accra, I managed to build  a new pc. It’s not excatly what I wanted but it’s performing admirably. Rather than buy one of those grossly over-priced brand name machines, I bought all the components separately an assembled them myself… a decision I feel pretty smug about. Here’ s the thing, a 2.53ghz Core 2 Duo, 2gb RAM, ASRock Wolfdale 1333-D667 motherboard, 40gb 7200 rpm internal hard disk + dvd writer costs something in the region of GHC 900 but I pulled it off with GHC 400 by avoiding a branded machine. It’s not that hard to assemble your own pc… all you need is a screw driver. When I get a bit more time over the weekend, I’ll write more about it.

Apparently Zain is allowing potential customers in Ghana to reserve their cell phone numbers even before they become fully functional. It’s not bad… I reserved my current Kasapa number – minus the 028 of course. Here’s the link: http://app.rancardmobility.com/final.zain/

Often, firefox just hangs on me (when I’m forced to use it) and sometimes brings the entire machine to a grinding halt. I’d rather not start a rant about how bloated firefox is… it’s not worth the breathing exercises I’d have to do. Usually, when an application becomes unreasonable, it’s quite easy to kill it and get on with other things. My favorite method is to add the Force Quit widget to the gnome panel. This way, all I have to do is click on the widget, click on the misbehaving  window and problem solved.

There’s also the xkill command. Hold down Alt + F2 to bring up the Run dialog, or bring up a terminal, type in xkill ( the mouse pointer should turn into a skull and crossbones) and click on the offending window… it should die immediately.

There’s always the trusty command line option. On a terminal, type

killall application_name

To kill firefox this way, you’d type

killall firefox

The killall command is ‘merciful’ as it allows the offending application to end gracefully. However, it doesn’t always work … some apps are really stubborn. To kill firefox without giving it a chance to clean up, try

kill -9 `pidof firefox` (those are back ticks, people… located on the ~ key)

Of course you could go the long hard way: use the ps aux command to find the pid of the app (let’s say 6849) and do kill -9 6849 but I prefer the easier method. It’s also possible to bring down the entire X server and hope the hung program will die too (in my experience, firefox is immune to this sometimes… it becomes an “uninterruptible” service.) To restart the X Server:

press Ctrl+Alt+Backspace

I haven’t used them before but the magic sysrq keys are quite interesting. Apparently, if you get really desperate, you can talk directly to the kernel. This would be quite useful if your box totally stopped responding and none of the earlier methods worked. They have to be compiled into the kernel first though. To check if they are, do:

cat /boot/config-`uname -r` | grep CONFIG_MAGIC_SYSRQ (again, back ticks)

You should get something like this:

CONFIG_MAGIC_SYSRQ=y

Also, do:

cat /proc/sys/kernel/sysrq

to make sure they are not disabled. You should get

1

To safely reboot your box without data loss during hangs, this is the method suggested on Wikipedia’s Magic SysRq key page.

1. Press Ctrl+Alt+F2 to switch to a terminal window. Sometimes, this is not possible.
2. Press Alt+SysRq+R to get the keyboard
3. If pressing Ctrl+Alt+F2 before failed, try it again now.
4. Press Alt+SysRq+S to sync your disks.
5. Wait for OK or Done message.
6. Press Alt+SysRq+E to term all processes.
7. Press Alt+SysRq+I to kill all processes.
8. Press Alt+SysRq+U to umount all disk drives.
9. Wait for OK or Done message.
10. Press Alt+SysRq+B to reboot.

Finally, if all else fails turn off the power to the computer and hope you don’t do any permanent damage to the system.

It’s been almost a month since I applied for a broadband connection at home and I still have to go to an internet cafe to blog! This totally sucks. I guess I was naive to expect better service from Ghana Telecom.