August 03

/mnt/usbdrive

When I reinstalled Debian sid, I had left a partition on the new hard drive large enough to hold my music but had yet to rsync the music from my server. I had never really put much thought into the transfer speed over my home network, I just knew that copying that many files would take a while. It took almost 26 hours.
Number of files: 21726
Number of files transferred: 19310
Total file size: 93259911200 bytes
Total transferred file size: 93259911200 bytes
Literal data: 93259911200 bytes
Matched data: 0 bytes
File list size: 782736
File list generation time: 31.767 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 93272942604
Total bytes received: 439336
sent 93272942604 bytes  received 439336 bytes  997997.89 bytes/sec
total size is 93259911200  speedup is 1.00
mp3sync  7481.33s user 2922.24s system 11% cpu 25:57:58.61 total

That's quite a long time! I then realized that if I bought a USB harddrive enclosure and put a blank drive in it, I could just plug it in any computer and copy the files over the USB cable instead of ethernet. I picked one up for about $50 and put in a blank drive. I thought that if I wanted to allow iTunes on my roommate's Windows XP computer to see the files I should format the drive with a FAT32 filesystem. I don't know how to that stuff in windows so I first plugged it into my linux desktop, partitioned the drive and then formatted it with mkfs.vfat and mounted it, then copied all my music onto the new drive again using the incredibly awesome rsync, this time it was much faster...
Number of files: 21726
Number of files transferred: 19310
Total file size: 93259911200 bytes
Total transferred file size: 93259911200 bytes
Literal data: 93259911200 bytes
Matched data: 0 bytes
File list size: 780968
File list generation time: 1.217 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 93272940832
Total bytes received: 439336
sent 93272940832 bytes  received 439336 bytes  24036433.49 bytes/sec
total size is 93259911200  speedup is 1.00
rsync -avpsrd /mp3 /media/usbdrive  1060.05s user 879.75s system 49% cpu
1:04:55.25 total

...about 24 hours faster :-) The only thing left to do was to plug it into my roommate's computer and load up some songs onto my iPod with iTunes, but of course there was a hitch, Windows XP would not initialize the drive, because apparently, if your drive is over 32GB (mine is 250GB) it somehow hits some stupid FAT32 filesystem upper limit. XP would only offer to format the drive to NTFS but of course read/write support for NTFS in linux is still a little sketchy due to Microsoft's unwilling-less to open their source code to those hippie linux hackers. So, I can plug in a hard drive with a FAT32 filesystem onto a linux computer without a hitch, but Windows XP is unable to read a drive formatted with one of its own native file systems. Nice.

The solution was to boot my Windows XP cdrom, press f10 to get the recovery console, and then use the format command, for some reason Windows will then format the entire 250GB's, because your using the magical recovery console. Then I plugged it back into linux, redid the rsync and plugged it back into XP and finally saw the files. I decided to ask my IT guy at work why I cannot format a drive in windows XP using the exquisitely named "Windows XP Disk Management snap-in" and he said something along the lines of "you need to do low level formatting, cant do that in userspace" to which I replied "but its a snap-in!!" and stormed off.

Anyway, all is well now and I am kicking myself for not getting one of these USB drive thingies earlier.

Posted by æc♥ | Permanent Link