Sunday, August 30, 2009way out of even the extended warranty I'd bought. So I carefully extricated my data and started using an old Sony CLIÉ PEG-S320 to keep my life in order.
But then I heard of a man named Chris Short of Mankato, Minnesota. His reputation preceded him; many people had already sent off all sorts of Palm handhelds to him and found joy in their little computers once more. So I shot him an e-mail—you can reach him yourself at firstname.lastname@example.org—and, perhaps unsurprisingly due to the rather widespread occurrence of the problem I was experiencing, he knew exactly what was up and how to fix it. (He also offered a MicroDrive-to-Flash conversion for an additional $50, but I declined for now.)
I sent my little LifeDrive off with a check for $49 via Priority Mail and got it back in one week. Chris had fixed it up same day. And, joy of joys, it works again! Chris said in our initial exchange that what happens with the LifeDrive is that the ROM chips have cold solder joints. He resolders the chips in place and applies an adhesive that he says Palm should have applied in the first place that's supposed to protect against a repeat occurrence. Obviously, I've not had it long enough to see if this holds; but I already feel like I'm in better hands.
Amusingly, the night I was working on restoring my data to the LifeDrive, it apparently spontaneously rebooted on me before I'd even had a chance to install my apps back. My heart leapt into my throat. As it turned out later, though, it's karmic's fault. Not Chris's.
It's good to have my constant companion back again.
posted by zigg 6:50 PM 0 Comments
Thursday, August 20, 2009Handspring Visor came out and my core Palm databases are the same ones that I created nearly nine years ago, brought from system to system.
These days, I've been using Palm's disavowed LifeDrive, and mine just kicked the bucket. This is my second one; the first I got replaced through PalmCare (found an absolute steal on a packaged PalmCare plan.) As part of the LifeDrive lifestyle, I've been using Alexander Pruss' NVBackup to keep my data safe.
A hopefully brief history: traditionally, Palm handhelds have kept everything—programs, data, you name it—in battery-backed RAM. This works a lot better than it sounds, but it does have the disadvantage that it all goes pfft if you don't keep tabs on your Palm's battery. The early models actually ran on standard off-the-shelf batteries; my Visor used AAAs and it lasted a good while unless I was using it constantly. Backup was done by syncing your system to a PC.
Anyway, this battery run-down situation must have irked enough people, because in the last round of Palm handhelds they ever made, they introduced something called NVFS, non-volatile storage for the system's data. This wasn't the first time Palms could write to something other than RAM; over the previous several years they'd been adding storage (usually some form of Flash card) to some models. With those came backup applications, which did sadly have some problems with restoring the system to its original state, owing mostly, as I understand it, to the weird machinations that it took to serialize an in-RAM database to a
.pdb(Palm database) file.
NVBackup, by comparison, was as simple as could be. It took these NVFS-based Palms and simply copied their databases straight out of NVFS, where they were already serialized, and dumped them straight to your Flash card. The advantage was that in almost all situations, restoring a backup brought your system back exactly the way it was before, which is great if you still have your system. My LifeDrive was dead, though, and I didn't have another to restore to—so I needed to do something to get these databases into
.pdbfiles so I could then stick them onto one of the other old Palms I have lying around, lest my life become tragically disorganized. Er, more than it is now.
Alex Pruss has done some work on a tool called unbackup which reportedly does a passable job on some of the NVBackup dumps. I tried this tool. It did not do well on my LifeDrive's output at all, sadly. Crashed a lot, failed to make sense of other files... it just wasn't pretty. Facing the prospect of reverting to two-month-old data (my last PC sync), I started scraping together ideas for one last shot.
Palm developers (of which I am one, though I've never released a bit of anything) have access to the Palm Simulator tools, which emulate a Palm device on a (Windows, sigh and boot up the VirtualBox) PC. See where I'm going with this? Now I do have a spare LifeDrive that I can restore my NVBackup set to, and use the onboard tools to make
.pdbfiles out of my core databases. It wasn't quite that simple, but I did eventually get it working.
First problem: getting NVBackup to restore to the simulated LifeDrive in the first place. Palm Simulator sets up an area on your disk to simulate external storage, so I copied the NVBackup set off my SD card and into that area, along with the NVBackup program itself. NVBackup got installed to the virtual LifeDrive and presented me with the backup set I copied. But then I tried to do the restore, and things started to get ugly.
I knew I only needed a small set of databases—
ToDoDBalongside those from my two key other Palm apps,
Keys-Gtkrfrom Keyring and
Months-HkDtfrom Eat Watch—so I started off trying a selective restore of just those databases. It looked like it was working, but then when it came time to restore the files, I kept getting "error opening" messages. I had no idea what was going on. I tried disabling the Simulator's write-protected storage option (honestly, I have no idea what this actually does) as well as PACE extended checks (I have a slightly better idea but this was last-ditch); neither gave me any joy.
Eventually I even wrote to Alex Pruss, since I saw in one of many Google searches that he had originally started work on NVBackup in the Simulator; unfortunately, he had no idea how to solve my problem. One lead caught my eye, though: "error opening" could also show up if there was trouble dealing with an encrypted backup. Although I'd never had trouble with it on a real device, I wondered if perhaps NVBackup's default gzip compression might be at fault. So I ran gzip -d over the NVBackup set, hopped back into the Simulator, ran the restore, and—success!
Well, partial success, at least. The Simulator crashed irrevocably on reboot, possibly because the Simulator is compiled for x86 and my backup was full of ARM code. So I hard-reset my virtual LifeDrive and went back in again, this time only restoring the databases I needed (as well as their new PalmOS counterparts'—Contacts, Calendar et al.—since the old databases are really just there for compatibility purposes), then used FileZ to make
.pdbs out of them on the virtual SD card, and voilà!
Okay, almost voilà.
DatebookDBwas being stubborn, no matter how many times I tried to restore, convert, and copy it, pilot-xfer was refusing to copy it—likely corrupted somewhere along the line. I eventually ended up HotSyncing that one directly from the simulated LifeDrive over a "network" from within the VirtualBox VM to my host Linux netbook. And now it works. The old CLIÈ has all my important data and apps just as I left them before the last backup.
It was a lot of work, but it was worth it, I think. And hopefully there's some clue in here that might help someone else in the same boat down the road.
posted by zigg 11:17 PM 0 Comments
Saturday, August 8, 2009my broadband, and I'm a cheapskate.
But I'm pretty thrilled with it. My early TFTP proof-of-concept code was only able to manage a bit over 40 KiB/s—though you'll fall back to near that (maybe slightly faster? I have tuned a number of key things) if your client makes a request without requesting a larger block size. Use one that can ask for a larger
blksize, though, and mustafa will shave it down to its largest supported size and the transfer will use that, improving substantially on the speed.
Apart from speed, my current build of mustafa accepts a filename which it passes to libfat and will transmit that file—or, using the newly-supported tsize and toffset options, any part of that file—to any TFTP client. I also built a little fuzzing tool that will send packets full of garbage to test its resiliency; on a 16 MiB payload, it slowed the whole transfer down by only about 30 seconds (the entire transfer took a hair over 4 minutes otherwise). Write support is only literally another screenful of code away, since I worked out most of the groundwork over this weekend in-between viewings of the featurettes on The Lord of the Rings extended edition DVDs.
I think it's still a way off from being able to share it, though. Right now, I think the core TFTP support is stable and probably even good, but the mustafa and "device" layers atop it (the
fatdevice handles libfat;
cardwill eventually handle save chip requests and the like) feel a bit early and messy. I imagine I will be going in and reworking some of those. The good news is that they're a lot smaller than the TFTP core.
But yeah, I feel like I'm in the home stretch now. Very happy to be where I am with this project.
posted by zigg 10:37 PM 0 Comments
Thursday, August 6, 2009previously-announced Nintendo DS Save Host.
It isn't that I don't want to create that tool anymore, but I found myself wanting a different approach to the problem. I was spurred down this road by my finding that dswifi's TCP stack isn't quite up to the task of sending large amounts of data reliably and quickly. One bug was found and fixed that was creating hard hangs, but in the process of finding said bug, I decided to try a UDP implementation instead—namely, a very basic TFTP server—and found I really liked the possibilities that opened up.
So now what I'm working on instead is a slightly more-ambitious project. It'll still do everything savehost was going to do, but the new program, mustafa (named after John Ratzenberger's character from Ratatouille), is going to be a little more generic and hopefully useful. Above and beyond savehost's goals, it will (okay, should):
- add the ability to upload or download files via libfat (meaning, file transfers on or off your homebrew-capable card or cart without shuffling µSD cards around)
- allow any TFTP client—simple clients are included with Windows and Mac OS X and are included in most Linux distro's repositories—to do basic backup/restore of the save chip and basic upload/download of files
- have its TFTP functionality contained within a couple files that can be added to any other homebrew project to give it data transfer functionality (and it's not limited to files!)
Hopefully, this will all be done in a few weeks.
posted by zigg 6:24 AM 0 Comments