zigg/journal

Saturday, August 8, 2009

mustafa downloads to PC at 70 KiB/s

Okay, so 70 KiB/s that's not as good as your broadband. Heck, it's not even as good as my broadband, and I'm a cheapskate.

But I'm pretty thrilled with it. My early TFTP proof-of-concept code was only able to manage a bit over 40 KiB/s—though you'll fall back to near that (maybe slightly faster? I have tuned a number of key things) if your client makes a request without requesting a larger block size. Use one that can ask for a larger blksize, though, and mustafa will shave it down to its largest supported size and the transfer will use that, improving substantially on the speed.

Apart from speed, my current build of mustafa accepts a filename which it passes to libfat and will transmit that file—or, using the newly-supported tsize and toffset options, any part of that file—to any TFTP client. I also built a little fuzzing tool that will send packets full of garbage to test its resiliency; on a 16 MiB payload, it slowed the whole transfer down by only about 30 seconds (the entire transfer took a hair over 4 minutes otherwise). Write support is only literally another screenful of code away, since I worked out most of the groundwork over this weekend in-between viewings of the featurettes on The Lord of the Rings extended edition DVDs.

I think it's still a way off from being able to share it, though. Right now, I think the core TFTP support is stable and probably even good, but the mustafa and "device" layers atop it (the fat device handles libfat; card will eventually handle save chip requests and the like) feel a bit early and messy. I imagine I will be going in and reworking some of those. The good news is that they're a lot smaller than the TFTP core.

But yeah, I feel like I'm in the home stretch now. Very happy to be where I am with this project.

Labels: , , ,

posted by zigg 10:37 PM 0 Comments

Thursday, August 6, 2009

savehost's bizarre metamorphosis

I think this is the first project I've ever actually abandoned—well, sort of—but I don't intend to do any more work on my previously-announced Nintendo DS Save Host.

It isn't that I don't want to create that tool anymore, but I found myself wanting a different approach to the problem. I was spurred down this road by my finding that dswifi's TCP stack isn't quite up to the task of sending large amounts of data reliably and quickly. One bug was found and fixed that was creating hard hangs, but in the process of finding said bug, I decided to try a UDP implementation instead—namely, a very basic TFTP server—and found I really liked the possibilities that opened up.

So now what I'm working on instead is a slightly more-ambitious project. It'll still do everything savehost was going to do, but the new program, mustafa (named after John Ratzenberger's character from Ratatouille), is going to be a little more generic and hopefully useful. Above and beyond savehost's goals, it will (okay, should):
Based on my initial tests, it's doing about 40 KiB/s transfer rates, which is great for most saves and probably most homebrew files. I'm also patching up the last feature-complete version of tftpy so that I can create a few PC-side GUI clients in Python, including one particularly neat idea that I actually half-implemented with savehost: a mass save backup utility that, once started, only requires you to swap cards on the DS as you work through your collection. No keypresses needed.

Hopefully, this will all be done in a few weeks.

Labels: , , , ,

posted by zigg 6:24 AM 0 Comments

Monday, April 13, 2009

Nintendo DS Save Host up on SourceForge

Almost two weeks ago now, I uploaded the read-only code I had so far for the Nintendo DS Save Host (which I first talked about back in February) to SourceForge. Honestly this is kind of new to me, putting incomplete stuff out there, but I have at least one person I'm talking to already who is interested in early-access stuff.

Having an audience helps a little. It doesn't magick time out of thin air, but it's given me a little more impetus to actually do things. What I did this evening was piece together an algorithm for writing to the 256-byte pages (or 64-Kbyte sectors) that make up the Flash chips on many of the larger Nintendo DS games. On these chips, a page must be erased completely, then rewritten completely, which I suppose is a requirement I could have left in the hands of the client authors, but what's the fun in that? They'd have to download entire pages over the wire, modify them, then send them all back.

So I implemented a routine that will simply let them write at any ol' offset, and it will walk as many pages as the write spans, reading out those which aren't slated for complete replacement so they can be patched, then erasing each page and writing its new content back out again.

It's really kind of ugly at the moment, and it's not hooked up to actual write machinery; it's just proof-of-concept that runs entirely on my laptop. I think it could stand a little improvement; when dealing with 256-byte pages it's okay to read the whole original page in, but if we're patching 48 Kbytes of a 64-Kbyte sector we might not really need to read the whole sector in first, so maybe instead of reading then overwriting the in-memory buffer I might just read in the portions I am not going to overwrite (Is that premature optimization? Probably. But I'm a perfectionist.)

It's a nice feeling of accomplishment, though. Maybe I'll finish this up before summer.

Labels: , , ,

posted by zigg 9:12 PM 0 Comments