zigg/journal

Wednesday, April 15, 2009

Ubuntu's missing crc32sum tool, in Python

Did I hear you say "I want a crc32sum application for Linux that can not only output CRC32 sums for a list of files, but also handle stdin if I leave its argument list blank, and I don't mind if it's hackish, so long as it works?"
#!/usr/bin/env python
#
#       A CRC32 summing utility that functions mostly like md5sum or
#       sha1sum.
#
#       Public domain by Matt Behrens <matt@zigg.com> with NO WARRANTY.
#
#       Usage: crc32sum [<filename> [<filename>...]]
#

BLOCKSIZE = 0x10000

import sys
from zlib import crc32

if sys.argv[1:]:
      names = sys.argv[1:]
else:
      names = [None]

for name in names:
      if name is not None:
              f = open(name, 'rb')
      else:
              f = sys.stdin
              name = '-'

      crc = None
      while 1:
              data = f.read(BLOCKSIZE)
              if data:
                      if crc is None:
                              crc = crc32(data)
                      else:
                              crc = crc32(data, crc)
              else:
                      break

      print '%08x\t%s' % ((crc & 0xffffffff), name)
Don't say I've never done anything for you.

Labels:

posted by zigg 10:24 PM 0 Comments

Monday, April 13, 2009

Nintendo DS Save Host up on SourceForge

Almost two weeks ago now, I uploaded the read-only code I had so far for the Nintendo DS Save Host (which I first talked about back in February) to SourceForge. Honestly this is kind of new to me, putting incomplete stuff out there, but I have at least one person I'm talking to already who is interested in early-access stuff.

Having an audience helps a little. It doesn't magick time out of thin air, but it's given me a little more impetus to actually do things. What I did this evening was piece together an algorithm for writing to the 256-byte pages (or 64-Kbyte sectors) that make up the Flash chips on many of the larger Nintendo DS games. On these chips, a page must be erased completely, then rewritten completely, which I suppose is a requirement I could have left in the hands of the client authors, but what's the fun in that? They'd have to download entire pages over the wire, modify them, then send them all back.

So I implemented a routine that will simply let them write at any ol' offset, and it will walk as many pages as the write spans, reading out those which aren't slated for complete replacement so they can be patched, then erasing each page and writing its new content back out again.

It's really kind of ugly at the moment, and it's not hooked up to actual write machinery; it's just proof-of-concept that runs entirely on my laptop. I think it could stand a little improvement; when dealing with 256-byte pages it's okay to read the whole original page in, but if we're patching 48 Kbytes of a 64-Kbyte sector we might not really need to read the whole sector in first, so maybe instead of reading then overwriting the in-memory buffer I might just read in the portions I am not going to overwrite (Is that premature optimization? Probably. But I'm a perfectionist.)

It's a nice feeling of accomplishment, though. Maybe I'll finish this up before summer.

Labels: , , ,

posted by zigg 9:12 PM 0 Comments