Karl Dubost, on #rdfig: "My information is lost everywhere."
Having shown the world the primitive n.py, and having been introduced to the advanced GNOME storage and Dashboard, I thought it might be a good idea to outline my current state-of-the-art when it comes to making information findable using notes programs.
The most interesting piece of code I developed since n.py was a mini-server that would take my simple datestamped lines, and make them available via HTTP. Its main feature was that it'd hyperlink common words to notes-wide searches for those words. It could also let you go back and forth through notes, provide summaries, and so forth. Its power was in the fact that it was an interpretative thing—it didn't modify the notes or require that one added any metadata. And that's a big lesson; people don't like adding metadata, especially when it's derivable from the content.
I'd like to have had a scripted SVG interface to it.
My current plan is to have special syntax for metadata, but I don't want it to get in the way. I want each note entry to have a bag of special tokens. Special tokens could be dates, URIs, etc. relating to the entry—anything regexpable that has a special syntax. It's possible that these could just be derived from the content, but at the moment I'm using "@:" as a seperator. Using "{}" to interpolate is another possible choice.
There's always a tension between regexping and wanting to do Natural Language Parsing. Learning a little-language or something like lojban would be possible, but rather an immense strain on the user! And that's taking it too far, but identifying and preserving the patterns is certainly still the goal.
The special tokens would be attached to the note via a property for each type of token. So, for example, if you were providing a todo item, you might input: "write a letter to John Peterson @: todo next-tuesday". That could be stored as {(0, "write a letter to John Peterson"): {'type': ['todo'], 'date': ['YYYY-MM-DD']}} in Python, where YYYY-MM-DD is the normalized date. You might also want to preserve the date as it was originally entered.
Searches would be conducted by setting up constraints, so you could search for all notes within a particular date range, that contain certain words, and so forth, ANDing and ORing constraints together.
It'd be nice to make old posts editable with revision storage, and have searches linkable so that they act like folders do in filesystems, but that's getting towards being like a wiki. Combining a wiki and notes program is a design I've considered, but then I've also considered a LISP/transclusion style design too, so you can't take it too seriously.
Note that b.py and n.py are slightly confused about where to store config data; it's like they naturally want to store it along with the data, but it would probably be better to seperate it out into a configuration file. Karl pointed me to a little ConfigParser to dict hack he wrote in Python that'll come in handy. I've also been wondering, though, about using an RSS 3.0 parser. It's frustrating how large interesting applications like this are often made of small tedious components. Not that I mind config parsing (it's actually one of the nicer things to work on), but date-munging is horrendous.
This all reminds me a lot of TimBL's tangle... "How much wood would a woodchuck chuck if a woodchuck chuck wood chuck chuck chuck wood wood chuck chuck chuck"—Weaving the Web.