Hivemind: Creating the ultimate Rolodex

This is more of a discussion prompt than a specific question. Maybe there's already an off-the-shelf software solution for this, or maybe it's something I'll have to build for myself. I want to create a private searchable, database that will allow me to index all of my contacts, call records, notes, and recordings, and supporting documents. For example, when I get a new source, I want to be able to assign that person an identifying number and key in his or her name, telephone number, job title, and other contact information. I also want to be able to key in the date of first contact, the name of the person who introduced us, the dates and times of each subsequent communication, and links to any notes or audio generated in that communication. I'd like to be able to tag various records by story as well. So, let's say that I talked to Jane, Sam, and Bob for the XYZ story. I'd like to be able to search for "XZY story" and see all the sources I talked to, the days on which I talked to them, and links to the notes and audio from each call. Is there any existing software that will enable me to do this efficiently? I'd like to stay away from giant Excel files, if at all possible.

Update: I run OSX.

Comments

[Karl Steel]

What's your OS?

If you're running windows....perhaps not the most efficient software for this task, but it's certainly flexible: mybase.

[brent]

I am sure such things exist but it would be pretty easy to build a custom version of this type of application with a program like Filemaker or Access. You could do it in an afternoon. Filemaker would be the easiest, especially for mac users, but it is a bit expensive. Base, which is the open source db in Openoffice, would work as long as you had x11 for the mac.

[SomeCallMeTim]

Increasingly, I have difficulty distinguishing between a personal database and a search-and-index program. Would a searchable note-taking program like OneNote work for you? The search is, in my limited experience, pretty good. It allows tagging. I think it also allows for the easy embedding of pdf, audio, and (maybe) video. OTOH, it's less clear how easy it is to get stuff out of OneNote.

[Lindsay Beyerstein]

I run OSX.

[cfrost]

Right. Is there a truly capable user-friendly relational database program that one doesn't need to use SQL for?

[etaoing]

you should ask this over here:

http://www.43folders.com/

[brent]

Is there a truly capable user-friendly relational database program that one doesn't need to use SQL for?

Yes. Filemaker works pretty well for this type of application and it is extremely easy to use. It is, however, a bit pricey.

[brent]

I run OSX

Right. Included with your software is x11 which you can use to run Open Office and many other open source applications. Base is a component of Open Office which you can use to develop custom dbs. Its not quite as simple as Filemaker but it will definitely do the job. And its free.

[epc]

Might look into Highrise by the 37 Signals crew http://www.highrisehq.com/. I'm not sure if it could handle the audio files, but it has a concept called 'cases' which you could use for articles (you can then match people to each case).

I've only played with it and am not really using it myself.

[Bruce]

Re: FileMaker, older versions with the bulk of the features of new versions can be obtained for fairly cheap, the software got its start with Mac and only later went to Windows.

May I suggest searching deligio.com for shareware options as well? At a minimum, they may help you refine the features that you want or don't want through free download and testing other packages.

[marc sobel]

it seems like you are mixing two different applications contact management which people like salespeople use http://sourceforge.net/search/?type_of_search=soft&words=contact+software
and document management

being on mac probably limit the number of open source solutions but I would look at a combination of contact management, use of google desktop and stuff like slogger which save webpages easily. Please contact me if you want to discuss in more detail.

[Anacher Forester]

Lindsay,

I'm a Mac OS X user, too. I highly recommend Filemaker Pro. It's extremely customizable. Once you enter in the info (the most time consuming part), you can assign an almost infinite number of groups (sets) to each contact i.e. West Coast + Liberal Bloggers + GLBT + Facial Hair + right-handed. You can then sort them by any group combination for e-mail blasts, interview schedules and the like. Articles, reports or just about any other info can be attached as you please. It's great for financials such as accounts payable, accounts receivable and royalty statements. You can automatically e-mail things like invoices and payment reminders at a set time each month.

I use Filemaker Pro for my occasional consultant work to the music industry. As a one man operation, I'd be unable to be effective without it.

AF

[Eric Hamburg]

You might want to look at a contact manager like SOHO Organizer:
http://www.chronosnet.com/Products/sohoorganizer.html

or Contactizer:

http://objective-decision.com/en/products/contactizerpro/

[PhoenixRising]

We use FM Pro to generate all the kinds of reporting you need.

You can take the contact manager layout that comes in the box and modify to do what you're describing in a few hours.

Have fun and remember, making it prettier isn't actually revenue-related. I sometimes get off on a wave of design and realize I've killed an otherwise billable 45 minutes. I'm not graphics savvy, but that's just how darn easy FileMaker is.

[revenantive]

try this:

http://www.chronosnet.com/Products/sohoorganizer.html

SOHO Organizer. I think that might be a good relatively inexpensive solution for Mac.

there is a free 30 day trial available.

i am not sure how easy it is to sync audio the way you would want, but my experience with StickyBrain on my old mac was satisfactory.

[The Great Beast]

I used to use Maximiser (sorry, 'Maximizer', zeds for Americans), a contact management software that worked very well for just the kind of searching and organising you mention.

There must be a similar version for Mac-y types.

[greensmile]

I coded up a set of JavaScript functions and a UI that can all be embedded in a blog template. I use it to support all kinds of lookups for a team of media coordinators, a glorified rolodex. Its more work than an off the shelf product but has other advantages. That can free you from any particular computer but you probably want some privacy for your information. My solution is platform independent but data entry is tedious.

[Cranky Observer]

Lotus Agenda used to do this, but has been dead for quite a while. Mitch Kapor founded the OSAF ( http://www.osafoundation.org ) to reimplement Agenda as Chandler, but as with all post-Lotus Kapor projects it is stalled due to over-ambition and under-delivery.

If you want to do real storage and real searching, you are eventually going to have to learn SQL (or a full-text search language {e.g. Nexis'} that is just as difficult) and probably something about database technology as well - there is just no way around that knot.

Back around 1992-1995 I did a lot of work in this area, and while products were rough and direction uncertain it seemed that good solutions would arrive within 5 years (1997). Instead, for some reason I still don't fully understand, the web killed off all movement toward reasonable document storage, management, and indexing systems (below the $10 million level anyway).

Cranky

[Ghost of Joe Liebling's Dog]

You just might want to look at some wiki apps ... TiddlyWiki comes to mind in particular (simple, javascript-based -- but might not scale up too well).

[RickG]

http://www.theatlantic.com/doc/200611/fallows-tech

http://www.theatlantic.com/doc/200607/mind-meld

James Fallows often writes about such software. The above are among the more recent. I remember sometime back he recommeded a program called Zoot that hooked into Outlook and pretty much could reference anything that ever came through there.

With the general purpose programs, you are kind of stuck with the developers paradigm. If it matches yours great, if not, how big is the miss?

One thing about trying to stuff audio into a database is size. A lot of databases have maximum single data element sizes which won't accomodate. Also, a lot of lower end databases have maximum database sizes which might get blown out by large amounts of content.

One of the things you are looking for is a reverse index function, so that you can access data in both directions (e.g the part about who first referred a contact). If you can define these, you can build a relational DB, whether via SQL code or one of the visual products. I would also think that maybe some soft of large content registration system (audio, video etc) which stored those files in its own way, but provided your database with the address and perhaps some key descriptions might have a better chance of surviving long term.

I don't even want to think about the maintenance of the obsolete data!

[J. Bruce Hughes]

I would recommend you check out DEVONthink Pro. Several other writers have spoken highly of it, and I use it for research tasks similar to what you say you are trying to do. See http://www.stevenberlinjohnson.com/movabletype/archives/000230.html

[Ghost of Joe Liebling's Dog]

RickG, for someone running the Windows OS, I'd second your recommendation of reading Fallows. He was an Ecco advocate, if I remember, and later (after Ecco was abandoned by its publisher -- from whom it's now downloadble free in its last incarnation, circa 1993) a Zoot booster. Both are *very* useful programs for just this type of thing.

Ecco will run (I think) in a Wine box in Linux with most of its functionality intact; Zoot I'm not sure about.

But Lindsay's running OSX ... and that I don't know about.

[Doyle Saylor]

Small scale database applications like Access can handle what you ask. You can also use Google desk top seach to do roughly the same thing. Just make the documents to search with google desktop with whatever tag you want which you have to do anyway with databases. Then the desktop seach automatically adds the info to its search matrix or crawl. That works fast and it unites your whole hardrive into a giant searchable db.

Structured databases are a lot of trouble, fussy demanding and not very intuitive. There also visual 3d applications to access your work that might have a more intuitive feel.

A database of course forces you to systemize. But I've seen people with scraps of paper find things in books better than databases work.

What you want is simple and fast. That should tell you something isn't for you if it is not simple and fast.
Doyle

[SamChevre]

Unless you LIKE building databases (I do, but it's not a common taste) and interfaces, I'd go with a contact manager; ask a friend who works in sales for a recommendation. Most such softwares have a way to link to documents given to the prospect.

[Marc]

It's more free-form than what you're talking about, but it lets you put in pretty much any kind of information you want and then assign arbitrary tags for searching: Yojimbo. May or may not work for what you want to do.

http://www.barebones.com/products/yojimbo/index.shtml

DevonTHINK, as mentioned before, is in the same marketspace and is another option. I think there are other similar tools available - the name Tinderbox comes to mind. I like Yojimbo because it's trivially easy to use and very free-form - you never know what kind of data you want to capture and organize.

Yojimbo is NOT set up to do a particularly sharp job for contact information, which you listed as key requirement, so it might not work for you. It DOES excel as a general place to stuff information for later access. Myself, I keep my contact info in AddressBook; it just makes life easier in a lot of other ways, such as syncing to other devices. If you use Address Book for your contact data, you could easily set up 'Groups' for each story you were working on as a way to easily access the right contacts.

All entries in Yo are timestamped. I also think you can dump in almost any type of file or data, including media files, but I'm not 100% sure about that.

Doing up a custom database is also an option, but a lot more work as well as a bit of a pain to maintain over time (I do Filemaker dbs for work; it's a fine system, but perhaps overkill for your needs). You really need to be clear on exactly what data/datatypes you want to work with when you start designing the database, or be prepared to do lots of fiddling.

The big advantage of a database solution is the option of making your content directly web-accessible with a bit of work.

Hope this helps, and keep up the great work with your blog!