Ink is giving way to nodes and networks, ledes and inverted pyramids are being swallowed up by a tsunami of blogs and memes. Amid the din and aggressive edge of the digital conversation, how do we figure out what’s really going on in the world? The aim of NewAssignment.net is to harmonize these worlds, do a mashup of the best of each. On the one hand, there’s traditional shoe-leather reporting, where you call people up, assemble data and information, extract insights and ultimately a storyline that says something interesting. Though oft-derided these days, this is a craft, and done well it can have a tremendous impact – on individual lives and the political process. On the other hand, you have the digital world, where distance is obliterated (reducing, in some ways, the wear and tear on shoe leather), distinctions between “journalists” and “everyone else” are blurred, any curious citizen can post insights and ideas, and the pool of available digital data is growing exponentially.

So far, few people have managed to skillfully straddle these worlds; Bill Allison of the Sunlight Foundation is one of them. He worked at the Philadelphia Inquirer, where he was a researcher for investigative reporters Donald Barlett and James Steele; afterward, he spent nearly a decade at the Center for Public Integrity. At Sunlight, he has placed himself at the emerging nexus of citizen journalism and national politics, specifically Congress. Since starting at Sunlight last year, he has had a run of interesting stories and projects that capture something of how journalism will look like in the future – and, while it looks quite different, the fundamentals are the same.

I sat down recently with Allison to get his insights on the Internet and reporting, social networking, data and other topics – his remarks are excerpted below.

Bill Allison: I certainly don’t think that we’re at a point yet where the Internet could do something like the series you did on what was going to happen to New Orleans. They certainly can’t do a Barlett and Steele type investigation. There’s things the Internet isn’t capable of doing yet. There are bloggers who have expertise in a certain area who will write about their area of expertise, but that’s the opinion of one expert…this is just one person’s experience, and journalism is trying to put together, the joke is, two people’s experiences.

But the Internet, of course, makes it much easier to get two people’s experiences – or 200, or 2,000.

Everybody’s seen in a newsroom when the email goes out: I’m looking for someone who had to move out of Philadelphia who had to move because his insurance rates are too high…On the Internet, those people are out there now, and they’re scanning their insurance bill and putting it up as a pdf, saying ‘look what I have to pay because I live here.’ The sources are a little bit louder and a little less hard to find. So you can really et a much better sense of what’s out there. The question is, how do you organize it? How do you make sure that what’s important gets out – organize the pathways.

There are people out there writing who really know the issues. You can read people with a lot of military experience writing about what’s gone wrong in Iraq, and it’s really as informed as anything you’ll read in the Washington Post or the New York Times – and in some ways more informed, because they know institutionally what the military’s good at, what it’s not good at, where it’s wearing down. Those are amazing things. You’ll also obviously get the pro-military side too from that kind of thing. You’ll never get hurt from hearing more perspectives, getting more sources of information.

There are people out there who want to inform, and people who want to be informed. It’s a two-way street, it’s a conversation. If you give them the opportunities to find out something – let’s answer this question, who is putting all these earmarks into bills, who is responsible for them, who is putting a hold on the legislation – they will respond.

This is kind of what happened with the Hastert thing. There were people writing about the Prairie Parkway on their own websites and blogs, and there was one guy who had the community activist site who was trying to stop the parkway. They had found out that it was quite close to where Hastert lived, and then what I was able to do was to start from that and say, does he own any other property, what else is going on here?

What you have are these tantalizing pieces of information that it takes five or six people to put together. It does kind of snowball. After we did the Hastert thing, Ed Morrissey at Captain’s Quarters, a conservative blogger, looked at census data, settlement patterns, and was able to make a pretty strong argument that this was being built to spur development, and not the other way around. There’s almost nothing along where this road is. Somebody else was looking up and found all of these home sellers and developers, not the ones Hastert sold to, but they were advertising in their model home areas that they were going to be a few miles from Prairie Parkway. That’s interesting information that we didn’t have that people ferreted out on their own. That makes much richer stories, I think.

The Hastert revelation generated a lot of media attention and interest from readers, so they decided to capitalize on that, Allison says. But that’s when the truly experimental, trial-and-error nature of this new kind of journalistic enterprise became clear.

We thought we’d do a citizen journalist effort where people would look into their own members – read your own member’s financial disclosure form, contact the member. And so we did that – and that one was a disaster.

We were very open-ended, do whatever you want; there wasn’t any kind of guidance. I think the instructions were to just read it and see what you find, contact them and let us know. We got a lot of interest, but it was frustrating because people were looking for either exactly the same situation as Hastert – even if there was something wrong with a member’s form, you weren’t necessarily going to find a Hastert-like situation with an earmark. The forms certainly don’t disclose anything about their legislative acts, so people were confused: ‘so, why isn’t there anything about the Prairie Parkway in this particular thing?’ There were bad instructions. There was no way for people to post the information they had. There were all kinds of problems with the effort. I think we lost a lot of the good will we engendered, just as we really didn’t think at all about how we were going to design this.

Our thought was, anybody can go out and be an investigative reporter. The fact of the matter is, if it were that easy everybody would be one – you wouldn’t need newspapers if everybody could find out whatever information they needed on their own. It doesn’t work that way.

They learned more while working to expose the “secret hold” Senators. Asking readers to call their own Senators’ offices and find where they stood on the bill, they were able to zero in on their quarry by the process of elimination.

It’s such a delicious thing, a secret hold on something supposed to promote transparency and accountability in government. So [blogger] N.Z. Bear came up with this kind of checkerboard thing with all the senators. It had a parlor game feel to it, which is part of what made it so successful. … We started with two or three. Soon we had 20 or 30. I think they got down to three or four people left who hadn’t denied it, and then Rebecca Carr, who works for Cox News Service, got confirmation from Stevens’ people that Stevens had a hold. Then we also got word there was a Democratic hold. It may have been TPM Muckraker that outed Robert Byrd.

They then used that lesson in developing better interactive tools.

The idea was, could you come up with a very simple web tool to let people do investigations, do things in five minutes, essentially. …If we were doing a Center for Public Integrity piece and we’re using a lot of interns to do the work, we’re saying get this form, go to this office, get that. Now, because of some of this stuff online – contracts on line, grants on line, campaign expenditures online, now you don’t have to do that kind of legwork. You develop tasks where a computer should be good at, but isn’t, and instead you have a human being eyeball this stuff. So, this is where we’re at right now.

They used it in the earmark project, which asked readers across the country to research 1,800 earmarks in a Labor/HHS appropriations bill. Allison also trolled blogs and news sources, looking for stray references to such projects.

There was a guy in Delaware, a college student, who found this weird nutrition group that’s actually like a marketing agency for produce companies and the food and vegetable industry, getting all this money for this eat healthy campaign – which is really an advertising campaign for the fruit and vegetable industry. Just something that was a line item in a bill, nobody would have noticed it, but he dug down into it. It was in comments to somebody else’s blog. You would never ever find this unless you were someone who just incessantly goes through Technorati as I was doing, searching for earmarks, labor bill, or whatever.

You have this wonderful effort all over the country, but it’s so diffused. Some blogs, only a hundred people will read. Some, hundreds of thousands will read. You can have the best information on a blog that for whatever reason has only a couple 100 readers or less. We really learned that for this to be successful, you have to have a way of collecting and collating and presenting results, the findings. You have to have a central repository for it. I don’t think we’re quite there yet.

One big issue is, this type of effort is always going to be incomplete, very rough around the edges. But, Allison says, don’t worry about that. Every great story starts out as a fragmentary picture.

Maybe you get only like, 35 House members and 12 Senators. But say you did get that – that’s enough to say, is there a pattern here, is there a trend here, and is it worth continuing? Maybe it is, maybe it isn’t. But there’s a lot of things that you just don’t know until you start. It’s only by doing that initial research. It’d be great to find people and not have to spend the week it would take 1 person to do it. You could get 15 people to do it they could all do it in a day or two. It would be kid of an early warning system for what people should be looking at. And that would also serve to save reporters a lot of time.

For efforts like Sunlight and NewAssignment, the inquiry’s done in the open, not held back until the end like a standard journalistic investigation. That means other journalists, bloggers, or whoever, can freely borrow – and ideally, develop – the material. Stories become more free-floating, distributed efforts. Competition is defused.

If you are a reporter and you see this kind of effort, you could just jump in. Rather than it be a competitive – ‘oh we had this first and it’s our story’ – it becomes more of a, this is the buzz we’re picking up on, and we’re advancing the story. As an institution what we are able to do is, we can get that member of Congress on the phone, or his staff on the phone and put the question to them in a way a blogger can’t, or isn’t able to do or isn’t comfortable doing.

What about accuracy? The more participants such a “distributed” effort has, the greater the risks of errors or even sabotage. Allison explained how it worked on another project, where readers were asked to find out if politicians had put family members on campaign payrolls. Sunlight staff checked everything carefully, and the web interface was set up to discourage random or malicious contributions. They found a few mistakes, but overall, no big errors.

Shifting gears, though more and more government and campaign data is coming online in interactive forms – a tremendous opportunity for journalists of any stripe – some stuff is also disappearing.

I’ve head that a lot of stuff, particularly in energy, is no longer online, in the environment, a lot of infrastructure information, bridges and roads that need repair. There used to be website you could go to in the Department of Transportation, where you could see what were the priorities of spending, what roads needed repair. All of that is gone. Also, there’s a bill in the Senate that’s being blocked that would force FOIA offices to release things in the time that they’re actually supposed to release them. Anyone that that’s ever filed a FOIA request knows this almost never happens.

What kinds of stories lend themselves to the distributed, web-based approach? Like his former boss Chuck Lewis, Allison pointed to the relentless march of outsourcing government functions to private firms.

Where you’ve got to begin is with agencies that deal a lot with the public. The IRS is going to start to use private collection agencies, including one that had one of its partners in a bribery scandal in Texas. There will be more interactions with the public, more areas where the public starts bumping into this as time goes on.

Look at Medicare Part D [the prescription drug benefit]. These are all private companies administering this government benefit, and what is the experience of people who are dealing with these companies? How is that working? You’ve outsourced a government benefit to a whole bunch of private firms that are given a whole lot of leeway in how to structure these benefits. How good a job are they doing? Everybody has a relative who takes some prescription medication. That’s a huge reservoir of information to tap into. The Internet could be a great vacuum for information about people’s experiences with this. One thing you could do would be to develop a database of testimonials of people’s experiences with this, and their experience with companies. You would hear good, bad, how it could be better, where things are working, whether they’re not working. The advantage of this is, you get citizens engaged with this in a way that the end result is, you point to things that work and don’t work, expose the bad guys, point to programs people are happy with. There may be people who think this is the best it’s going to get, and they’re going to find out jeez, they get this, and this is the drug I take, and they get it for $18 and I’m paying $225 a year?