Let's build a huge distributed audio timeline

Let's build a huge distributed audio timeline

There's something I would like to try building, but it requires a nontrivial amount of help from other people. This one needs other people because I can't realistically travel to many parts of the world in a relatively short amount of time. I also don't have the kind of local knowledge for all of those areas. Residents of distant lands are already there and know the area.

I'm talking about a massive radio aggregator which works on a per-call basis. Right now, I just run a relatively tiny thing which has one full-time data source: my USRP, decoding the Santa Clara city system. Once in a while, I will stand up an analog (!) feed to fill in some coverage from another system. While that's running, it'll intermix with Sunnyvale, Mountain View, San Jose, or whatever else happens to apply.

I want to do this but much much bigger. I want someone to set up one of these systems in Oakland so we can hear what's going on with the Occupy people. I want all of the South Bay cities to be fully represented, even the dozen or more channels just for San Jose's police force.

When something happens which spills over into another jurisdiction, I want people to be able to just "snap in" another feed to their timeline and keep on going. Oh, Santa Clara PD is chasing someone onto a freeway? Add CHP. Hey, they got off the freeway in San Jose and they're joining in now? Add that area's channel to the mix.

Trouble is, there are far more systems than I can possibly log myself. It just collectively takes up too much CPU time and disk space. It really needs to be distributed to work properly.

The good news is that since the last time I thought about this, the cost of hardware has come down quite a bit. Instead of spending $1000 on a USRP and appropriate daughterboard, you can now spend $20 on a specific type of TV tuner (the so-called "rtlsdr") instead. It'll give you a nice 2 MHz swath of spectrum over USB.

You'll still need a relatively fast machine to keep up with that raw data if you want to do anything complicated with it. Decoding just one channel is easy, but if you want to split them out and record a bunch at a time, it's going to need some more horsepower.

After that, it's a matter of getting the call off the logger and out to a point where other people can hear it. The logging systems would need to stream out calls to the aggregator(s) as they receive them along with the pertinent metadata. There, they'd become sources for the timelines of whoever wanted to join in.

So then we come to the matter of money. The backend would consume a fair amount of bandwidth by pushing audio streams out to everyone. One way to handle this would be to require paid subscriptions to listen. Another way would be to require paid subscriptions to push data to the server. Still a third way would be to charge both sources and listeners. I don't think ads would work.

Now, yes, some people reading this are thinking about sites which already do this. Well, they do, and they don't. You can listen to scanner audio on them, sure, but I have yet to see one which handles things in terms of calls. They're all just glorified Internet radio stations.

Most of them are where someone took the line-out jack (if you're lucky) or headphone jack (if you're not) from a regular scanner and hooked it into the line-in (again, lucky) or mic-in (not) port on their computer. Then they started up the modern-day equivalent of Icecast to push a stream to the site's broadcasting servers.

This gives you exactly what the scanner hears. This really stinks when listening to archives, since there's tons of dead air in there. In other words, it'll take you an hour to listen to an hour of traffic, even if the units only spoke for 2 minutes total in there.

Guess what. On my system, if they open the squelch for 2 minutes, then it takes 2 minutes of your time to hear it. Then it goes on to the next call in the timeline, which could have been minutes or hours later. You don't have to sit there and listen to white noise and MP3 artifacts in between.

This dead air problem is also bad when listening live to multiple streams, because there's no logical point to switch. You just have to turn both of them on and hope you can keep up with that. Good luck!

Then there's the whole problem of using traditional one-channel-at-a-time scanners to follow a world which is inherently parallel. While you're getting channel A, channels B and C have good stuff happening which you will never hear. I've ranted about this before, so check out that post for more on this topic.

Also, how do you say "stop playing me traffic from the electric company and sewer people"? On a stream, you can't. It just isn't possible. With mine? Click the little filter button and say goodbye to that traffic.

How about this? You're listening to a bunch of calls and this one is boring. It's yet another license and registration check. You don't care. There's better stuff going on. Do what I do and hit the right-arrow key to jump to the next call. Life's too short to waste it on uninteresting calls!

This requires a shift in the way people think about this field. To do it and do it well, you need to abandon traditional line-in analog streaming and think in terms of actual calls. Then you need to abandon traditional round-robin channel recording and think in terms of parallel recording. Then you need to switch from a "click here to play a stream" mindset to a "here is your timeline with everything you requested" one. Think Twitter's timeline or Facebook's feed or that other thing which is only crickets.

Only then will this sort of thing move into the future.

All of this is possible right now. I've done my part. Who's next?