Interactive Fellows Friday Feature!
Join the conversation by answering Fellows’ weekly questions via Facebook. This week, Jon asks:
How do you deal with data deluge (too much email, too many tweets, etc.)?
Click here to respond!
Congratulations on winning the Knight News Challenge! What is your project that the award will be supporting?
Ushahidi’s open-source project, SwiftRiver, aims to help verify real-time information from the web, largely for journalists and disaster response teams. It’s critical for them to act and make decisions in the moment, collecting information from the public (from Twitter, email, SMS, blogs and news articles), and to sort it by accuracy or priority.
We’re trying to create algorithms that enable researchers and journalists to make sense of disparate pieces of data, to help verify that information on the fly — as opposed to in retrospect.
A huge portion of this is the computer science aspect. It’s all very heavy algorithmically, and so it requires lots of engineering talent. This can be a challenge for an open source project, but it’s a problem that the Ushahidi team handles through their community volunteers. The sacrifice is that things take longer to build that way.
For the first year I was working on SwiftRiver with just myself and my friend Matthew Griffiths, with a small budget to hire contractors. That went really well, and we were able to cobble together enough of a platform that people started to see where we were headed. But it wasn’t scaling fast enough to solve the problem in a way that was stable enough for people who wanted to use it. The funding from Knight helps us bring on new core engineering talent and data scientists, to help get things done faster.
How does SwiftRiver work, exactly?
I’m writing a white paper with Heather Ford, our new research scientist, explaining in technical detail what is called “Quantifiable Trust.” We’ll post it on Swiftly.org when it is complete.
But in short, people opt-in to using our platform to capture information in a given context or scenario. They’re aggregating massive amounts of information from the mobile and social web, and our platform adds context to that information as the user receives it. Because a lot of this data is being processed in the cloud, we have the benefit of having the global history of all our users, and all the content that they’re trying to capture from the public.
We use a number of techniques to enable the user to sort and prioritize information: location disambiguation, reputation monitoring, natural language processing, influence detection, and duplication filtering.
What do those techniques look like?
With location disambiguation, we look at the text, and we try to extract elements or clues that will tell us where the user was when they produced that content. If they mention a shop — let’s call it “Café Port-au-Prince” — we know that “Café Port-au-Prince” may exist in 20 places around the world. That eliminates a lot of other places it could be. Then we look at other clues: maybe a block name, a place name, or previous messages. We can bundle those together and say, “You’ve mentioned these place names, we can statistically narrow down the possibilities for a positive match.” The likelihood of your messages being about a specific “Café Port-au-Prince” in Port-au-Prince, Haiti, becomes more apparent.
It’s really just following clues in the language to try to deduce where the location might be. Because SMS doesn’t carry location data, this is a work-a-round. It’s only so accurate, but it’s often better than nothing.
With reputation monitoring, we look at the history of the person providing information. For example, maybe this person has contacted you in the past, or the person is a member of other communities and carries a great deal of authority there. In other words, just capturing the history of the content and the content’s producer to add a bit of historical context.
Influence detection helps users take the credibility they’ve created in one place, and take it with them to another. For example, Wikipedia editors, might want to carry their reputations with them when volunteering for an Ushahidi deployment. Our platform gives them the ability to carry that credibility around, so they’re not always starting from scratch.
There are a lot of tools out there that measure social influence. Marketers try to capture social influence to sell things. What we do differently is try to capture social influence — what I would call “social currency” — so that we can use it as the basis for an element in our verification algorithm.
We also use auto-categorization and tagging, which is looking at a message and extracting the uncommon keywords, so as to cluster messages that mention the same words.
The combination of all these different algorithms, with the help of input by the user, is what gives us the ability to use their definition of “accuracy” in a given context as the baseline of what to compare new real-time data to. This allows us to de-prioritize information that doesn’t fit their criteria of accurate.
SwiftRiver had a trial by fire after the 2010 Haiti earthquakes. Is the platform best suited for crisis situations?
I would say crisis situations are a relatively small target for the platform, although it’s Ushahidi’s primary focus. Swift is certainly useful for crisis and first-responders, but we’re also looking at newsrooms, research and data science. There are situations where brands and corporations might want to use something like this. So crisis response is just one element of a bigger picture of helping to make vetting any kind of information more efficient.
How has SwiftRiver been used so far?
There are quite a few case studies on our new website, Swiftly.org. But here’s a specific one: in April, during the Nigerian elections, there were four Ushahidi deployments in the country focused on the elections. And there was a group called NEAT (the Nigerian Election Aggregation Team) who wanted to mash-up information from all those Ushahidi deployments.
So we worked with them to aggregate data from all these different deployments, pass it through our platform, add different types of structure and context as meta-data — and then helped them pull it in to another Ushahidi deployment.
So in that case, the data structuring element of Swift, as well as the prioritization filters, were what they were interested in: the platform has many uses.
Appfrica started in Uganda in 2008 with the mission to build local capacity to do high-tech work in East Africa. It’s self-funded and for-profit, which means we do a lot of client work. Although we’re for-profit, we take its profits and reinvest it into philanthropic initiatives like our innovation hub, Hive Colab.
Hive Colab is a place where East African technologists can go to cut their teeth, work together, share ideas, get feedback from more experienced developers and investors and people from NGOS, universities, and so on.
We call it a “hive” because it’s a place with lots of activity. We have workshops where they get together to come up with smaller ideas to solve problems, as opposed to big ones that require funding.
Now that you’ve moved to D.C., is Appfrica continuing to do well without you?
Absolutely! I would say it would be a failed project if I had to be there to make it work. So it’s really exciting to see things still progressing nicely. There’s such a great team on the ground.
Why did you decide to move back to the US?
I originally followed my girlfriend to Uganda. And then I followed her back to the States where we got married. I’m a slave to love. [Laughs]
But beyond that, I still self-fund most of the work that I do. To sustain that, I need to focus on the projects that can help support the rest. My new project metaLayer is a tech startup that requires me to be here in the US to focus on the customer base and market opportunities.
Has the TED Fellowship helped you with metaLayer and your other projects?
In fact, I was just at SupporTED Collaboratorium in Lousiana. It was really cool to see so many TEDsters and the SupporTED coaches taking a hands-on approach to help shape the Fellows’ projects. They are helping us find ourselves and find our way with our work.
You’ve said metaLayer “augments reality.” What do you mean by that?
Our mission statement at metaLayer is to add layers of context to the mobile and social web. So not just meta-data like we were doing with Swift, but also visual context. The easiest way to convey really complex data sets to people is through visuals. With metaLayer, we hope to create applications that help people see the unseen stories in information.
So let’s look at our first product, metaLens. I could open up a book and wave my phone, with the metaLens app, over the table of contents. The table of contents has the names of places and things. metaLens would overlay information on the screen. For instance, it might show you the Wikipedia article for some of those names. Or, if it’s a place, it would tell you how far from that location you are right now. Or maybe we might show you all the other books that reference similar names and places. It’s sort of like Google Goggles, but going a step beyond search. The idea is to add value to the content that’s being viewed at the time. This is a very passive experience, you just hold the phone up and see the world through a different lens. So that’s one product, which we hope to be able to demo at TED Global in July.
There’s a second product we’re developing, that seeks to help expose relevant information mined from your social graph. That means from across all your files, all your tweets, your address book, all the blogs you read, all the people you know on Facebook etc. It might be useful to have some of that information recommended back to you when you’re in a situation that requires it. Again, I’ll use a Google example: think AdSense but for your own useful information instead of ads.
Not everyone has the luxury of being a data scientist or having a background in information analysis. So the more we can present complex stories visually, I think the better off we’ll all be.
There are many aspiring social entrepreneurs out there who are trying to take their passion and ideas to the next level. What is one piece of advice you would give to them based on your own experiences and successes? Learn more about how to become a great social entrepreneur from all of the TED Fellows on the Case Foundation’s Social Citizens blog.
If you have a good idea, the most important thing, if you want to be successful, is to execute on that idea, no matter how sparse your resources.
When I started Appfrica, I used my own savings. Although I thought it was a good idea — and people tell me it’s a great idea now — at the time, no one would support it. The only reason it was even moderately successful is because I did it anyway.
I can say that, all across the industry, most success stories come from perseverance and tenacity; not necessarily having all the right resources at the right time.
You have been working on a science fiction novel, I hear.
I was working on a science fiction novel called Muxtionary. I still am, but with so many different projects, I have to prioritize the ones that pay the bills and feed the family. I still write lots of fiction. I’ve never published anything. I do it because I love it.
Muxtionary looks at the future development of Africa, the convergence with technology, and how it’s all playing out. Some of that might be good, but some of that will be negative. It looks at subjects like pollution, over-population, automobile safety, self-replicating machines, bio-engineering …. What would that look like in a few years, when it all goes wrong and the developing world becomes the dumping ground for these things? That’s the theme for the book.
‘Mux’ is short for multiplexer which is a device that takes two different input signals and outputs them as a single result. So Muxtionary is mash-up of the word ‘Mux’ and ‘Missionary’ — those who traditionally go to developing countries to propagate religion. In this fictional world people go to evangelize technology in the same way.
Currently I’m writing a nonfiction book about some of the research I’ve done while working on SwiftRiver and Ushahidi. It’s about crowdsourcing and how many new technologies are successful because they augment the nascent relationships and connections between people. It’s an audiobook and podcast called Cult of the Crowd.
You used to be a music and film producer and audio engineer. What caused the shift?
I went to art school, Savannah College of Art and Design. I never thought I would be in the computer science field. The arts are where my heart is.