Photos: Ryan Lash

Jason Pontin, editor and publisher of MIT Technology Review, wants us to think about why we can’t (or think we can’t) solve big problems anymore — what is our generation’s moon landing? Some people blame the culture of Silicon Valley, or VCs unwilling to invest in big problems, but Pontin doesn’t buy this excuse. The real problems are that humanity’s big challenges are hard, our political systems are unwilling, and too often we don’t really understand the real issue. Landing on the moon, it turns out, was relatively easy. “The solutions of our future will be harder won,” he says. A sobering start to the morning.

In a hilarious talk, David Pogue shares some basic tricks for using our technology — tricks that you might think everyone knows, but they don’t. For example, hit control (or command) and “+” to make the text in a web browser bigger. When writing, double-click a word to highlight just that word. (We’re asking him for the list.)

2006 TED Prize winner Larry Brilliant, the president and CEO of the Skoll Global Threats Fund, is here to tell us the good news about pandemics. Hurray! This is important; as he and his team helped to lay out in the movie Contagion, which they advised on, pandemic viruses are a huge, insidious threat to humanity. But with social media, participatory surveillance, better policy and better regional cooperation, global pandemics might become a thing of the past. “I think we can end pandemics in our lifetimes,” he says.

Ghislaine Maxwell gives a stirring call to care about the oceans — a resource that is held by law to be for the benefit of all, but in reality is being exploited by the few. She proposes six things we can do to help: 1) Enforce the Public Trust Doctrine; 2) Demand more marine public areas; 3) Adopt models that produce more revenue without as much waste; 4) Ban wasteful fishing practices; 5) Fish sustainably; and 6) Come together as a community around the seas. We are all citizens of the oceans, after all.

“Look to the person to your left; look to the person to your right,” instructs Scott Noggle, director of the New York Stem Cell Foundation Laboratory. ”One of you will get Alzheimer’s by the time you are 80.” This is not a cheery thought. “This is a catastrophe,” confirms Noggle. Yup. He’s here to tell us about his work, which involves taking living cells from cadavers of those who died from Alzheimer’s. He and his team have figured out how to re-create stem cell lines, and therefore brain tissue, to try and figure out how the disease starts and develops–with the end goal of creating more effective therapies to treat the disease. Astonishing.

Dan Miller, Managing Director of the Roda Group, is concerned with the growing food crisis facing the world — and is looking for solutions. He’s found one possibility: hydrogels, chemicals that can hold 100 times their weight in water. These can be put into the soil at the same time as seeds and fertilizer. Because of the way the gels retain water near the plants, this could increase yield while reducing water use. Some convincing tests on broccoli make his point.

Jessica Green is here to talk about the microbes that both define who we are and that exist in their own ecosystems on everything we touch. She’s been working with architects and biologists to take samples from rooms at the University of Oregon to get a deeper understanding of the microbial community within space. “Bathrooms are like a tropical rainforest,” she says. “Offices are like temperate grassland.” The implications for designers of this genre she calls “bio-informed design,” particularly for those thinking about designing air systems or working in health care, are huge.

Tony Tjan studies entrepreneurs, and tries to work out what makes them successful. He has found four attributes that each can contribute: Heart, Smarts, Guts and Luck. He says there is no one way to success, but a key is the self-awareness to understand which part is their own primary driver.

Harper Reed, CTO of Obama for America, is here to talk about how politics is changing, and in particular about what will be important in 2016. On his list: micro-targeting; micro-listening; micro-media buying. We’re going to get a lot more focused, in other words. Other challenges: voter suppression; voter contact; and potential cyberattack, Reed’s biggest fear. Did you know this: The presidential campaigns of both John McCain and Mitt Romney were hacked by a foreign entity. “We were safe, because we invested in security,” he says, but he doesn’t think it’ll be so easy next time around. The solution? Trust the experts. Get the right people in the right place, and let them do their jobs to the best of their ability.

TED Fellow Julie Freeman is an artist who thinks about how to represent data in art. She was asked to curate a set of artworks based on data for the Open Data Institute, and she found some remarkable examples, such as a vending machine that only dispenses snacks when there is news of an economic downturn.

David Peterson creates languages for a living, including the language Dothraki, which he developed for the television series Game of Thrones. He’s here to give us an insight into how he does it, and to take us on a whistlestop tour through the evolution of language. Why go to all the trouble? Fans, of course. Every single detail of a hit show like GoT is analyzed in depth; results are shared instantaneously, and they’ll realize quickly if a fake language is systematic or just gibberish. This respect for viewers might be the difference between a hit and a multimillion-dollar flop. That’s why it matters.

And finally, David Pogue, who turns out to be a former Broadway conductor as well as technology writer for the New York Times, returns to sing his new composition, ”The Twitter Song.” a show tune take on living in the 140-character age.

Deb Roy: The birth of a word

His team wondered: could this kind of analysis be applied to television or, say, Twitter to discover communication trends?

These are the kinds of questions that today’s TED Weekends on the Huffington Post explores. Here, three of the great essays that are available now for your reading pleasure. 

Deb Roy: The Birth of a Word

Three trajectories came together in 2005 and took me to new frontiers of cognitive science (and subsequently, it turns out, the media industry).

• The first trajectory: I began to see an unexpected connection between my research in robotics at MIT and theories of how children learn to talk, leading to studies of child language that I did with my wife and collaborator Rupal Patel over the past decade.
• Second: The era of Big Data was dawning, and the far-fetched idea of video-recording everything that happens in a home had become a practical reality.
• Third, Rupal and I learned that we were expecting our first child in July 2005.

This confluence of events sparked an unusual study of child language featured in the first half of my TEDTalk. Read the full essay »

Gayatri Devi: How Do I Improve My Memory? Forget More!

Do you know what is essential for a good memory? The ability to forget. To completely and thoroughly forget. Forgetting, like breathing or sleeping, is physiologically normal. This is at odds with our modern compulsion to record and remember everything and is a perfect recipe for anxiety.

Deb Roy, a cognitive science professor at MIT studying language, recorded 8-10 hours daily of the first three years of his son’s home life. He compiled a quarter million hours of audio and video, creating a 200,000 gigabyte “ultimate memory machine.” (Most computers store about one gigabyte.) Consider how much information each of us is exposed to in 24 hours, on streets, subways, screens and in sleep. Imagine recording and remembering all this. Thankfully, we were never meant to.

Fact: We are evolutionarily programmed to forget. Our brains evolved over millennia with built-in forgetfulness. Our brain is engineered to remember tastes, smells, voices, touch and visions, not names. Our brain is engineered to solve problems (How do we keep track of cattle? Mathematics; How do I communicate? Language), not remember disjointed facts. A fact not linked to a sense, an emotion, or a concept is quickly forgotten. Read the full essay »

Ben Hecht: Big Data Gets Personal in U.S. Cities

Much has already been said about how big data is dramatically changing the way that organizations make decisions. Today, more data is being created from more places than ever before. Blogs, Facebook, YouTube videos, retailer loyalty cards, mobile phones, and sensors on buildings are producing tons of data daily. Private sector companies, in their real-time data warehouses, are storing, analyzing, and harnessing it to help them to better understand their customers, dynamically alter pricing based on real-time demand, and even change their business models. And, increasingly government is putting the wealth of data that it generates to work to increase efficiency, save dollars, and create more proactive policy. But, as Deb Roy highlights in his TED Talk, the true promise is where the numbers and patterns from this data connect and become personal — enabling us to understand and to respond to humanity and the world in ways previously unimaginable. This type of analysis has infinite potential for improving the human condition on an ongoing basis; and strengthening people’s commitment to our democracy. Already, in U.S. cities, we are seeing many promising signs of the transformative personal application of Big Data:

Mass Personalizing of Government Data and Services: The movement towards open government data in the U.S. has already had huge implications for the relationship between citizen and government. Read the full essay »

Artist Julie Freeman uses data as a source material to make biologically inspired artworks — giving musicality to the movement of fish and expressing city lights in the quiver of moths’ wings. Now she’s finding ways to translate data so that we may gain new perspectives on what it’s trying to tell us.

What do you do?

I make artwork that allows me to be curious about nature in different ways, and to share that curiosity. The driving force behind my work is, generally, What is it about natural systems that are so compelling? How can we understand more about them to get a fresh perspective? And how can we understand phenomena that exist beyond our own sensory perception? That’s where the technology comes in. It allows us to get to grips with hidden elements of biological systems, and can allow us to see or experience things in new ways. In my practice, I use technology as a kind of communication bridge between the natural world and ourselves.

Technology is often seen as something that is at odds with nature, something that tries to control, change, or supercede. My view is that we can use technology to try and understand the natural world better, getting a deeper knowledge of biological systems will allow us to acknowledge and empathise with nature, to garner peace with our environment.

What’s the value of art that translates data?

In addition to exploring how natural systems can be translated, I’m also preoccupied by how much our lives rely on data in the form of data-driven decisions — from the ones that we make personally (think social networks) to those made by our employers, suppliers and governments. It feels important that artists should be working with data not only to reflect what’s happening in the world, but also to help create a level of understanding that in some way reduces fear. The more information is generated, the more our need to understand it grows. We need to be literate to comprehend the interpretations of data that we are being exposed to. And we need to find a way of talking to each other about data that is clear and understandable.

The Lake (2005): A site-specific installation in which 16 freshwater fish were tagged in their natural environment. The tracking data was used to create music and animation display by the lake side in a 80ft tall cylindrical structure — an experience composed by the fish.

For example, there’s a dynamic light sculpture by Fabio Lattanzi Antorini that flickers in response to the number of crimes against humanity being perpetrated. It pulls information from a series of different real-time news feeds, parses them into a sequence and then displays that as a rhythm of light. So although you don’t know any specifics, you do know is that crimes are happening and that it is being monitored. That’s possibly enough to have an awareness without being overwhelmed with detail. Of course it is important that the awareness then leads to change in some way.

It’s no news that there is exponential growth of information. People are trying to tell us their side of the story all the time. It’s overwhelming. We need to find some way of managing that process of absorbing information, and learning from it, without it becoming a pressure. A lot of data-driven artworks do that by condensing information and then giving you a simplified idea.

Digital Wave (1998): Participant faces are manipulated and streamed down a giant wave-shaped interactive digital sculpture measuring 45 x 6 x 10 ft.

What made you want to do The Lake project — one of your early works (shown above), where data gathered from the movement of fish in water was translated into animation and sound?

Again, I was curious: I wanted to understand more about the underwater environment and why is it that we’re so fascinated with fish. How does a fish behave in its natural environment? Can the movement and behaviors that fascinate us be translated into art? More personally, part of the reason I wanted to pursue this idea was because I come from a fishing family, and I have fish-like tendencies — I’m an active swimmer, a total water baby. And, being a geek, my artistic tools were technology. It started off as a prototype and then around eight years later became a full-fledged, fully supported project. I let my curiosity out by using technology to get to grips with how fish swim and what their relationship to each other is. I learned a lot from the project, and the data I collected is very unique — I’m still working on further analysis now.

Would you say that your primary interest growing up was biology and the natural world, and then technology just became your tool?

Yeah, I think so. Biology, always. When I was very small, I used to be petrified of animals, anything apparently — cats, rabbits, dogs. My parents got so fed up with my fear that they ended up buying a dog. I remember coming home one day and the dog was in the backyard. I freaked out, screeching “I can’t live here. I can’t live here anymore” — a total drama queen. I was only about six or seven. About an hour later, I was rolling around with the dog. From that point onwards, my whole view of animals shifted completely.

Thinking back on it, I just assumed that these critters were going to bite me or attack me. It was the unpredictability of animals that made me nervous. I guess that was the beginning of wanting to understand more. I’m still interested in critters of any shape and size, but also things at the nanoscale, and more increasingly, this idea of how biological systems get represented by data, and what if data *was* a biological system? How would we see it if it wasn’t digital? What would it be like if it was akin to something like a slime mold, or nematodes — a living entity?

Why didn’t your curiosity about biology translate into a biology career?

I studied biology through school. I had some issues when I was leaving school age where I ended up being told I couldn’t continue studying at the school, and I had go and do something else. I stumbled into graphic design at the local college, and I took art at the same time. That steered me into the design world, and then I studied design technology at university. Based in the mechanical engineering department it was then that I first started putting together computers and was introduced to the product design world.

It was only when I studied for my Masters in Digital Art, all the concepts I was coming up with linked back to something from the natural world. The idea of tracking fish (later The Lake), my animations about plankton — all my work had a biological element. I got really interested in artificial life systems and how we can mimic life through cellular automata. All the software art that I was developing then was all based on this idea of unpredictability and life systems within the machine. From there I started thinking, well if I’m building them in the machine, how can we start making the software and the hardware connect with the real life systems? It became a fusion of technology and biology in a very literal way.

Lepidopteral (2012): A multi-object kinetic artwork that responds to environmental data fluctuations from remote sensors.

Is data then the bridge for that?

Data is a byproduct of our curiosity. Nearly every scientist I know and many artists that work in technology all have this relationship with data where data is the substance that sits between initial curiosity and knowledge and understanding. It plays such an important role. How can data been seen as something that has an ephemeral behavior which changes depending on how you treat or perceive it?

I’ve read and heard people say, “Data is information.” Actually, I don’t think it always is. Data is much more akin to an artwork in some ways, because as a mass it has this potential ambiguity and subjectivity that exists before an analysis happens or before it gets processed in some way. And I think that bit is really quite intriguing.

Do you mean information isn’t information until we interpret the data that’s there?

It’s tricky. I was thinking about this: If all data is information, say, does that mean that everything we see is information? Does it mean that everything has got some kind of message behind it that it is trying to impart which we can then gain knowledge from? And I was trying to think, well if that is true, does it mean that art is information as well? And as soon as art becomes information then suddenly it seems less interesting to me, because I don’t know if it should be information. I think it should be experience. But then could you deconstruct experience into being a process of information gathering? I don’t know. I tied myself up in knots with that one.

Tea Flock (2011): Migrating rituals and emergent behavior of grouped objects are displayed in this kinetic work, using data from imaginary migratory tea-birds that fly to and from tea-growing countries and the UK.

What types of data you work with?

I’ve worked with all different types. I’ve worked with data that’s been generated by sensors — tracking data. I’ve worked with data that has been collected as sound files and spoken word — sonic data. I’ve worked with data that’s been processed by other people and then delivered either as streams of values or as video feeds. I’ve worked with light levels from different parts of the world. I’ve worked with geographic data about trajectories of bird migration from India and other countries. And I’ve worked with data collected from people. There isn’t any piece of my artwork that hasn’t used data in some way, in some format.

In terms of making art with different types of data, using data as a material, it’s really important to define it so that we know how it manifests within the work. The definitions range from how the data is delivered, where it’s coming from, the temporality of it and also things like: Does it relate to the living world? Is it natural data? Does it relate to the social and political world? Does it relate to a personal individual? Is it economic data? Is it geographic data? Is it generated, real-time or processed data? These are simple descriptions, but they’re important to understand that for data-driven art, the type of data within it can give you a different experience of the work.

How do you go from a sense of curiosity about something to coming up with designing an artwork and making it happen?

To give you one example, a few years back I started making moth capture devices to film moths in action. Sometimes, I’d string up a big white sheet in a woodland and put lights behind it, to attract moths. Every time, nearly every other kind of insect joined the party, and only just one or two moths if any! That made me wonder, if I was a moth, why would I be attracted to light? I wanted to capture, on video, the flickering moth movement.

I went swimming in a lake in Italy and saw what looked like flowers all over the lakeside. But when I looked closer, they were clumps of little lilac butterflies. And they were almost static, they were moving really gently. It was fascinating to watch. Every now and then they’d flit about from clump to clump. I thought, it’s interesting that when you think of a butterfly or a moth, you think of a very flickery, very high-speed flying movement, but actually spend a lot of their time still, just being quite peaceful. These experiences inspired me to make a piece of work, Lepidopteral — these small plastic moth-like creatures that flap in a very organic way in response to light levels in Berlin.

Why Berlin?

Berlin was just a reliable light feed that I found on the Pachube website. It appealed to me to have a remote light level driving the system, a data feed that’s nothing to do with where the work is — and is outside of my control. A lot of my works have this element where I set up a framework using different hardware and software, and then it takes a feed from something — from the environment, or from animals — which directly affects how the artwork performs, whether it’s a sculpture or animation or sound work. It’s not random, but uncontrollable. The work acts as a conduit between something in the environment and something in the gallery or in the studio. When everything gets switched on, I don’t know what’s going to happen, so I don’t know how the final artwork is going to be until it actually comes to life. I’ll spend months, often years, working on these projects, and that’s the moment that really gets me, because I can’t predict it. I also feel like I’ve built something that is communicating in a new way, that it’s not just out of my head, but it’s coming from somewhere else. That’s a great thing to be able to share with people.

Specious Dialogue (2007): A pair of concrete forms spew an emotional dialogue, they bicker, coo fragments of love, they shout, scream and whisper, they are lonely lovers or clinging siblings. Randomised data produces a conversation that is plausible but false.

What are you working on now?

At the moment, for my PhD I’m working on how we can experience different types of data sets from natural systems through physical objects. I’ll be looking at how those objects relay the essence of data and whether you can determine whether that data is biological, technological or economic, simply through a sense of movement. So it’s kind of like the body language of an artwork in a way. There’ll also be viewer-response feedback mechanisms. So for instance, if you saw a data-driven sculpture that was kind of slumped and it made you feel a bit unsure about why and so you sort of slumped a little bit, then that could be detected and then the object could kind of chirp up a bit, which you’d probably try and mirror, perhaps subconsciously. It’s all very early days.

Will you be making objects for that?

I’ll try! I’m excited because it’s a computer science PhD at Queen Mary, and they’re happy for me to make artworks that explore psychological response to data-driven artwork in a scientific arena, but also in an artistic space as well. We’ll make the artworks and then use them as a stimuli in an experiment and have them as an artwork in a gallery, and see if there’s any link between the two in the way that people perceive the work depending on what context it’s in. It’ll be interesting. It’s quite a challenge to establish whether, if you’re designing an artwork as part of a science experiment, is it still an artwork in itself. Where is the line between rigid design and artistic flow? I think a lot of artworks act the same way as a stimuli in a psychological experiment, even if the artist doesn’t realize that’s what they do.

Tell me about the Open Data Institute show you recently curated.

The Open Data Institute was set up by Sir Tim Berners-Lee and Prof Nigel Shadbolt, and their idea is to create a culture of open data. A key premise behind the institute is to teach people what they can do with open data, and how they can make their data open — all the way from political leadership level to schools and universities and small businesses.

Ellie Harrison’s piece Vending Machine, chosen by Julie Freeman for exhibition at the Open Data Institute, dispenses free crisps in response to recession data. Photo: Ellie Harrison

When I visited their new space in London, I said, “What you need is data-driven artworks.” I was worried that data is seen as dry information, that it’s seen as a very academic or commercial thing. I wanted to make data very tangible for their visitors. We curated and commissioned nine works, nearly all of them are physical pieces. There are a couple of dynamic sculptures. There’s a kinetic wall-based piece, and a painting. There’s a newspaper and an archival book. There is only one piece that is screen-based. It is important to have physical objects in the space, displaying data in a energetic and abstract way. I think the collection we put together surprised them.

What was the response?

The idea of data as a material in an artwork was quite new to those working in the space, so there was some apprehension. As they saw the artists setting work up and building it, they formed a bond with the works because they got to know the people behind them and to understand them quite deeply. Now, the artwork is what everyone talks about. There’s one in particular — by artist Ellie Harrison — an old 1970s vending machine that feeds off of economic data. If there is talk of economic crisis or if there’s a big budget announcement in the news, it dispenses crisps for people to take for free. Staff come in the morning to see if any “recession crisps” have been dispensed, getting a very rapid grasp of the night’s fiscal activity — 10 bags of crisps in the drawer generally means trouble brewing. It’s really interesting to watch that piece in action because it’s a perfect venue for it in a weird way — not many artworks work in a kitchen!

How has the TED fellowship had an effect on you?

Being a Fellow has broadened my ideas about how far I can push things and how important and serious my work is. In practical terms, any question you could have about nearly any subject, you can just email one of the Fellows and they generally know exactly what you need. In terms of resources and opportunities, that is invaluable.

Before I met a lot of the other TED Fellows, I didn’t quite understand where I fitted in — I’m such a kind of mix of things. Being with the TED Fellows, it’s just like: good grief, yeah. It’s completely fine and normal to be doing what I’m doing. Just having that shared knowledge of people all over the world doing the same kind of thing, and being part of that gang, is reassuring. It’s definitely something I carry with me in terms of facing any fears.

TED speaker Rick Smolan is asking students between the ages of 13 and 18 to become “Data Detectives” for a new project he’s unveiling today — and that he will talk about at TEDYouth this Saturday.

By answering a 20-question online survey, teenagers will help build a data set that will let then compare themselves to teens all over the world. Some sample questions from the survey: “Are you more like your mother or father?” “How do your parents discipline you for bad behavior?” “How do you get to school: by bus, public transportation, limo, donkey, or skateboard?” The survey is anonymous and takes about 10 minutes to complete.

Take the Data Detective survey here >>

You can watch a FREE livestream of TEDYouth on Saturday, Nov 17, 1-6pm EST. Just bookmark this page and check back at 1pm Eastern on Saturday: http://new.livestream.com/tedyouth

In his recent TEDTalk, “What doctors don’t know about the drugs they prescribe,” Ben Goldacre sounded a warning about the vast numbers of pharmaceutical studies that go unpublished. “Positive findings are about twice as likely to be published as negative findings,” said Goldacre, noting that this is a big problem because it means doctors are prescribing pharmaceuticals without full knowledge of their side effects and overall efficacy.

However, Goldacre is excited about a hint of change displayed in this article from the British Medical Journal. In the editorial, BMJ editor-in-chief Fiona Godlee tips her hat to pharmaceutical giant GlaxoSmithKline for opening its vaults and allowing access to its trial data. The system is not perfect — all requests must travel through a panel and be deemed “a reasonable scientific question” before data will be released — but it is a step forward. At the same time, Godlee pledges that, beginning in January 2013, the journal will only publish studies on drugs and medical devices when there is a commitment to make all data available upon request. Meanwhile, she writes that the publication is continuing its three-year battle to gain access to all the data existing for the drug Tamiflu.

This, of course, does not mean that the problem is solved. But it is a sign that a sea change may soon be under way in the area of medical data.

Just by living our plugged-in lives, each of us is producing a constant stream of data. Little snippets are left behind of what we search, what we buy, where we go, what we tweet … This endless flow of numbers is referred to as “big data,” data sets so large that they require sophisticated parsing to give them meaning. But big data has the potential to tell us a lot about ourselves — unearthing patterns in information flow, energy consumption, weather patterns, disease spread, education trends, and more.

At first glance, big data may not sound like a topic for teenagers. But TED speaker Rick Smolan is on a mission to make it not just accessible but fun. Smolan — who held a conference called The Human Face of Big Data in New York City a few weeks ago featuring TED alums Juan Enriquez, Deb Roy, Esther Dyson, Aaron Koblin and Jer Thorp — is asking students between the ages of 13 and 18 to become “Data Detectives.”

For anyone in the age range, becoming a Data Detective is easy. By answering a 20-question online survey, you’ll be helping to build a data set that will allow you to compare themselves to other teens all over the world. Some sample questions from the survey: “Are you more like your mother or father?” “How do your parents discipline you for bad behavior?” “How do you get to school: by bus, public transportation, limo, donkey, or skateboard?” The survey is anonymous and takes about 10 minutes to complete. Take the Data Detective survey here >>

By participating now, you won’t see much happen. But over the next few weeks, as more and more students donate their information and time too, the data will compound. On November 12, the Data Detectives website will be open for business, allowing you and other students to visualize the data in real time, with explosive animation by R/GA.

The juiciest insights from the data set will also be shared at TEDYouth on November 17 in New York City, as well as at the 100+ TEDxYouthDay events happening worldwide the same week.

Hey, it’s a detective job that doesn’t even require a magnifying glass.

In the information age we have access to more data and knowledge than at any previous point in human history. But more accessible data doesn’t necessarily mean more processable data — tax returns, court cases and newspaper archives may be available to the public, but they are often hard to interpret and understand. Data artists process this mess of information for us and break it down into comprehensible pieces, often enhancing it by placing it in deeper contexts and by finding surprising patterns. The designers and artists below are on the cutting edge of this work, experimenting with new technologies to take numbers and facts and turn them into beautiful multimedia works of art and knowledge.

Nathalie Miebach: Art made of storms
Nathalie Miebach’s art is both beautiful and informative: she sculpts colorful models using weather data, and then converts this information into musical scores. Thus she makes weather patterns — something inherently invisible to the naked eye — “visible, tactile and audible.” In this short talk from TEDGlobal 2011, she describes her nuanced ways of translating information into different mediums, blurring the lines between art and science.

Aaron Koblin: Artfully visualizing our humanity
Numbers can humanize us. Artist Aaron Koblin uses data to discover the patterns we make as a collective whole, so that we can better understand society and ourselves. Koblin’s groundbreaking work harnesses modern technology to produce art that is sometimes crowdsourced and collectively produced, and sometimes hyper individualized. Here, he describes his work at TED2011.

Hans Rosling: Stats that reshape your world-view
This is the first of many great things to come from data whiz Hans Rosling. In this seminal talk from TED2006, Rosling beautifully and energetically illustrates fascinating trends about global health and wealth distribution.

David McCandless: The beauty of data visualization
Graphic designer and “data detective” David McCandless is trying to make sense of the world by compressing and overlapping information, he says at TEDGlobal 2010. His gorgeous data visualizations squeeze an enormous amount of knowledge into digestible tidbits, throwing the data into new light as he contextualizes it. By making information beautiful, he makes it make sense. (Check out McCandless’ latest work, an interactive visual that lets you calculate the number of alien civilizations in our galaxy.)

Jer Thorp: Make data more human
Information artist Jer Thorp traverses the “terrain of data” to tell narratives of human experiences. These elegant data-driven stories are based on information collected from smartphones or newspapers, but breathe life back into the lives we live. In this talk from TEDxVancouver, Thorp explains how this human-centered approach to data analysis is the best route to take as technology becomes more and more central to our daily lives.

Hans Rosling: Religions and babies
Data master Rosling takes us through the effect of religion on birth rates across the globe in his most recent talk from TEDxSummit. Intersecting separate data sets enable Rosling to deeply explain human patterns of population growth and reach surprising and fascinating conclusions about how we are born and how humanity will continue to grow. (Rosling has a lot to say and a lot to show us. Check out his prolific library of talks about data visualizations.)

Silver’s book looks not only at political forecasters, but also at those who predict hurricanes, poker games, national security risks and the stock market. And it’s a good understanding of probability that tends to make for success.

Silver writes, “Our views about predictability are inherently flawed. Take something that is often seen as the epitome of randomness, like a coin toss. While it may at first appear that there’s no way to tell whether a coin is going to come up heads or tails, a group of mathematicians at Stanford is able to predict the outcome virtually 100 percent of the time, provided that they use a special machine to flip it. The machine does not cheat — it flips the coin the exact same way (the same height, with the same strength and torque) over and over again — and the coin is fair. Under those conditions, there is no randomness at all. The reason that we view coin flips as unpredictable is because when we toss them, we’re never able to reproduce the exact same motion.”

Check out the book, out today, or read an excerpt explaining why you should never call a weatherman a moron. Below, watch Silver’s TEDTalk from 2009, which is still sadly relevant.

Nate Silver: Does race affect votes?
How did race play out in the 2008 presidential election? Math whiz Nate Silver unpacks the many layers of variables and gives fascinating insight into how town planning could promote tolerance in the future. Because, as he says, when something is predictable—it is also designable.

Wired UK brings news of a fascinating collaboration to explore data from NASA’s Kepler mission, which has brought back evidence of thousands of exoplanets, or planets outside our own solar system. Some exoplanets are huge, some tiny, near or distant, hot or cold — and some may even be Earth-like, offering clues to the origin and existence of biological life. It’s a truly inspiring dataset:

Data artist Jer Thorp wanted to bring Kepler’s discoveries to life, so he hooked up with John Underkoffler, the inventor who built the gesture-based g-speak spatial operating environment showcased in the 2002 film Minority Report. “I was curious what these planets actually looked like,” says Thorp. “How big, how hot, what they were made of.”

“It was such a deliciously spatial dataset that it cried out for gestural control,” says Underkoffler. So over five days in May, the duo created Exo, an immersive g-speak interface at Oblong Industries’ Los Angeles studio.

Check out many more images in the Wired UK story >>

Meanwhile, learn more about Kepler from two TEDTalks. One talk was given by Dimitar Sasselov while results were still coming in: “How we found hundreds of potential Earth-like planets.” Then last summer, TED Fellow Lucianne Walkowicz gave a quick primer in “Finding planets around other stars.”

And while you are at it, watch these talks from the two data collaborators as well:

Jer Thorp: Make data more human >>
John Underkoffler: Pointing to the future of UI >>

Top photo: James Duncan Davidson. Bottom photo: Wired UK.

What is the nature of the data problem?

We now live in a world where our every purchase, every non-purchase, every curiosity, every tweet and “like”, every movement, is being tracked. Like it or not, We are Data. We are the customer, but our data are the product. Right now, we’re at a critical juncture where we can use our data to empower people to topple dictators, or empower dictators to suppress citizens. We can broaden economic opportunity by empowering micro-entrepreneurs, or further consolidate resources in the hands of a few. Once we are able to identify the data we generate, how we store it, who has access to it, and how we use it, we will more likely benefit from the explosion of data we generate, while avoid being harmed by it.

What does it mean to “increase the vibrancy” of data and to “democratize data”?

To quote Jon Gosier, “Vibrant data is data that doesn’t exclude participation from people that lack expertise.” Right now most people have very little, if any, control over all the personal data they generate — and large corporations are battling for control of this valuable information. For the privileged, maybe it just means dealing with a few more targeted ads, but for others it could mean unwarranted surveillance, profiling, discrimination … even imprisonment or death. Increasing the vibrancy and democracy of data means enabling everyday people to benefit more directly from the explosion of data, while also avoiding negative consequences. Doing that requires considering challenges at all stages of the “life cycle” of data — from how we generate data and store it to how we access it, share it, find meaning in it and derive value from it.

How did you and David Gurman go about starting to map this problem?
The cool thing is that we didn’t map anything! We mobilized a community to map the “ecosystem” of this complex problem — in other words, are all the moving parts to the problem, and how they influence one another.

To do that, we collaborated with Juliette Powell (The Gathering), Emily Aiken (The Story Studio), Jerri Chou (The Feast) and a research group at Intel Labs to extract information from a community. The first challenge was simply to identify all the moving parts, which we called the “mini-challenges,” to democratizing data. So we interviewed a community of experts that ranged from cyberluminaries like JP Barlow, Bill Joy and John Batelle, and a group of TED Fellows working on the front lines of data for social good. We asked them: “What are the biggest challenges to democratizing data?” From those conversations, we broke up the big problem into about 90 mini-challenges.

Then David and his team at Brainvise were key to developing an online tool, the Tru North MAPPR, to enable a community to remotely map how all these challenges are related. With that tool, people identified where solving one problem has a strong influence on another — either by making it better or worse. In the first phase, which lasted less than two weeks, I was blown away that a community of about 40 people voted on over 6,500 links between the 90 mini-challenges to create a network of ~3,800 links (many with multiple votes). We partnered with Sean Gourley’s new company, Quid, to visualize this network. To my knowledge, it’s the first ever crowd-mapped network of a complex problem!

And what did you discover?

From the structure of that network, we identified four Grand Challenge areas for democratizing data: Digital Infrastructure (broader access to the underserved), Digital Trust (protection of personal identity, and reputation systems for accountability), Digital Literacy (widespread access to intuitively asking questions with data and thinking critically about the answers) and Platform Openness (the ability to copy, edit and customize platforms that facilitate finding meaning from data). I think we did an amazing job identifying some key areas at the foundation creating a longer-term, more vibrant data ecosystem that benefits more people and encourages more people to participate.

These Grand Challenge areas are still really broad and will continue to evolve with more input on vibrantdata.org. But they were defined by a community, and they target problems that, if solved, could solve many others. Right now, we are calling for more people to join us in the problem mapping — to grow it and see how our understanding of the problem evolves with more input.

Can the Tru North MAPPR be repurposed to solve other complex problems?
Yes, and that’s the exciting thing. This tool we’ve created can be customized for other types of problems. At Tru North Labs we are at the same time exploring the frontier of network science — there are still big unanswered questions about what we can decipher from the structure of a complex network about how it operates and what controls it. And ironically, some of the theory I’m working on suggests that the more complex the problem, the more difficult it has been to change, the more “diffuse” its control, the easier it might be to understand. My goal is to get others interested in this general approach to having community input on “breaking down” ANY complex problem — whether it’s conflict in the Middle East or climate change — and collectively defining some important challenges in need of creative attention.