Andreas Weigend
Social Data Revolution
MS&E 237, Stanford University, Spring 2010

Class 1: Overview

Class Date: March 30, 2010
Audio | Transcript
Powerpoint: NA
Paper: NA

Speaker: Andreas Weigend

Andreas' May 2009 HBS blog entry on the Social Data Revolution.

Three Revolutions
  • The Industrial Revolution - The transport of energy enabled manufacturing that was independent from the location of coal or water sources.
  • The Information Revolution - The transport of bits enabled knowledge creation that was independent from the location of knowledge production.
  • The Social Revolution - The ease of the creation and sharing of information has enabled a society that shares information.

This is the first time this class is being taught as MS&E 237, and it will be more project focused than the previous class, Stats 252. The goal of this class is to produce 10-12 groups with startup-quality ideas in the area of shared, social data. Social networking startup veterans from Foursquare, Bitly, Rapleaf, will help us accomplish this by delivering guest talks. We'll be thinking not just about social technologies, but also how society has changed by sharing more data.

Social capital serves a similar purpose with financial capital. Can both be measured in the same currency?

Focuses of past decades:
  • 1990s: Search: people no longer needed to ask other people where to find information.
  • 2000s: Social: we discover information through our friends.
  • 2010s: Mobile.

PHAME methodology:
  • P: Problem
  • H: Hypothesis
  • A: Action
  • M: Metric
  • E: Experiment

The Chase credit card

Case: co-branded credit card
They needed people to sign up for a credit card where Amazon gets a $100 for referring customers to Chase and an additional $30 to pass on to the customer.

There were two competing hypotheses:
  • Give customers $30 towards their next purchase so they will think about Amazon and become a repeat customer.
  • Give customers $30 now so they'll sign up now.

They ran an experiment where half of their customers were offered the $30 now and half were offered the money later. Almost five times more customers signed up when the money was offered immediately.

Case: Effect of scanning and indexing books
What is the effect of scanning and indexing the text of books sold on Amazon? Using pairs of books with similar price, page rank, and Amazon sales rank, they surfaced the scanned version for one and didn't for the other. There were 7% more sales for books that were scanned and indexed.

Case: Why do people come to
This survey question was presented in an ad on which allowed a user to enter free text, and they classified one million responses. When people wrote that they wanted to buy a product, only a third of them actually did. They hypothesized that poor search quality could explain for the low conversion. If you conditioned on people making a purchase, only half of those that made a purchase began with the intent to buy. Recommendations are powerful.

Metro Group's Future Store

What is identity? Is it socially constructed or engineered on a Facebook page? What are the tradeoffs of sharing personal information?

We've moved from connecting computers, to connecting pages, to connecting people. The amount of data each person creates doubles every 1.5 - 2 years.

  • 0 computers
  • 1.0 pages
  • 2.0 people

A survey on social data is accessible from our website and it will also be conducted in classes in Berlin and Singapore, so we'll be able to inspect our cultural differences.

We still have funds for more TAs so contact Andreas if you're skilled and interested.

People post interesting stuff about the class at

VCs will be reviewing the final projects and sharing their opinions.