Sam Altman - Lecture 6: Growth lyrics

This text is annotated! Click on the highlights to read what others are saying. If you'd like to add your own insights, comments, or questions to specific parts of the lecture, visit the lecture page on Genius, highlight the relevant text, and click the bu*ton that pops up. Your annotation will appear both here and on Genius. This is awesome; I've been watching the lectures on this course, and isn't it absolutely amazing, the content? And now, you're stuck with me. We'll see how that goes. Unlike Paul when he was talking in the Q&A and you guys asked him what he'd do if he was in college today and he said physics, I actually indulged myself. I went and did physics at Cambridge. I think physics is an amazing cla** to give you transferable sk**s that are really useful in other areas, but that's not why you're listening to me today; physics isn't the cla**. So I paid for college doing online marketing, directions marketing. I started with SEO in the 1990s. I created a paper airplane site, and had a monopoly in the small niche market of paper airplanes. When you want to start a startup also see how big the market could be. (In the long term, it wasn't great.) But what that taught me was how to do SEO. And back in those days it was Alta Vista, and the way to do SEO was to have white text, on a white background, five pages below the fold, and you would rank top of Alta Vista if you just said planes 20 or 30 times in that text. And that was how you won SEO in the 1990s. It was a really, really easy sk** to learn. When I went to college, being a physicist, I thought paper airplanes would make me cool. I was actually the nerdiest person in the physics cla**, so I created a co*ktail site, which was how I learned to program and that grew to be the largest co*ktail site in the UK. That really got me into SEO properly when Google launched. So with Google you had to worry about page rank and getting links back to your site, which basically at that stage meant one link from the Yahoo directory, got you to the top listing in Google if you had white text and a white background below the fold as well. When Google launched AdWords, that's really when I started to do all my marketing. That meant buying paid clicks from Google and reselling them to eBay for a small margin of like 20% using their affiliate program. That was what really kicked me into overdrive, into doing what everyone nowadays talks about as growth, growth hacking or growth marketing. In my mind it's just internet marketing using whatever channel you can to get whatever output you want, and that's how I paid for college and that's how I went from being a physicist to a Marketer - transitioning to the darkside of the force. So what do you think matters most for growth? You've had tons of lectures, and people have said it over and over, so what do you guys think matters most for growth? Audience: Great product. Schultz: What does a great product lead to? Audience: Customers. Schultz: And what do you need those customers to do? Audience: Spread the word. Schultz: Yes that's it, retention. Retention is the single most important thing for growth. Now we have an awesome growth team at Facebook and I'm super proud to work in it, but the truth of the matter is, we have a fantastic product. Getting to work on growth of Facebook is a ma**ive privilege because we are promoting something that everyone in the world really wants to use, which is absolutely incredible. If we can get people on, and get them ramped up, they stick on Facebook. So many times, I got to advise multiple startups. My favorite was working with Airbnb, but I've worked with Coursera, I've worked with other ones that haven't done as well as those guys. But the one thing that's true, over and over again is, if you look at this curve, ‘percent monthly active' versus ‘number of days from acquisition', if you end up with a retention curve that is asymptotic to a line parallel to the X-axis, you have a viable business and you have product market fit for some subset of market. But most of the companies that you see fly up, we've talked about packing and virality and all of this stuff, their retention curve slopes down toward the axis, and in the end, intercepts the X-axis. Now when I show this job to people, they say that's all well and good, you had a million people a day in terms of growth, when you started the growth team at Facebook, or ‘you were at 50 million users, you had a lot of people join your site so you had a ton of data to do this.' We used the same methodology for our B2B growth , getting people to sign-up for services advertisements, we used this to understand how much growth we were going to have in that market as well. And at that point when I joined Facebook, the product was three days old. And within 90 days of the product launching, we were able to use this technique to figure out what the one year value of an advertiser was, and we predicted it for the first year to 97%. So I think it's very important to look at your retention curve. If you see here, this red line is the ‘number of users' who have been on your product for a certain number of days. So a bunch of people, will have been on the product at least one day, but if your product has been around for a year, you'll have zero users who have been on it for 366 days. Make sense? So what you then do is look for all of your users who have been on your product one day. What percentage of them are monthly active? 100% for the first 30 days obviously, because monthly active, they also end up on one day. But then you look at 31. Every single user on their 31st day after registration, what percentage of them are monthly active? Thirty-second day, thirty-third day, thirty-fourth day. And that allows you, with only 10,000 customers, to get a real idea of what this curve is going to look like for your product. And you're going to be able to tell, is it asymptotic? It'll get noisy towards the right side, like I' m not using real data, but you'll be able to get a handle on, whether this curve flattens out or does it not. If it doesn't flatten out, don't go into growth tactics, don't do virality, don't hire a growth hacker. Focus on getting product market fit, because in the end, as Sam said in the beginning of this course: idea, product, team, execution. If you don't have a great product, there's no point in executing more on growing it because it won't grow. Number one problem I've seen, inside Facebook for new products, number one problem I've seen for startups, is they don't actually have product market fit, when they think they do. So the next question that people ask over and over again is, what does good retention look like? Sure! I can have 5% retention, but I'm guessing Facebook had better than that. That's not going to be a successful business. I get really pissed off when people ask me that question, because I think you can figure it out. I love this story; this is like my one gratuitous story (link on powerpoint) that I'm throwing out here, so the rest of it may not be as gratuitous. But this is a picture that was published in Life Magazine in 1950 of one of the Trinity nuclear bomb tests. There's a guy named Jeffrey Taylor. He was a British Physicist who ended up winning the Nobel Prize. He was able to figure out, from looking at this picture (picture on powerpoint) what the power of the U.S. atomic bomb was, and Russians were publishing similar pictures, just using dimensional reasoning. Dimensional reasoning was one of the best sk**s I learned during my time studying physics back in the UK. What dimensional reasoning is, you look at the dimensions that are involved in a problem, so you want to figure out energy, newtons, meters, newtons as a kilogram, meters seconds to minus two. You want to figure out kilograms, meters squared, seconds to minus two, and then you try to figure out how you can get each of those numbers from what data you have. The ma** is the volume of this sphere, so that's a meter cubed, so you've got meters to five over seconds to minus two and he was able to use that to figure out what the power of this atomic bomb was and what the ratios of the power between the Russian and the U.S. atomic bomb was, and essentially reveal one of the top secrets that existed in the world at that time. That's a hard problem. Figuring out what Facebook's retention rate is, is not a hard problem. How many people are there on the internet? 2.4 billion, 2.3 billion. Okay, Facebook is banned in China, so what now? Audience: 2 billion. Schultz: So 2 billion people on the internet. Facebook said around 1.3 billion users in terms of active users. You can divide those numbers by each other. And yet that won't give you the right answer. Of course not! But it's going to give you close enough to a ballpark answer of what the retention rate looks like for Facebook. If we signed everyone on the internet up, then you will know it's higher than that. Similarly, if you look at WhatsApp. They've announced 600 million active users. How many people have Smart Phones? You can figure out that number - that number is out knocking around. It can give you an idea of how many users there are. Amazon has a had a pop at signing up almost everyone in the United States. You know how many people are online in the United States, and you have a good idea of how many customers Amazon has from the numbers they throw out. Different verticals need different terminal retention rates for them to have successful businesses. If you're on ecommerce and you're retaining on a monthly active basis, like 20 to 30% of your users, you're going to do very well. If you're on social media, and the first batch of people signing up to your product are not like, 80% retained, you're not going to have a ma**ive social media site. So it really depends on the vertical you're in, what the retention rates are. What you need to do is have the tools to think, ‘who out there is comparable' and how you can look at it and say, ‘am I anywhere close to what real success looks like in this vertical?' Retention is the single most important thing for growth and retention comes from having a great idea and a great product to back up that idea, and great product market fit. The way we look at, whether a product has great retention or not, is whether or not the users who install it, actually stay on it long-term, when you normalize on a cohort basis, and I think that's a really good methodology for looking at your product and say ‘okay the first 100, the first 1,000, the first 10,000 people I get on this, will they be retained in the long-run? So now, how do you attack operating for growth? Let's say you have awesome product market fit. You've built an ecommerce site, and you have 60% of people coming back every single month, and making a purchase from you, which would be absolutely fantastic. How do you then take that, and say, ‘now it's time to scale.' (Now it's time to execute was the last thing on your forum right? *to moderator*.) That's where I think growth teams come in. My contrarian viewpoint is, if you're a startup, you shouldn't have a growth team. Startups should not have growth teams. The whole company should be the growth team. The CEO should be the head of growth. You need someone to set a North star for you about where the company wants to go, and that person needs to be the person leading the company, from my opinion, that's what I've seen. Mark is a fantastic example of that. Back when Facebook started, a lot of people were putting out their registered user numbers. Right? You'd see you registered user numbers for MySpace, you'd see a registered user numbers for ___11:38, you'd see registered user numbers. Mark put out monthly active users, as the number both internally he held everyone to, and said we need everyone on Facebook, but that means everyone active on Facebook, not everyone signed up on Facebook, so monthly active people was the number internally, and it was also the number he published externally. It was the number he made the whole world hold Facebook to, as a number that we cared about. If you look at what Jan has done with WhatsApp I think that's another great example. He always published sends numbers. If you're a messaging application, sends is probably the single most important number. If people use you once a day, maybe that's great, but you're not really their primary messaging mechanism, so Jan published the sends number. Inside Airbnb, they talk about ‘nights booked' and also published that in all of the infographics you see in side TechCrunch. They always benchmark themselves against how many nights booked they have compared to the largest hotel chains in the world. They have at each of these companies, a different north star. The north star doesn't have to be the number of active users for every different vertical. For eBay, it was gross merchandise volume. How much stuff did people actually buy through eBay? Everyone externally tends to judge eBay based on revenue. Actually, Benedict Evans has done this amazing breakdown of Amazon's business, which is really interesting to look at their marketplace business versus their direct business. So eBay is all marketplace business, right? So eBay's being judged by its revenue, when it actually has 10 times Gross Merchandise Volume going through the site. That was the number that eBay looked at when I was working there. Every different company when it thinks about growth, needs a different North star; however, when you are operating for growth it is critical that you have that North star, and you define as a leader. The reason this matters is, the second you have more than one person working on something, you cannot control what everyone else is doing. I promise you, having now hit 100 people I'm managing, I have no control. It's all influence. It's like I tell one person to do one thing, but the other 99 are going to do whatever they want. And the thing is, it's not clear to everybody what the most important thing is for a company. It would be very easy for people inside eBay to say, ‘you know what? we should focus on revenue,' or ‘we should focus on the number of people buying from us' or ‘we should focus on how many people list items on eBay.' And Pierre, and Meg, and John, those guys as various leaders, have always said ‘no, its the amount of Gross Merchandise Volume that goes through our site, the percentage of e-commerce that goes through our site, that is what really matters for this company. This means that when someone is having a conversation and you're not in the room, or when they're sitting in front of their computer screens, and thinking about how they built this particular project or this particular feature, in their head it's going to be clear to them that it's not about revenue, it's about Gross Merchandise Volume, or it's not about getting more registrations, registrations don't matter, unless they become long-term active users. A great example of this was when I was at eBay in 2004, we changed they way we paid our affiliates for new users. Affiliate programs are a bit out of fashion these days, but the idea of an affiliate program is essentially, you pay anyone on the internet a referral for sending traffic to your site, but it's mostly about getting access to big marketers who do it on their own. We were paying for confirmed registered users, so all of our affiliates were lined up on getting confirmed registered users to the eBay site. We changed our payment model to pay for activated confirmed registered users. So you had to confirm your account and then bid on an item, or buy or list an item, to become someone that we paid for. Overnight when we made that change, we lost something like 20% of confirmed registered users that were being driven by the affiliates. But the ACRUs (15:45) only dropped by about 5%. The ratio between CRU to ACRU went up, and then, the growth of ACRUs ma**ively accelerated. The cause of this is, if you want to drive CRU, if someone searches for a trampoline, you land them on the registration page because they link you have to register and confirm before they get their trampoline. If you want to drive ACRUs, you land them on the search results page, within eBay for trampolines, so they can see the thing they want to buy, get excited, and then register when they want to buy it. And if you drive just CRUs, people don't have an amazing magic moment on eBay, when they visit the site. And that's the next most important thing to think about: How do you drive to the magic moment that gets people hooked on your service. In the lecture notes for this course, I've stuck in a bunch of links to people I think are brilliant at this stuff. For example, regarding the retention curve I showed you earlier, there's a link to this guy Danny Ferante who is incredible talking about retention curves. The magic moment there are two videos linked: one is Chamath talking about growth, who is the guy who set up the growth team at Facebook, and the other, is my friend Naomi and I talking at f8 four years about how we were thinking about growth back then. In both of those videos, we talk about the magic moment. What do you think the magic moment is for when you're signing up to Facebook? Audience: See your friends. Schultz: See your friends. Simple as that. I've talked to so many companies, and they try to get incredibly complicated about what they're doing, but it is just as simple as when you see the first picture of one of your friends on Facebook, you go ‘Oh my God, this is what this site is about!' Zuckerberg talked at Y Combinator about getting people to 10 friends in 14 days; that is why we focus on this metric. The number one most important thing in a social media site is connecting to your friends, because without that, you have a completely empty newsfeed, and clearly you're not going to come back; you'll never get any notifications, and you'll never get any friends telling you about things they are missing on the site. So for Facebook the magic moment, is that moment when you see your friend's face, and everything we do on growth, if you look at the Linkedin registration flow, you look at the Twitter registration flow, or you look at what WhatsApp does when you sign up, the number one thing all these services look to do, is show you the people you want to follow, connect to, send messages to, as quickly as possible, because in this vertical, this is what matters. When you think about Airbnb or eBay, it's about finding that unique item, that PEZ dispenser or broken laser pointer, that you really really cared about and want to get ahold of. Like when you see that collectible that you are missing, that is the real magic moment on eBay. When you look on Airbnb and you find that first listing, that cool house you can stay in, and when you go through the door, that's a magic moment. Similarly on the other side, when you're listing your house, that first time you get paid, is your magic moment or when you list an item on eBay, the first time you get paid, is your magic moment. You should ask Brian what he thinks, because they've done these amazing story boards which I think has been shared, about the journey through a user's life on Airbnb and how exciting it is. He'll be talking in around three lectures time; he's awesome about talking about the magic moment, and getting users to feel the love, joy, and all this stuff. Think about what the magic moment is for your product, and get people connected to it as fast as possible, because then you can move up where that blue line has asymptotic, and you can go from 60% retention to 70% retention easily if you can connect people with what makes them stick on your site. The second thing to think about, that everyone in the Valley gets wrong is, we optimize when we think about growth for ourselves. My favorite example is notifications. Again, I've talked to and advised many different companies; every single company when they talk about notifications goes ‘Oh, I'm getting too many notifications, I think that's what we have to optimize for on notifications.' Okay, are your power users leaving your site because they're getting too many notifications? No. Then why would you optimize that? They're probably grown-ups and they can use filters. What you need to focus on is the marginal user. The one person who doesn't get a notification in a given day, month, or year. Building an awesome product is all about think about the power user, right? Building an incredible product is definitely optimizing it for the people who use your product the most, but when it comes to driving growth, people who are already using your product are not the ones you have to worry about. So in this Danny Ferante video there's also talk about our growth accounting framework that we use to think about for growth. We looked at new users, resurrected users (people who weren't on Facebook for 30 days and came back) and churned users. The resurrected and churned numbers for pretty much every product I've ever seen dominate the new user account once you reach a sensible point of growth a few years in. And all those users who are churning and resurrecting, had low friend counts, and didn't find their friends so weren't connected to the great stuff that was going on on Facebook. So the number one thing we needed to focus on, was getting them to those 10 friends, or whatever number of friends they needed. So think about the user on the margin; don't think about where yourself (21:08), when you're thinking about growth. So for operating for growth, what you really need to think about, is what is the North star of your company: What is that one metric, where if everyone in your company is thinking about it and driving their product towards that metric and their actions towards moving that metric up, you know in the long-run your company will be successful. By the way, they're all probably all correlated to each other, so it's probably fine to pick almost any metric, whichever one you feel the best about, that aligns with your mission and your values - probably go for that one. But realistically, daily active users fairly correlate to monthly active users; we could have gone with either one. Amount of content shared, also correlates with how many users there are, because guess what? You add a user, they share content. So a lot of things end up being correlated. Pick the one that fits with you and know that you're going to stick with for a long time. Just have a North Star, and know the magic moment that you know when a user experiences that, they will deliver on that metric for you on the North Star, and then think about the marginal user, don't think about yourself. Those are, I think, the most important points when operating for growth. Everything has to come from the top. So the last area is tactics. So let's say you've found your niche market that you're going to have a monopoly on inside the mousetrap market. It's a silenced mousetrap fir sitting under beds, so if that the mice come to your bed overnight, they can be k**ed without waking you up. That's your niche market. Your mousetrap is better than anybody else for that market. What typically happens in Silicon Valley is, everyone thinks marketers are useless. I thought marketers were useless when I was a Physics student, so I'm sure that you guys as Engineering students you must think that we're awful people who aren't useful to have around. ‘Build it and they will come.' That is something that is very much the mantra in the Valley, and I don't believe it's true; I believe you actually have to work. There's a good article in the lecture page from interviewing Ben Silverman. We talked about how the growth of Pinterest was driven by marketing. I'm biased of course. The first tactic I want to talk about is internationalization. Facebook internationalized too late. Sheryl said it broadly in public and I definitely agree with that. One of the biggest barriers to our long-term growth, and one of the biggest things we had to deal with, was all the countries where there were clones. Famously ? (23:55) had Fakebook.css in their HTML, and there were a ton of sites like that out there, whether it was ?, a clear clone, Mixie, Cyworld, Orkut; they were all these different social networks around the world that grew up when Facebook was focused around the U.S. Internationalizing was an important barrier we needed to knock down, and knocking down barriers is often important to think about for growth. Facebook started out as college-only, so every college that it was launched in was knocking down a barrier. When Facebook expanded beyond colleges to high schools, I wasn't at the company, but that was a company-shaking moment where people questioned whether or not Facebook would survive,if the culture of the site could survive. Then after, expanding from high schools to everyone - that was just before I joined - it was a shocking moment; that's what spurred the growth up to 50 million, and then we hit a brick wall. When we hit that brick wall, that was when a lot of existential questions were being asked inside Facebook whether any social network could ever get to more than 100 million users. It sounds stupid now, but at that time, no one had ever achieved it. Everyone had tapped out between 50 and 100 million users, and we were worried that it wasn't possible. That was the point at which the growth team got set up; Chamath brought a bunch of us together. He said very publicly he wanted to fire me on multiple occasions. Without Chamath, I think none of us would have stayed at the company; we were a really weird bunch of people - but it worked out. The two things we did, I think that really drove growth initially was, 1) We focused on that 10 friends in 14 days 2) Getting users to the magic moment. That was something that Zuck drove because we were all stuck in an*lysis paralysis and, ‘Is it causation? Is it correlation?' Zuck would say ‘You really think that if no one gets a friend, that they'll be active on Facebook? Are you crazy?' The second thing was internationalization - knocking down another barrier. When we launched it, I think there were two things we did really well: 1) Even though we were late (and stressed about being late) we took the time to build it in a scalable way; we moved slow to move fast. You can actually view the full story from Naomi on one of the video links from the lecture page. What we did was draw all the strings on the site in FBT, which is our translation extraction script and then, we created the community translation platform, so we didn't have just professional translators translating the site, but we could have all our users translating the site. We got French translated in 12 hours. We managed to get, to this day, 104 languages translated by Facebook for Facebook, 70 of those are translated by the community. We took the time to build something, that would enable us to scale. The other thing is that we prioritized the right languages. Back then, the four main languages were French, Italian, German, and Spanish (and Chinese, but we are blocked in China). Now look at that list - that's today's distribution of languages. Italian isn't on the list anymore; French and German are about to fall off. In the last year we quadrupled the number of people on Facebook in Hindi. Building for what the world is today is an easy mistake to make, and it's a lot of what the other social networks did. We built a scalable translation infrastructure that actually enabled us to attack all of the languages, so we could be ready for where the future is going to be. You'll probably be able to see some of our Internet.org summit in India about where we want to go with language translations. These are the tactics I want to go through now: Virality, SEO, ESPN, SEM, Affiliates/referral programs. I think there are two ways to look at virality. There's a great book by Adam L. Penenberg called the Viral Loop that goes through a bunch of case studies of companies that have grown through viral marketing. I strongly encourage you to read this book if you're interested in viral marketing, as well as advertising. I think Ogilvy on Advertising is great as well because in the chapter 7 you can't think of anything else stick a car to billboard with super glue and people will buy the super glue. He has some really great creative tips. So virality. Sean Parker has this really great model that he told us about when I joined Facebook, which is to think about virality about a product, in terms of three things. First, is payload - so how many people can you hit with any given viral blast. Second, is conversion rate, and third is frequency. This gives you a fundamental idea of how viral a product is. Hotmail is the canonical example of brilliant viral marketing. Back when Hotmail launched, there were a bunch of mail companies that had been funded and were throwing huge amounts of money at traditional advertising. Back in that time, people couldn't get free email clients; they had to be tied to their ISP. Hotmail and a couple other companies launched, and their clients were available wherever you went. You could log-in via library internet or school internet, and be able to get access to that. It was a really big value proposition for anyone who wanted to access it. Most of the companies went out there and did big TV campaigns, billboard campaigns, or newspaper campaigns; however, the Hotmail team didn't have much funding as they did, so they had to scramble around to figure out how to do it. What they did was add that little link at the bottom of every email that said, ‘Sent from Hotmail. Get your free email here.' The interesting thing was that it meant that the payload was low: You email one person at a time, you're not necessarily going to have a big payload. Maybe you send around one of those spam emails, but I'm not sure I'd click on your link. The frequency is high though, because you're emailing the same people over and over, which means you're going to hit those people once, twice, three times a day and really bring up the impressions. The conversion rate was also really high because people didn't like being tied to their ISP email. So Hotmail ended up being extremely viral because it had high frequency and high conversion rates. Another example is Paypal. Paypal is interesting because there are two sides to it, the buyer and the seller side. The other thing that is interesting is that its mechanism for viral growth is eBay. So you can use a lot of things for virality that may not look necessarily obviously viral. If you said to a seller that you were going to send them money - I can't think of a higher conversion rate. Frequency was low, and payload was low. But Paypal did this thing where they gave away money when you got your friends to sign up, and that's how they went viral on the consumer side. They didn't have to do that for sellers, because if I said ‘I am going to send you money via this,' you will take that. And even on the consumer side they went viral because if someone says ‘Sign up for this thing and you'll get ten bucks.' Why wouldn't you? So they were able to go viral because their conversion rate was high on the buyer and the seller side, not because their payload and frequency was high. Make sense? This is a really good way to look at virality if you want to say, ‘Is this product viral?' Facebook was not viral via email sharing or anything like that. Facebook was purely viral via word of mouth. The interesting thing about Paypal and Hotmail, is to use them, the first person has to send an email to a person who wasn't on the service. With Facebook, there is no native way to contact people who aren't on the service. Everyone thinks that Facebook is a viral marketing success, but that's actually not how it grew. It was word of mouth virality because it was an awesome product you wanted to tell your friends about. Q: In the first round, it makes sense for there to be a low payload. Will the payload increase in later rounds as the campaign grows and people send more and more e-mails? A: First and foremost, I think you only send emails to a small number of people. So compared to the ma**ive viral engines that exist today, where you import someone's entire contact book and send them all an e-mail, or where you post to everyone's friends on Facebook, the actual payloads are still very small even if it's everyone that you e-mail on a frequent basis you hit. I'm also thinking per email sent out, how many people are on it. But it's a fair point that as more people get on Hotmail, they'll send more emails, and as more people use email, the product grows more and more successfully. Q: Does a point of conversion matter as well? A: On Hotmail you click to sign up, but on a billboard you have to remember the URL, go to the website, type it in, find the registration bu*ton, click register and sign up. Anything you can do to move friction out of the flow, do it. Going from a billboard ad to an online ad removes huge amounts of friction from the flow. Q: Are frequency and conversation rate related? A: Absolutely. If you hit someone with the same email over and over again, or the same banner ad, the same rules apply to every channel. The more times you hit someone with the same Facebook ad, the less they'll click. That's why we have to, like creative exhaustion, rotate creatives on Facebook. Same with banner ads and news feed stories. The fiftieth time you see that IQ story on your news feed, you are not going to want to click on it. The same is true with these emails. So if you send the same email to people over and over again with an invite, you will get a lower conversion rate. ‘The more you hit someone with the same message, the less they convert' is fundamental across every online marketing channel. Second way to look at virality, which I think is awesome, is by this guy Ed. Ed runs the growth team at Uber now; he was at the growth team at Facebook. He was a Stanford MBA student, and did a cla** similar to this where they talked about virality and built viral products. The interesting thing is, if you look at Uber, they're incredibly focused on drivers. It's a two-sided market place, so they need drivers. It's a huge part of their focus as a team, even though they've got probably the best viral guy in the world at the company. So with virality, you get someone to contact import (35:12) let's say. Then the question is, how many of those people do you get to send imports? Then, to how many people? Then, how many click? How many sign up? And then how many of those import. So essentially you want people to sign up to your site to import their contacts. You want to then get them to send an invite to all of those contacts - ideally all of those contacts, not just some of them. Then you want a percentage of those to click and sign up. If you multiply all the percentages/numbers in every point in between the steps, this is essentially how you get to the point of ‘What is the K factor?' For example, let's says 100 people get an invite per person who imports, then of those, 10% click, and 50% sign up, and of those only 10 to 20% import, you're going to be at 0.5 - 1.0 K factor, and you're not going to be viral. A lot of things like Viddy were very good at pumping up stories. They got the factor over 1, which is perfectly doable. But if you've got something that doesn't have high retention on the backend, it doesn't really matter. You should look at your invite flow and say ‘okay, what is my equivalent to import, how many people per import are invites sent to, how many of those receive clicks, how many of those convert to my site, how many of those then import,' in order to get an idea of you K factor. The real important thing is still to think about retention, not so much virality, and only do this after you have a large number of people retained on your product per person who signs up. A couple more things we are going to touch on: SEO, emails, SMS, and push notifications. In SEO, there are three things you need to think about. First one is keyword research. People do this badly all the time. So I launched this co*ktail site I told you about, I spend a year optimizing it to rank for the word co*ktail making, but it turns out in the UK, no one searches for co*ktail making- about 500 a month; I dominated that search, it was awesome! 400 visitors a month, it was amazing. Everyone searches for co*ktail recipes, and in the U.S., everyone searches for drink recipes. So I optimized for the wrong word. You have to do your research first about what you're going to go after. Research consists of, what do people search for that's related to your site, how many people search for it, how many other people are ranking for it, and how valuable is it for you? Supply, demand, and value. So, do your keyword research to figure out which keyword you want to rank for. There are many great tools out there. Honestly the best one is still Google AdWords keyword planner tool. Once you've done that, the next most important thing is links. Page ranks is essentially how all SEO is driven, and Google is based on authority. Now there's a lot of other things in Google's algorithm now, like, do people search for your website, there's a lot of stuff about what the distribution of what the anchor text is that's sent to your site, so that if you abuse it or spam it, they can pop out with spam. White text on a white background five pages below the fold doesn't work anymore. But the single most important thing is to get valuable links from high authority websites for you to rank in Google. Then you need to distribute that love inside your site by internally linking effectively. We launched SEO in September 2007; I joined Facebook November 2007. When we launched it, but we were getting no traffic from the pages we had launched, public user profiles. So when I went in and looked at it, the only way you could get into any public user profile was to click on the foot of the page for the about link, then click on the blog articles, then click on one of the authors, and then spider out through their friends to get all their friends. Turns out that Google was like, ‘They bury these pages, they're not very valuable. I'm not going to rank them.' We made one change: We added a directory so that Google could quickly get to every page on the site, and we 100Xed SEO traffic. Very simple change drove a lot of upside by distributing the link love internally. The last thing is that there's a whole bunch of table stakes stuff for XML sitemaps, and making sure you have the right headers; it's all covered really well online for you. Email is dead for people under 25 in my opinion. Young people don't use email. They use WhatsApp, SMS, SnapChat, Facebook; they don't use email. If you're targeting an older audience, email is still pretty successful. Email still works for distribution, but realistically, email is not great for teenagers - even people at universities. You know how much you use instant messaging apps, and how little you use emails. And you guys are probably on the high scale for email because you're in Silicon Valley. That being said, on email the things to think about: Email, SMS, and Push Notifications all behave the same way. They all have questions of deliverability, so to finish to finish first, first you have to finish. Your email has to get to someone's inbox. So if you send a lot of spam, end up with dirty IPs, or send email from shared servers where other people are sending spam from, you are going to end up being put in the spam folder consistently and your email will fail completely. You may end up being blocked and have your email bounce. There's a lot of stuff around email where you have to look when you receive feedback from the servers you are sending emails to, 500 series errors versus 400 series errors; you have to be respectful how those are handled. If someone gives you a hard bounce, retry once or twice and then stop trying because if you are someone who abuses people's inboxes, the email companies spam folder you, and it's very hard to get out. If you get caught in a spam house link, or anything like that, it's very hard to get out. It's really important with email that you are a high cla** citizen, and that you do good work with email because you want to have deliverability for the long run. That counts for push notifications and SMS, too. With SMS, you can go buy SMS traffic via grey routes with people who are having phones strung up attached to a computer and pumping out SMSs. That works for a time, but it always gets shut down. I've seen so many companies make these mistakes where they think they're going to grow by using these kinds of tactics. If you can't get your email, SMS, or Push Notification delivered, you will never get any success from these. You actually spam your power users and give them notifications they don't care about, making it really hard for them to opt out. Well, they start blocking you, and you can never push them once they've opted out of your Push Notifications. And it's very hard to prompt them to turn them on once they've turned them off. So number one thing to think about regarding email, SMS, and Push Notifications is, you have to get them delivered. Beyond that, it's a question of open rate, click rate. So what is the compelling subject line you can put there so the people can open your email, and how can you get them to click when they visit? Everyone focuses towards doing marketing emails that are just spam in my opinion. Newsletters are stupid. Don't do newsletters because you'll send the same newsletter to everyone on your site. Someone who signed up to your site yesterday versus someone who's been using your product for three years - do they need the same message? No. The most effective email you can do is notifications. So what are you sending? What should you be notifying people of? This is a great place where we're in the wrong mindset. As a Facebook user, I don't want Facebook to email me about every ‘like' I receive, because I receive a lot of them since I have a lot of Facebook friends. But as a new Facebook user, that first ‘like' you receive is a magic moment. Turning on notifications throughout all of our channels, increased on our emails, SMS, and Push Notifications, but we only turned it on for low-engaged users who weren't coming back to the site, so it wouldn't be spamming for them. So it was a great experience to think about that. The first thing you need to think about when sending emails, SMS, and Push Notifications is what notifications should we be sending. The second thing you need to be thinking about is how can you create great triggered marketing campaigns. When someone created their first cross-border trade transaction was one of the best email campaigns I was ever part of at eBay in terms of click through rate. It was awesome because it was really timely, and really in context - the right thing to do for the user. I'd say make sure you have deliverability. Focus on notifications and triggered based emails, SMS, and Push Notifications. There's one thing I wanted to finish with, which is my favorite quote by General Patton. It's so cliche; it's crazy, but it's awesome. “A good plan, violently executed today, is better than a perfect plan tomorrow.” And one other thing that Chamath instills in us and Mark still instills across the whole of Facebook is move fast and don't be afraid to break stuff. If you can run more experiments than the next guy, if you can be hungry for growth, if you can fight and die for every extra user and you stay up late at night to get those extra users, to run those experiments, to get the data, and do it over and over and over again, you will grow faster. Mark has said he thinks we won because we wanted it more, and I really believe that. We just worked really hard. It's not like we're crazy smart, or we've all done these crazy things before. We just worked really really hard, and we executed fast. I strongly encourage you to do that. Growth is optional.