************************************************
The following talk transcript was recorded at QConSF 2017. The text
And its copyright remain the sole property of the speaker.
****************************************************************************************


>> Welcome to day three of QCon San Francisco.  Please welcome back the chair of QCon San Francisco, Wes Reisz.
(Applause).  
>> What did you think of day two? It was awesome.  It was one of the best days I have ever seen at QCon, absolutely incredible.
All right.  So, just like yesterday, these are the top five.  This is Monday and Tuesday, so the top is attended and the bottom is rated.  Number one attended talk yesterday was Randy Shoup, managing data in microservices.  And also, if you didn't see his last talk, that was phenomenal, watch it when it comes on video, it was absolutely phenomenal.  The best-rated talk, any ideas? The best talk?
Donkey cars was pretty good.  I heard good things about that, it was number four.
Total silence.
That's awkward.
(Laughter).
  All right, number one was SQL over streams, ask the experts, 100 percent green.  Pretty good.  Give them a hand. .
  (Applause).
All right, some of the other numbers, this is probably hard to see.  But I put it up here anyway, knowing full well it was hard to see.  These are the tracks and the ratings from yesterday.  Yesterday was 91 percent green.  That's awesome.  The highest I have ever seen is 80 percent, so when I said it was awesome yesterday, I meant it.  That is awesome.  Give all of these people a hand, too.
(Applause).
And, by the way, I'm going to apologize in advance for the number of times I will make you clap today, there's a lot of people to put up here.  Just soyou know.
  What was the highest-rated track? Microservices was the highest attended.  The hardware track was 95 percent green.  Where is Marty at? Where did you go? Awesome.
(Applause).  And overall, 86 percent green so far across the conference.  It felt like it was literally two minutes ago that Floyd was talking about software changing the world, and our purpose in helping teams adopt new technologies and practices.  And the last two days, more than five minutes, the, sorry, I lost my screen for a second, people have been asking me a lot of questions.  What I do on the last day is put up the questions so everybody can hear what people ask me.
So the first question that I got was, what is this software you are using for the transcription?
(Laughter).
  I would like to introduce to the software package, her name is Lindsay.
The Captioner: =).
  Hey, everybody!
>> Lindsay is taking a digital bow, she is doing an amazing job.  What do you think of this, by the way? This is awesome.  One of our experiments.
  All right.  And the next thing we get, what is your call for papers?
>> (Laughter). 
>> And so, every single talk at QCon is picked by a person.  Every single one.
And so we do have an area that you can submit talks, but it is not a call for papers.  And that submit for talks, I will look at it, I will talk to people, it might not be for this conference, I might make a relationship with you.  But every talk is picked by a track host, they work with us to pick every single one of the talks.  There is no call for papers.
  So can I submit a talk? Absolutely.  So the way you do that, go to the bottom navigation, on all of the sites, there's a talks submissions, it is a Google form.  Pop in a talk there, I look at them every week and I will make connections and relationships and we will go from there.  And maybe from this conference, the next conference, whatever it may be, but that is how you can get into the mix so the community can start to see about it.
  What do you do to connect the  -- select the speakers? There are three things that we look for at QCon.  One is practitioners, we believe that you want to hear from the maintainers, the commiters, the people that are pushing the code.  We want to hear from the people that are making decisions and living with those decisions.
And so that's why we focus on practitioners.  It doesn't mean that we don't love advocates and evangelists, we do, and we have them from time to time on innovator areas, but the focus is practitioner to practitioners, the committee, me, we are all practitioners.
  And on the early adopter curve that Charles talked about it, we are aligned on that, that's what we are looking for in the content.
  And then actionable take-aways, we want something that you can TalkBack -- take back in all of our talks and work with your team on.
  This is what we look for.
  What can I expect in being a speaker? This process is growing and gathering more inertia.  So what we do is the speaker interviews, me or somebody that works with me will do interviews on the audience, we will go through the numbers, what types of questions people look for, your roles and the types of questions that you might ask.  Those are the types of things that we do in this interview, we post them online, we are trying to get to post all of them online, it is a challenge.  We do speaker webinars, we take experts from the valley that are focused on technical presentations, we do webinars with all of the speakers and we do rehearseles and code reviews with the presentation, maybe a track host or somebody from a rehearsal community, we will work with them on a one on one basis and give critical feedback.  And even the advanced folks love this, it is a code and peer review of a presentation. Is this -- this is what we ask speakers to do and what is required.
  What goes into the planning of QCon? This is the timeline.  So greater than 40 weeks, or years ahead, we have all of the contracts.  32 weeks, we do co-chair collection, 25 weeks, we meet and talk about what is going on in technology, we talk about speakers and tracks, and that's what becomes the conference.
20 weeks out, we select the tracks, 16 weeks out we start working with the speakers and the things that go into the talk, and then we have QCon.  And after that, we do a retro.  This is a release, we do it in two-week sprints, and then we do a retro for the release and go all the way back.  We execute like a software project.
  It seems like you have a lot of people involved.
  Yes, we do.  We will start with the committee.  Every conference has the co-chairs, this is the program committee, and so I think I said the track hosts were the engine, this is the brain of the conference, this is the think tank that makes them happen.  These are the folks that meet every week to help us define this conference.  So please, give them a round of applause.
(Applause).
  And after that, the track hosts.  You have met 12 of them, you will meet six more today.  Give them a round of applause.  I warned you.
(Applause).  176 speakers, one of the things that I'm proud about this one is that we increased by 20-something the number of speakers from last year, but reduced the speaker to attendee ratio.
And so, great speakers, give them a round of applause.
(Applause).
And volunteers and staff, these volunteers are the ones that are out there taking these votes, please give them a round of applause.
(Applause).
And keep it going, our staff.
They're the ones that this is dedicated staff, we don't hire an events company, these are the people that are behind the scenes, making this stuff happen.  So they are awesome.
  And reminders: QConSF is the hashtag, we will give a pass to somebody that blogs, transcripts are a great thing to flesh out blogs and to get you quotes.  Facebook is at QCon.  And attendee survey, we do a retrospective, and the main thing that we use is this feedback survey.  So you will get an email with the Survey Monkey, please fill it out.  We use that and change the content, if you loved what you saw, let us know.  If you didn't love what you saw, let us know, we are going to iterate on it.  We are going to constantly iterate on it.  As Floyd mentioned, we do a lot of experiments and use those to validate our experiments, whether we should continue with the text, the live transcription, those types of things will be in this things.  So please, please, fill out the survey.  I'm the worst one on it, I get it, but it helps us to give you the content that you de -- deserve.
  And yesterday's videos, there are 50-something videos that are being uploaded, I checked a couple hours, they are on the way up.  You will get an email with the access code so you can go in and get to them.
And upcoming QCons, I mentioned that every QCon are individually organized.  And we have individual committees, new track hosts, and new speakers for all of the different QCons.  They have their own flavors, each city has a different feel.  London has a finance in tech feel, each city has a different characteristic about what resonates with the crowd.
And I mentioned yesterday something called QCon AI, this is the new conference, this is the first conference, as I mentioned, that we added in the last six years, we want it to be focused on a -- AI and ML, we are not changing the folks of QCon, it is for senior software leaders,e -- engineers, and architects.  So is AI, it is for artificial intelligence in engineering but we are focused on the developer, not on the data scientist.
  This is what we are working with, the names are mocked in, don't hold us to it, we are giveing a peek at what we are looking at doing.  Three tracks across two days in different areas.  There will be something on AI, ML, data engineering, and what we are going to do, I will talk about the tracks first and then talk about what we're going to do.  These are the six tracks that we are going to run, one is two half-day tracks, and deep learning applications and practices, ML in action, sequential data like an expert, operations, I will not read, these are the tracks focused on the software developer, not necessarily on the ML engineer or data scientist.
  We know that some people are coming into this new and some are experienced in it.  What we have done is decided to break things up just a little bit to try to kind of hit different levels.  We will have session just like at QCon, 50-minute sessions, practical applications for adopting TensorFlow.  And if you think about the first AI talk you are going to, the first 10 minutes is a set up, this is what are doing in the next hour.  We have cut off the first 10 minutes and made it a separate talk.  If you are experienced with the foundational things, you can go to another 10 minute talk, because nobody knows everything in software, and then you come get introduced and come back to the deeper dive.  These are the things that we are doing there, we will introduce codelabs or hands on, we will have 60-minute labs that are self-paced.  That's what we are doing with QCon AI, there are multiple ways to learn about it, and adopt technology and understand things better.  What do you think? Sound good?
That was half-hearted, I'm a little nervous now.
(Laughter).
  Okay.  So sponsors, they are amazing, we could not do it without them.  Some of the ones I have not mentioned, SparkPost, thought works, AWS, AppScatter, all of these folks are on both levels.  I think up until lunch, somewhere around there, so make sure that you go and chat with them.  There is plenty of time.  James Truitt is on building Node.js, microservices on Azure is the most popular online with the schedule builder.
  And closing reception tonight.  So Netflix is sponsoring the closing reception tonight from 6:15 to 7:15.
And all right, the next up we are going to bring up the six track hosts.
We are going to bring up the six tracks.
We are going to bring up our six track hosts.
  (Laughter).
  You will notice on the side, there are -- I'm so engaging, that's what it is.
The stars on the side, I forgot to mention this the last couple of days, and some people noticed.  If it is starred, that's a top-10 talk on the schedule builder.  So if you see a star on it, it is labeled as the top-10 talk.  So next, I'm going to introduce Sid Anand, he is hosting our going serverless track.  Please welcome Sid.
(Applause).
>> Hi, okay, I will take you back to 2008 when an online retailer introduced a new service offering for engineering like you and I to launch, in an unlimited variety of services, in their vast data centers.  That was Amazon.  And one of the interesting features of Amazon was their pay as you go model.  And while that was pretty brain-dead simple to take the value of in services like S3 and Kinesis, it was a bit harder in the world of compute, because you had to do a lot of stuff like, you know, bundle your services and manage auto scaling groups and that sort of thing.  Fast forward to two years ago, and Amazon introduced Amazon AWS lambda, that was the advent of what we can call the next, like, generation of serverless functions, and the serverless, like, revolution.
And so, today, we're going to have a pretty interesting set of speakers that will walk you through the different aspects of serverless.
We will start with Jared Short, Jared is going to talk, he is going to dive deep into GraphQL and explain how GraphQL and serverless, especially serverless functions, are a marriage made in the clouds.  Guy Podjamy is going to entertain you with live hacking of various serverless function exploits, and while they are focused on serverless functions, it existed way before AWS lambda, because you can use S3, BigQuery, those are also serverless functions, or serverless features.
And Nate Kupp is going to talk about how he took Thumbtack from using various data services in AWS to arguably better services in GCP.  And then Ben Kehoe , who is the robotics research scientist at iRobot is going to show you how IoT use cases like iRobots are supported by the vast feature set in the cloud today.
And last, but not least, Dana Engebretson will detail her experience scaling data pipelines using a various serverless functionalities.  Thank you very much.
(Applause).  
>> Good morning, who has heard of chaos engineering before? Who heard of it this year for the first time? Last year for the first time? Who knows exactly what it is? We will help with that today.  We have passed a lot of big companies that have done with, I'm excited about willy and Sahar talk is they will talk about how to bring it into a company that has not had it before, they will talk about how Expedia brought this in technical, cultural, and other aspects.
  Do you ever think about self-driving cars? I think about them when I'm driving my motorcycle, lane-splitting in the bay all the time.  Nate will talk about why self-driving cars might work okay, and why chaos engineering might play a big role in that.
  This whole chaos engineering thing is built upon the premise that I don't like to get paged at night.
  No, but the engineers don't like to get paged, and that incidents are a problem that we want to solve, the high availability is very important.  So I think it is key to have Dave here, as someone that I have worked with in the past, someone that is a great call leader and that I looked to for help in the middle of the night when everything is broken.  He is going to teach us the wisdom that has been hard-fought over the years and how to do incidents and how to really prepare for the failures that occur.
Adrian is going to be here, he is going to talk about not just about why chaos is a technical problem but, with all things, it is a social problem.  The intersection of our systems and the people that operate them, and the processes that manage them, is very important.
And so he is going to describe that from an architecture point of view.
And lastly, you know, you can get a lot of value, as we know, from the 80-20 rule, but doing  -- by doing some simple things and getting good results.  Heather will talk about chaos engineering on a budget.  She will show you that you don't have to spend a lot of time or resources to get a lot of value  from this approach.  I'm super excited, thank you very much.
(Applause). 
>> Good morning, QCon.  So before I proceed, so who does not care about security, please raise your hand.
(Laughter).
  And okay, we have a big room to host all of you.
Today, we're going to have six speakers, five talks, and one of  -- wonderful speakers, wonderful talks.  I have worked with many of them.  Instead of going through the detailed talks, like technical terms, I will give someflavors for each of them.
So the five talks are roughly divideed.  So there will be one talk about AI and security, it there will be a talk about how to find threats in the wild, and there are three talks about using hardware techniques, hardware-related techniques to improve security.
So Dawn Song from Berkley will talk about how to use security to make a better AI system, and also how AI can help to improve security.
And then so, the next talk is going to be about Dr. Zhao, SVP of research at NS focus, and thee will talk about how to find the bad guys in the wild, where they leave their traces, and how do you attribute an attack to an attacker.  We have three talks about the hardware-related technologies, you can say that Jaimie and Cody, they are from Endgame.  Jaimie is CTO, and cody is director of research there.  They are going to talk about how to use a hardware counter to enforce control flow integrity, basically, when someone is going to hijack your program, are you going to be able to know it, are you going to be able to stop it, and they are going to introduce the new technology to do that.
  We will have two more talks about another wonderful technology, which is Intel SJX.  So if someone, say, an attacker is able to hijack your operating system, but could you imagine he is not able to know what is going on in your application? So we will have Xiaoning and he is going to talk about wonderful things, Xiaoning is going to talk about how to avoid making mistakes using SGX, and he is going to talk about their SGX, SDK written in Rust, a memory safety system that will make good use of SGX.  That is all, see you there, thank you.
(Applause).
>> Good morning.  Sometimes, we run into problems that are so weird and so frustrating that we have to do something truly terrible.  We have to talk to an academic.
(Laughter).
  In my track, five speakers who have looked into the void and come out with great and interesting truths.
I recommend you come.
We are going to lead off with Philip Wadler, one of the co-creators of Haskell and a drive of generics.  He is going to lead us off talking about category theory and how to use it to think about data structures and how we can revert between data struckers in a more simple and clean way.
  He will be followed by Isaac Potoczny-Jones, and talking about how a little bit of abstraction can make sure you have the crypto right and feel confident in it, something that my security track colleagues will enjoy hearing about.
  Nathan Sobo will follow, talking about how CRDTs help the atom team at GitHub attack the collective editing problem.
  Temesghen Kahsai will be next, talking about how you can really understand what your code is doing.
  And then we will finish with Changhoon Kim, doing fascinating work not just thinking of networks as static things, but programable entities and how you can program your networks to do things and make your systems more fast and reliable.  So that is modern CS in the real world, I encourage you to come.  The other tracks will teach you many wonderful things, modern CS in the real world will blow your mind.  Thanks.
(Applause).  
>> Good morning, I'm Kevin, hosting web as a platform about this and I'm excited about this.  How many of you consider yourselves front-end or web developers?
Not very many.  How many of you are excited about a web track in a hard core software conference? We are under going the biggest transformation we have gone since the browsers were popular.  It is the best document delivery device in the world to becoming the biggest application platform in the world.  We are no longer looking at something that just puts out documents there, but is essentially an operating system for distributed applications all over.
  So for today's track, we are going to be covering the span of technologies that are making that happen, we are going to start in the guts.
  Natasha Rooney will be talking about networking technology, HTTP and the successors that are going to make latency drop like a rock, and really improve our ability to put dynamic applications on the web.
Jay Phelps will be talking about WebAssembly, and talking about back-end developers, you should be excited about this, this allows you to develop in the tool chains you are doing now, but to create a world-wide platform as big and as powerful as the JVM and distributed by browser.
  And Michelle Colvin will talk about the payments API, imagine the payments you could have is available to every website out there, that is there for you to check out and pay.
  And Dan Wilson, talking about the animations API, and how incredible and powerful an -- animations are going to be baked in.
  And then we will close out with an incredibly exciting talk, Kieran Farr, talking about web-based VR and why the web is a more powerful delivery mechanism for VR than native devices.  Exciting day, hope to see you in the web as platform track.
(Applause).
>> Good morning, I'm Katerina Kulagina and I'm hosting the culture track today.  I hope that many of you are convinced that culture matters, it is important for growing and retaining people, and it matters as we think about what we as an industry want to create, what kind of culture we want to create to get new people into our industry.
And so we have an incredible slate of speakers today, we will start with Sarah Von Grebmer from Google, she is going to be talking about Kubernets and the open source culture they are creating around Kubernetes.
  And next, we will have Linda Elkins, who is coming from a different perspective.  She leadsathize Silicon Valley innovation center, and she will be able to tell us about material science and the lessons they have learned in their industry and what we can apply.
  And next up, we have Marty Weiner, he was the CTO of Reddit, who is going to talk about scaling and engineering cultures, starting from tiny start-ups and scaling it up, which is something that many of us deal with.
  He will be followed with Bruce Johnson, who founded FullStory and was previously an engineering director at Google, and he is going to talk about how he and his colleagues are engineering culture with memes, shared jokes and metaphors.
  And we will round it out with Valerie Aurora, who is the diversity and inclusion consult ant, she will teach us about how to be allies, all of us can be allies of people of less privileged groups and how that can create a more inclusive and diverse atmosphere and culture.
And I think this will be a great talk as well.  I encourage all of you to come.
  We will finish up with a diversity and inclusion in tech panel discussion, with a fantastic slate of participants.  Randy, Karen from Netflix, Karen Catlin, Susan and Steve Johnson, we hope that many of you will join us.  Thank you.
(Applause).
>> Give them all a round of applause.  We could not do it without them.  Fantastic.  Thank you.
And so this is how we select speakers, right there.
And all right, AMAs, there are three of them today.  So this morning's keynote will have an AMA, if you have questions, you can ask more, Guy Podjamy will be doing one on open source security after his talk.  And the second to last one from the day, there will be an engineering culture one with Bruce Johnson and Shane Hastie.  Does this sound like a good day to wrap it up?
(Applause).
All right.  And just a few days ago, I ended with this talk, don't make the days count -- (laughter) -- don't count the days, make the days count.
  (Laughter).
  We put together a little video to show what the day looked like.
[Video].
  (Laughter).
  (Applause).  
>> Thanks for letting me kick off your day every day.  So I'm going to bring up one more of the committee members to introduce the next keynote speaker, this is Marty Weiner, he is a member of the committee and has been with the committee for a while, four or five years at least.  He was one of the original engineers with Pinterest and was formally the CTO of Reddit.  
>> Thank you.  
>> (Applause).  
>> How is it going?
Um, show of hands, who has heard of read it?
(Laughter).
  One? Okay.  So I'm super excited to be here, to introduce you to my friend Nick Caldwell, I met him for the first time while I was CTO of Reddit and we were doing a VPN of search, we were doing it for nine months, and we talked to 12 or 18 good people, but nobody quite met, we had an aggressive set of criteria that we were looking for, and we could kind of take our time.  Something about the criteria, we needed somebody with a track record of building large teams quickly, and somebody who cares about and knows how to build an awesome, diverse, and inclusive culture and cares about improving the world, somebody that wants to win, and somebody that doesn't mind solving a lot of hard problems.  I don't mean tech problems, but various interesting corporate culture management problems that we have to tackle.  And most important, somebody that has better taste in clothes than I do, which is all the candidates, so whatever, fine.
  So after interviewing a bunch of people, I chatted with Nick on the phone.  He was up in Seattle, and after an hour, I turned to Steve and I said, that's the guy.  Steve is our CEO.  We went to sale mode, I told Nick I would paint his fence, mow his lawn, that offer still stands.  And he joined us as VP of Evenge, and he smoothed out everything that we do, how we plan and execute and hire, everything. And that was important, we were rapidly growing and continued to do so.  In the last eight months, Reddit engineering grow by 3X, and if it follows normal operating procedure of a Silicon Valley company, it will grow again 3X in the next 18 months.  Before Reddit, he was at Microsoft for 15 years as a general manager, he has giant teams and has to make them work really well.  Right now, he participates with dev color, a non profit that works to maximize the impact of black software engineers, and is also a founder of color code, a scholarship dedicated to future leaders of color in tech.  So help me welcome Nick to the stage.  He is going to talk about growing teams by 3X.
(Applause).
>> Okay.
Hey, folks.
Hope you are having a good time at QCon, day three.  I will see if I can switch over to four.
Does that one work? Got it.
Okay.
And we will get started.  One sec, y' all.  Don't want to lose the energy in the room, I have to get the presentation going first.  Okay, hey, I'm Nick Caldwell, thank you for joining me here at QCon, day three, and I want to do an re-introduction of myself, VP of engineering at Reddit, I was previously a general manager at Microsoft, there are some folks from there in the audience today.  I went to school at MIT, go bears.  Any bears in the audience?
There we go.  Go bears.  So I want to talk to the few of you who don't know about what Reddit is, that is going to lead into explaining the challenges that I face.  So Reddit is the front page of the internet, a social network, there are tens of thousands of communities around whatever passions or interests you might have.
And so, the way that we think about Reddit is exactly that, a collection of passionate communities.
And one of the interesting things about that is it means that Reddit has one of the highest engagement scores of any site on the web.  So here is Alexa, you know, top 10 list, we have Google, YouTube, Facebook, and Reddit.
And if you check out that column daily time on-site, 15 minutes and 50 seconds, next closest is Facebook, with 9 minutes and 15 seconds.  I want to explain why and how that works, Reddit is unique in the following ways.  It has pseudo-anonymous accounts, you don't have to sign in with your real identity, and for those that are asking about the future of Reddit, it will always be that way.  We have passionate communities with rules, so that only certain types of content can go into a community.
We have voting, and we have this thing called the hot front page, which aggregates all of that great content from across the entire internet.
When you apply scale to that, you get the following thing, just an endless stream of addictive, you know, time-sucking, content.  
(Laughter).
  We like to think of Reddit as weaponized procrastination.
(Laughter).
  So that's what I'm to for a living.
(Laughter).
  So where are we at with Reddit? It is the 4th largest website in the United States, 25 million active users, 1.1 million active communities, what is new this year? A new executive steam, Steven has come back, a new product vision, a lot of the core people at Reddit agree that the site could be more welcoming, and that was a core focus for us last year, now we have a foundation to build on it toward the future. And then rapid hiring, that's what this presentation is going to be about.  Last year, we grew Reddit from 40 engineers, we have 130, plus about 25 part-time contractors.
  And I expect that that is going to continue through next year.
And so why do I want to talk about scale, why is this presentation important to me, and why I hope you guys care about it.  A few years ago, there was a reports, a start-up genome report, they surveys about a thousand start-ups, ones that were successful and failed and tried to ask them why and figured out the formula for a successful start-up.  This was an interesting statistic, within three years, 92 percent of the start-ups fail, and of those, 43 percent of those fails with issues due to scale.  This is a really important statistic for me, I traded my cushy executive-level position at Microsoft to join Reddit, and I would love for us to be successful.  So with that in mind, this is my first week at Reddit.  Marty introduced me to, in the first week, to a 30-person stand up.  And Reddit was still using the tools and techniques of a very small team, but trying to scale up with them.  We saw that 30-person stand-up, we saw that -- Google slides was tracking our work.  We had not fully implemented jury yet, we had a lot of people running around themselves and  -- calling themselves tech leads, there was not a management structure and resistance to that as well.  And there was a slow delivery pace, there was not a lot coming out of the engineering organization.  With all of that in mind, tons of stuff to do and ship.  Nobody looks at Reddit and thinks that it is done and we need to hire about 100 people.  So with Marty's help and the rest of the exec team, we fixed the problems and I will tell you about the rest of my experience and the rest of the exec team's experience at Reddit.  So the first thing we had to fix was roles and responsibilities.  When you are in a small company, I think that your classic example for any Silicon Valley start-up is you have the technical cofounder and the product cofounder and they do everything imaginable.  So you might have the CTO on sales calls if you are at a really small company.  And as you scale up, you can attract expertise and you can specialize.  That CTO and product co-founder split off into engineering and product disciplines, design, and a whole other constellation of specialties.  This was the first thing we did at Reddit and I will give you an example of why it mattered so much.  At Reddit, we encountered a lot of different legal issues, we're a big site, a big target.  And I want to talk to you about our chief legal counselor, her name is Missy Tidwell and her first week at Reddit.  And her first meeting was the Reddit legal counsel meeting, and in this meeting was 15 to 20IC level engineers that are all very passionate about the law.  As if you could take 15 passionate people, cram them in a room, and immediately gain the equivalent of a Harvard law degree.
(Laughter).
  This is what Missy walked into.
And she immediately shut that meeting down and proceeded to own legal council at Reddit.  And this extends to other areas of your organization, not just law.  But you have to figure out a way to tease apart responsibilitys as you scale.  And a common framework is this thing called RACI, maybe you have seen it before.  You make a chart of all the people in your organization, as well as all the different responsibilities that they might have.  And RACI stands for responsible accountable consulted and informed.  So you make the chart, and then you assign people R, A, C, or I depending on their relationship to the different tasks.  And the nice thing is that you get very, very clear ownership outcomes.  Everybody knows who is responsible for what after you go through this operation.  At Reddit, we had to tease apart the relationship between product, program managers, and engineering managers.  We had a lot of people who were program managers, and if you have them, they are switching, they are doing a lot of running daily stand up and the customer interaction and so forth.  So we split that into two roles, the product managers are responsible for the who, what, and why of what the product organization is producing.  And then the engineering managers are responsible for how and when things are going to be built.  This immediately allowed us to get people focused on the things that they were best at, the program managers converting to product managers and engineering managers allowed to run the engineering organization.
And now, the second thing with engineering managers, I will go back to what I said at first, we didn't have any engineering managers at Reddit when we started, we had folks calling themselves tech leads, and tech leads is a difficult title.  I don't know if you have tech leads in your organization, I remember when I was first starting as an engineer, I asked my manager what a tech lead was, and he said that nobody knows.  It depends on your organization.  And 15 years later, people ask me, what is a tech lead? Well, I don't really know.  I do know what engineering managers are and what architects are, so I needed to come up with a formula to convert all of these folks that are calling themselves tech leads into roles with clearer functions.  I came up with the VOight-Kampff tests for engineering managers.  Do you know blade runner? You go into the machine, it tells you whether you are a human or a robot, it will tell you if you are an engineering manager or architect.  Do you care more about people or architecture, pretty straightforward.  And what are your thoughts on shipping on a deadline? If you are a manager and don't ship on time, you might have a bad time.
  So your PM goes to the direct ports and tells you to start working on a feature.  As an engineering manager you need to stay on top of that, and you spend a day on LinkedIn, fishing with recruiting, how do you feel.  This splits the tech leads into people that care more about building the team and process, and another class of people that care about the code and the architecture.
And now, the disadvantages of using this RACI technique is it is inflexible, sometimes you have people that switch hit, sometimes you have an engineer that is good at design or product intuition, and it does not address some of the deeper issues, which is trust.  And you cannot scale your organization ultimately without trust.
So I won't go into detail on this slide, but it talks about the advantages and the disadvantages of running an organization with high trust.  This comes from a book called the Speed of Trust, I encourage you to check it out.
And the long story short, you are more efficient the more you can take dependencies that you trust.  And later on, I will talk about microservices and how that idea relates to microservices.  And for now, just imagine that to fix this, we had to go to executive off sites with leadership training coaches, we did not quite do trust falls, but we did find a way to put our faith in each other.
And I think, in general, people feel okay to say that is not my job, when they can also say the person that we hired to do it is amazing.  And a quick tip for you that are scaling your organizations and you might have this problem now, when you hire people into your organization, I strongly encourage you not to put them in a position where they have to prove themselves to the organization before you give them a responsibility.
Because you just spent weeks interviewing them, and convincing them to work for your organization, I think a better way to handle on-boarding new people on your team is to trust them, give them tasks, and then verify they are delivering what they said they are going to deliver.  So trust but verify when you are bringing new people into your organization.  All right?
And so, that's how we ended up in an organization with good roles and responsibilities, but we still didn't have an org chart.  Those are pretty scary, the most terrifying part of constructing your organization is really deciding on the structure, who reports to who, what the responsibilities are, etc.
  And so I caution you that this next section might be terrifying, but it is one of the keys to success at Reddit.
And now, I want to introduce a concept called con Way's Law, he is an academic, many years ago, he wrote a paper about organizational structures and one key insight on that was, organizations that design systems are constrained to produce designs that are copies of the communication structures of these organizations.
And all that really boils down to you will ship your org chart.  I think you can see this in products, many of you, I was at red fin the other day, I was trying to test this on them.  I said, when I look at the product, there's an area for realtors, there's an area for people who are searching for real estate, and there's an area for people who are trying to analyze data.
And I bet somewhere, in your org chart, there is a group for each of the three people and they said yes, of course. And I think you will see the same pattern emerging across any company.  So Conway's Law is fairly well-trot.
And the challenge, then, is that your organization needs to continually reflect the organization's most immediate goals, and the best people for each goal should be continually moved to where they are most needed.  That's an ideal statement, a very ideal statement, but intuitively, that should make sense.
The challenge is reorgs suck, nobody likes switching -- raise your hand in this room if you like re-orgs, you want to switch managers.
(Laughter).
  Marty, (laughter).
Thanks, Marty.
So re-orgs suck, people don't want to change, they don't like change in general, and they don't like the risk and uncertainty, the feeling that the ground is moving out from under them that comes with re-orgs.  And I should tell you a story about that, Reddit's first re-org.  And Reddit, when I got there, there were 35 people and we needed to move one of our key engineers around, you can guess why.  Reddit had a great front-end engineer and we needed to get him into a position where he could work on the site redesign, and in order to make this happen, in order to move one engineer from one part of the organization to another part of the organization where he was most needed, we had to get HR involved, they called his wife.
(Laughter).
  To get permission -- (laughter).
And many of the PM teams, who had never been involved in the re-org, also submitted their opinions through back channel, just to move one engineer from one part of the organization to another.  So re-orgs can be really painful.
  And just so you know, ultimately, that engineer became one of the key leaders at Reddit.  He is now leading our front-end guild.
  I will say that re-orgs have the following advantage: A, they put your resources where they are needed and, more importantly, they allow leaders who are comfortable with change and who want most urgently to participate in the most important aspects of you business to surface.
And so there can be good and bad, but I think that for the efficiency of your organization and for surfacing new leaders, re-orgs can be quite useful.
  And now, with Reddit, it took us about five or six months, it took us a month to get to the first person moving, and about five months later, here is Reddit's first org chart.
And so, I would like to revise Conway's Law in the following way.  You are going to ship the org chart, but you have to make sure you are continually shipping the right one.  With Reddit, we ended up with groups, product groups around significant investments, like the site redesign, doing chat, there was initiative called Reddit social vertical teams to attack specific customer problems, and then shared teams that go horizontally and are used by every part of the organization.
And those would include things like your foundational infrastructure team, we have a team called anti-evil, which fights crime, I guess, across the entire site.  And then each box in the org chart has a very clear, time-bounded mission statement.  This is what we ended uparve five months, and we are pretty happy with it.
  And what is next? If you go back and read that original Conway paper, it has an interesting note at the end.  The design that occurs first is almost never the best possible, the prevailing system may need to change, therefore flexibility of the organization is important to an effective design, which I think is a -- it is subtle, but it is very important.
And I think that, in today's modern world, it is something that everybody who is responsible for designing organizational systems needs to pay attention to.  And the idea here is that you can actually reverse conway's law.  That is to say, the architecture that you are building can inform the organizational structure.
And microservices, and the strong push toward that, which a lot of -- which is a lot of what the conference is about, enables that scenario.  Microservices fundamentally change how we think of an organization.  This is a cool diagram, it is called the death star architecture.  And, on the left, you have Amazon, on the right is Netflix, this is their constellation of microservices that hold up Amazon and Netflix.  And I think if you end up with an organizational, or an architecture of this form that, in the future, mission, value, and culture will probably end up being more important than rigid organizational hierarchy.  And if you look at Amazon and Netflix, and you examine how those companies exist and how they form, both of these companies have inch-thick culture decks.  I don't know if you have seen the Netflix one, it is literally, like, 120-page slide deck.  Amazon is similar about culture values, and both of these companies aspire to have small teams.  Amazon is famous for its pizza-box sized teams, and Netflix, I don't know if you are aware of this, but they only hire seniors because they want people to act independently, and they also have small teams.
And one thing you should be aware about through the days as we talk about culture and and microservices is how they are affecting the shape of organizations is something that is fascinating to me.
Okay, we will move on.  Other things that change, as you scale, people.
First photo on the top left, that is one of my first teams at Microsoft, the Power BI team at three months old, you can see every single person on that team fits on my porch.  They fit on my porch.  And the next photo is the same team a year later, I cannot even fit everyone in the team into the frame, and unfortunately I don't know most of the people in that photo.
  So the more people that you have, the more disconnected you feel from the team.  So as you scale, everyone is going have one of these moments, the elevator moment, I would like call it.
  If you are MadMen fans, here is Don Draper in one of his reports, he is in the elevator, and the reports says, I feel bad for you.  And Don says, I don't think of you at all.
  Your experience will not be that bad, but you will eventually start bumping into people that maybe work for you that you don't recognize.
  What do you do? Don't feel bad about it, there's a concept, Dunbar's number, and there's research done about how many people you can actually know.  And it turns out the answer is 150, and that above 150 relationships, the human mind simply cannot maintain the connections.
And, you know, Reddit, we are up to 300 total people now, we are well beyond Dunbar's number.
But, I will claim that you don't need to know everyone at the party to have a good time.
So it is safe if you are responsible for organizational design to assume that sub cultures are going to form.
But how do you handle it? Well, at Reddit, and I think at most companies, you can use a technique of forming technology guilds or affinity groups, these are great ways to give people a place to call home as the company expends.  At Reddit, we have a front-end guild, back-end guilt, a guild for dungeons and dragons, just give them a place to connect to.
And even though sub cultures are going to form, you have make sure that they are not forming in a way that goes against your core company values. And that's the final point.
Make sure that your core company values are crisp and clearly understood, and that they inform all of the sub cultures that may form within your organization.
And Reddit solution, guilds, weekly all-hands, where we celebrate key elements, and then we have company megaevents, we will take the company to Tahoe to talk about the successes, failures, and our cultural values.
  And what about communication? As you scale, you still need to talk to people and disseminate information.  Unfortunately, this is an unsolved human problem.  You can try all of these different techniques, newsletters, team syncs, company meetings, interest groups, documentation, press releases, Slack channel, email, Wikis, dashboards, unfortunately none of them work.  You have to use them all.  In my lifetime, I will see drones delivering pizza, I will see -- I will see self-driving cars, I will see stem cells solve most modern ailments, but I will not see a solution to getting two groups of people 50 feet apart to speak to each other.
(Laughter).
  You know I'm right.
(Laughter).
  And so, what can you do?
You do have to try all of those things, and I will add to that, you can beat the telephone game by messaging with extreme clarity.  You have to make sure that with you message, you have something in it that applies to everyone in your organization.  You have to repeat, repeat, repeat, reinforce, and getting the leaders to understand the core of the message and repeat it out, and over communicate.  This is a photo of some posters that we hung up in the wall of my last team, because we had quality problems and we had problems with people implementing telemetry.  So anywhere you would go in the building, you saw these posters: Quality and telemetry, and they became the core values for the year.  So extreme clarity.  I will end the section by talking about engineering, and what gets harder with scale.  And this is kind of the core of what I did at Reddit to get them operating effectively.  There are three things that change.  The first is awareness, and your company is bigger, and everyone wants to know what everyone ELSE is doing, but they don't necessarily want to say what they are doing.
(Laughter).
  Right?
And so coordination, all right, you have more teams with more dependencies, and not only do you need to can know what everyone else is doing, but you need to align ship schedules.  And bottlenecks can hit you hard, if you have tons of dependencies in your organization and you are getting network dependencies, a failure in one part of the organization can knock out the schedule for multiple other teams.
  And this is what gets harder.  I want to show you, and let me, sorry, one second.
I think my clicker is -- why does it get harder? I want to, I think I mentioned this before.  I went to Berkley so I could get an MBA, so I wanted to pull out this $100,000 slide.
(Laughter).
  And so, my MBA enabled me to make this efficiency curves.
  (Laughter).
  So one thing, you can add more people into an organization and things will get more efficient, it turns out to not be true.  As you add more people into an organization, the efficiency goes down, that is because of communication overhead, or breaks in your process.
  And coordination costs, scope increase, eventually kill your efficiency and you need a process change or re-org in order to overcome this curve.
  And so, with that, I want to show you where we started at Reddit, this is Reddit's first project tracker.  It is a Google spreadsheet, excuse, me a Google slide.
And you can see that we have copy and pasted Jura links into the slide carefully, some of the items in the slide have green or yellow check marks, also hand edited, and we also have importantly Aqua Man and Bat Girl.  We spent more time talking about the images than the actual information on the slide.
  (Laughter).
  And there were 12 of these slides that we had to go through during our stand ups.  This is where we started.
  And now, I needed to come in and convert that into something that would work for an organization and that we could scale with.  I want to caution anyone else that finds themselves in the this situation to do it carefully.
And the reason for that is, you know, you don't want to, in an organization that is not used to process, be the guy that comes and lands a ton of process on the team.  They will generally not react that well.
If you find yourself in this situation, I would say just avoid boilerplate solutions.  If someone comes to you with an, here is an agile framework book, I'm telling you to back away slowly.  There are some variants of Agile that promote inter-team coordination, and coordination more than they promote actually shipping software, and that's a very dangerous thing to bring into your team.  I will talk about the original Toyota Canban boards.  If you wonder where they come from, they came from Toyota.  They have a process, Kaizen, an important process for how they build engineering teams.  If you are in the factory and working on the production line, there is this thing called the add on cord on your production line.  If you are working on a part and something goes wrong, you pull on the cord, the whole production line stops, the managers descend from manager heaven, or where ever managers come from, (laughter), and they say what is wrong with your station? And only then does new process get added to the production line.
  And the reason I like this is because it means that process forms from the bottom-up in an organic fashion, and it is tailor-made to what that production line needs.
And now, stepping that back up, and getting a little bit more meta, I think that is important because your process will become your culture over time.  For Reddit, we want to be agile and flexible, and that is starting the company with this bottoms-up organic process is going to take us to the future in a way that I want the engineering organization to grow.
All right.
And so, let's walk through what we did really quick.  This is Reddit's JIRA evolution.  Actually, Marty and I put this together.  We started with a six-step instruction manual that we came up with: Definition, planning, staffing, execution, launch, monitor, any work in the organization can be one of the states, we work with the IT department to get that into JIRA, that's a screenshot of it there.
  And this allowed us to get to team dashboards in JIRA, you can see, this is for our Reddit community experiences team.  And you can see across that entire organization all the epics and what state they are in, backlog, staffing, planning, execution, monitoring, this was not useful for daily stand ups.  But when we got the team committed to putting stuff in a JIRA, we got everybody Kamben boards which was great for stand ups.  And then we needed to report out, it was not enough to get the boards, we had to report out to the rest of the team.  Here is the first attempt, which is integrating the JIRA with tableau, it allows you to see a realtime dashboard of all of the work happening across the teams at Reddit in realtime with a health monitoring metric and an automatically generated calendar, I was pretty proud of it when we put it together.  And what we ended up with, I think you will like it.  Where we ended up with, instead of this, is executive dashboards.
These are updated in realtime, and people like them a lot more because we brought the cartoons back.
(Laughter).
  And the results of that, I think, speak for themselves.  In 2017, the new releases, adplatform, new search platform, chat platform, complete site redesign coming soon.
All right.
Wow, nobody is happy about that?
(Applause).
Thank you.
All right.
Okay, so that's what we did at Reddit, and now, I know that there's a lot of non-managers in the audience, I want to wrap up my talk for a couple quick tips for how you can take advantage of scale as an IC.  So three quick career tips for scaling.
First one is to get comfortable being uncomfortable.  As your organization grows and changes, you have to believe me that there is opportunity in change.  Comfort is the enemy of progress.  And this advice kind of applies more broadly as well.  If you are in a rapidly-changing organization, you will see more opportunities and, in general, as a software engineer, one of the reasons you will like working in this industry is because you get to continually learn.  While I have given this talk, three new JavaScript frameworks have been released.
  (Laughter).
  And get comfortable getting out of your comfort zone.  For me, that meant leaving Microsoft to a start-up, it meant me doing an MBA while working full time, find ways to push yourself.
  Second, follow the investment.  And so re-orgs, you know, you may have a manager, your manager is always going to tell you that the work you are doing is the most important thing in the organization, trust me.  But nothing speaks louder than a re-org to tell you where the managers and executive team want to spend their bullets.  So if you understand the reorganization, then you understand the priorityles of the leadership team, if you understand that, you can put yourself in the most valuable positions in the company.
  And finally, be a process pilot.
If your organization is growing and scaling, it means that people like me, people like your managers, are all trying to figure out the right process for your team, and it is actually very, very hard.  Because you never know the right process up front, I told you explicitly, don't try to land on a team with this agile framework, it will solve all your problems, don't do that.  And you as the manager in the organization can help to pilot to find the right solution.  And the other thing that is important about this that I said before, your process will become your culture.  If you want to have a great culture, try to participate in this.  You will have your manager and your future engineers.  That is quite advice.  Final thoughts on scaling fast at Reddit.
Two opportunitys, and one challenge I want to leave you guys with.  And these are both maybe a little bit more personal to me.  I think that, at Reddit, because we are scaling so fast, we have an opportunity to do a great job of diversity and inclusion.  There will be a track on this later that I think will be interesting, but this is important for me, because I believe that diverse and inclusive teams build better products, and more importantly, I think they are more fun to work in and that is overlooked.  Coming into an organization and meeting all sorts of different interests and types of people is something that I enjoy and I think your teams will as well.
And, like any cultural aspect, you want to get this fixed early as you scale, because it is extremely difficult to fix later on.  I don't need to tell you guys about all the different stories about D&I problems in the Valley right now, but at Reddit we believe, as we grow, we can solve this problem, and make it part of the DNA from day one.
  And right and wrong ways to do this, I think that rather than hiring a head of diversity, the best thing to do is to get your entire executive team involved.  And so, at Reddit, each one of our execs as KPIs associated with D&I.  We don't assume that everyone cares about this topic, but we try to find ways to allow them to participate through employee resource groups.
  We don't complain about the hiring pipeline.  One of the fundamental things that I learned when I came to Silicon Valley from Seattle is the value of bootcamps.  I think my first week at Reddit, we invited a group called Hackbright to the company for mentoring and I got to meet the hardest working women that wanted to switch careers in technology that I have ever met in my lives. And we have took many opportunities to hire bootcamp graduates into Reddit, I can tell you for sure they are some of the strongest leaders in our organization, and this is not something I would have expected, but I'm a strong believe.  Don't complain about the pipeline, find ways to actively change it.
  And the final thing, don't ever in your organization say "diversity of thought." I will find you.  Never see that you don't see gender or color, I will find you.  When you say those kind of things, it kind of means, if you tell me that you don't see color, that means you don't see me and that can be hurtful.  If you are a leader who is responsible for building an organization, be intentional and be patient and stay focused.  You can move the needle.  I think you have the tools and the opportunity now.
And, uh, social impact.  As your company gets bigger, just remember that you can add more value to the world than the software that you shipped.  We have us at Hackbright, extra life, play video game and raise money, nothing wrong with that.  And this is where a director went to Uganda to train some up-and-coming entrepreneurs there.
  And finally, a note to managers about scale: Long ago, whether -- when I was a new manager.
Sorry, last story, I promise.  Long ago, when I was a new manager, I was trying to -- I was bold and I was starting to -- I was trying to understand what it meant to be perhaps a director some day, or even an executive:  And so one day, I went to the office of my senior vice president, Kirt Delbenny, I camped outside of his office until he showed up.  And I asked when he showed up if I could come in and pick his brain.  After a long conversation, I asked him, what do execs even do? What do you do in your office all day? He showed me, me took me to this desk and he showed me an Excel spreadsheet.  And he said, Nick, I change the numbers in this Excel spreadsheet.
(Laughter).
  I'm not kidding, he showed me that, I was baffled.  I thought he was joking.  I change the team sizes here, it changes this budget allocation here, that's pretty much what I do.  But he told me, following that, the hardest part of his job as a leader as an executive is remembering that when he changed the numbers, it affected hundreds of people.
  So, as your organization scales, you can sometimes find yourself in this very risky, very unsafe proposition where you forget about people and you start calling them resources.  So if you are a manager in this room and you are responsible for growing an organization, please be careful.  Just remember that people are the most valuable part of any organization, and that the reason that you became a manager in the first place.
  And so, with that, I have a few calls to action.  Reddit is hiring, (laughter), I have to get that pitch in.  We also have an internship program if anybody is interested in that.  Follow me on Reddit, Twitter, medium, or LinkedIn, and my wife and I started the Color Code fund, we are raising money for people of color in the tech industry.  With that, you have been an amazing audience and thank you so much.
(Applause).  
>> Thank you for attending the keynote.  The next session begins at 10:35.
.
  Live captioning by Lindsay @stoker_lindsay at White Coat Captioning @whitecoatcapx