Welcome to fundamentals of big data architecture. I have myself, Mike Beranek, and Tyson Griffin with me, we are going to be your instructors for the course. Do you want to say hi quickly? >> Yeah, everyone welcome. >> All right, that's Tyson. So let's jump in, I guess there's an interesting setup to get going here, and naturally we're going to talk about big data. But really the architecture around big data, and this is a fun quote, right? All problems in computer science can be solved by another level of indirection. I think throughout the course, we'll talk about things like abstractions, and abstraction will become your main tool. And I think that's especially important around architecting and architecting systems around big data. So another slide, that's a fun one maybe, it's a tweet I sent out a while back in terms of our long standing recipe for success. So Tyson and I, have both written software for quite a while now, we're at a company called pivotal labs for the better part of a decade. And now at our own consulting company, and most of the projects that we kick off follow at least some flavor of this recipe. So balanced team will talk about getting to small stories, it's fairly important, continuous integration you might have heard of and things like site reliability, continuous delivery. Ensuring that you could deliver secure code to production as well as then deploying it to a platform as a service. Retrospectives are important and today, we also focus on safe learning environments. Anything else that you'd add to the recipe or good start? >> I think that's a great start. Yeah, I think maybe calling out retrospectives, that's what kind of keeps the whole thing humming and allows you to get better every time. >> Yeah, exactly. So a couple other things, if you think about 98 is roughly maybe when I started writing software, and we kind of had it backwards, right? So we used to think about developing and then we would send a CD off to a QA team for testing, and then once tested, they would send it to another crew to deploy. And now that software development lifecycle was 6-9 months. And what we found is after nine months or even longer sometimes that the software that we put together wasn't relevant anymore. And that holds true today even more, so I'd say you want to talk about the right side and what we're doing today. >> Sure, yeah. So we've kind of flipped it on its head now, and the first thing that we do is deploy. Next, we write some tests, and the final thing that we do is actually developed against all that, the idea is to reduce risk, right? So if we if we deploy first and there's no risk of getting the production writing our tests, make sure that our software will behave correctly, and then we develop. And then maybe the other key thing to call out is that the cycle is much shorter. We're doing this daily rather than every 6-9 months, so we're going to make mistakes of course. So when we do make a mistake, we catch it the next day rather than 6-9 months down the road. >> Yeah, and when we think about architectures, there are architectures that lend to this call it rapid application development for the moment. And truly being able to bug fix or get up features the next day which is pretty important for companies these days. All right, so next slide there's one more bit around maybe that recipe, and when we think of minimum viable products or an MVP, will actually bake in some things early on. So maybe a hint of scalability, some basic monitoring will communicate up time, what it looks like to be healthy, and how much downtime were maybe allowed each week,each month. And then we'll try to deploy software continuously such that there's no downtime for our users. So a lot of times they don't even realize that a new release of the software has been deployed. And if you're not doing some of these things, it's truly just a prototype. And I guess where we've been for a handful of years now is driving towards that minimum viable product and then iterating on top of it. Anything else there? >> Yeah, maybe you know the whole point of getting an MVP out there quickly is to get user feedback, and get information so you can change direction. And I think the idea here is basically without these four concerns like it's hard to get feedback on your product itself. But if your product is always going down, because you're deploying, if you don't have monitoring then you're not getting that quality feedback that you need to change your product. Okay, so I guess where some of this started as I mentioned I've been teaching a class at the University of Colorado for several years now. And we put together an article back in maybe it was 2015 or so, around how to think about evolving software over time, and naturally a big emphasis on evolutionary architecture. And I think it was this article that some of the crew with the in the University of Colorado said, I think what Mike's up to is really big data, right? In terms of how he's thinking about architecture, and how he's putting things together for the software engineering course. And this lead to, sure me teaching at CU, but then also putting together this course for you here on Coursera. So we'll talk about the AppContinuum, and how it lends to architecture, evolving architecture, and especially around big data, we'll bring in some of those big data concerns. Okay, well, that's our first bit of intro here, we'll dive into some other topics next.
