00:00
Welcome to the Cables to Clouds podcast.

00:15
Cloud adoption is on the rise and many network infrastructure professionals are being asked to adopt a hybrid approach. As individuals who have already started this journey, we would like to empower those professionals with the tools and the knowledge to bridge the gap.

00:31
Hey everybody and welcome back to the Cable's Two Clouds podcast. I'm your host this week, Tim. And with me as always are my heterosexual life mates, Chris Miles and Alex Perkins. How are you guys doing today? Chris, what's going on in the Land Down Under? Hey man, not much. Nice Wednesday morning here for me.

00:54
Got a busy work week ahead, nothing too exciting. I think I'm gonna play squash this evening with my girlfriend, so that'll be fun. Yeah. I wasn't a squash man until recent. She got me to take it up and man, it's really fun. I love it. Is squash some kind of innuendo thing or I don't know what that is? Squash is a sport. I always thought it was some rich white person sport and it probably is, but it's very fun. All right, all right.

01:23
What about you, Alex? Are you doing any squashing? No, no, no squashing done here. Um, yeah, actually this, this week's pretty slow. Uh, had a update at work today that completely removed my Bluetooth and card reader drivers. So that's, that's always fun. You know, you don't need a card reader or anything, right? No, of course not. Yeah. I don't need to log into customer environments. Oh my gosh. That's all. Yeah. No, not, not a whole lot. Just, you know, it'll be busy later in the week. I got more maintenance is coming up, you know,

01:53
Like always, I'm the one that does all the work. You guys play squash and whatever you're up to, Tim. This is a momentous occasion for me actually, Alex, Chris. I just received my proof copy of my book that the hybrid cloud handbook for AWS. I've been writing this for, thank you, thank you. I've been writing it for a few months now, yeah. And I've never actually, so I've written books before, but I've never actually gotten

02:23
book like that I've written with the intention that it be a physical book So it's kind of cool to be able to hold your own book and kind of leaf through it and see the Diagrams and all the the cool stuff in there, man I'm I'm excited to I've already got people beating down my door Like when I posted a picture to Twitter and they were like I searched for it Amazon. I can't find it Why can't I buy it? I'm like, oh slow down I just want to make sure it was good before I hit the button, you know But that's a good problem to have so by the time everyone sees this

02:51
I guess it'll be on sale, but it was really cool to get that today. Anyway, we actually have our first guest with us today. I'm really happy actually that it gets to be a good friend of mine, Steve McNutt. We used to work together at Cisco.

03:11
We actually, well you know Chris, Chris, you, you, myself and Steve were in router gods together studying for our respective IEs. Steve for his CCIE security and you know, you and I for the route switch. So Steve, I will let you introduce yourself, but I'm really happy that you could make it man. I'm stoked to be here Tim. This is like super fun. And by the way, congratulations on the book. That's awesome. Thanks man.

03:37
So, yep, so I'm Steve McNutt and I work for Cisco Systems as a Cyber Security Technical Solutions Architect. And I've been computering for quite a long time. And so, yeah, happy to be hanging out here today and hopefully we'll have some fun chat. Awesome. Thanks again. Thanks for joining us. We're just kicking this thing off. And so it's great to be able to bring people on here who have a lot to say and specifically

04:06
people that are kind of on the same journey that we're on, whether that be fully on the cloud side, like Chris and I, and like Alex, and we're all getting fully in the cloud journey. And then there's people that are still on the on-prem that are just now starting to wake up to the whole cloud thing, whether they want to or not, right? We all agree that the future is hybrid. So it's great to be able to bring on other professionals that are on that journey as well, especially one.

04:35
that's focused on security. So this is really cool. I know that as your customers are moving to the cloud, obviously we're having to do a lot more of the security, work more in the security space and that it's like totally different than it was when we used to do it on-prem. So yeah, I think that's like the biggest, I think that's kind of where we wanna swim for a little bit and I would love to know your thoughts about like,

05:04
You know, kind of like when we started, and this is my own experience. I don't know if you agree or not, but when I, when we started, I always felt like when I was doing security on prem, a lot of times our security posture and our design had to kind of follow the cable a little bit, like we had to like all, you know, we had to find out where we're going to put the firewalls. Are they going to be in line? Are we going to be bouncing through switches to get there and like all the routing and everything? I don't know. What do you think? So yeah, let's start, you know, let's, yeah, let's start with just sort of like a 101.

05:33
And really the classic firewall design would be like a perimeter firewall that just sits on the edge of a network and a perimeter firewall, they typically had three legs on them, right? You had the inside and the outside and then the DMZ. And up until the cloud came along, networking was flood and learn, right? So everything tried to discover everything automatically and then tried to connect to each other. And then when you want to implement security,

06:01
you know, you intercept that flood learn behavior by, you know, various means of segmentation or whatever. And sort of, you know, early on, and I'm dating myself here, but I can remember like way back in the day deploying like old checkpoint HA pairs and PIX HA pairs. And the whole idea behind the HA pairs is you would use layer two tricks, right? You would use like Mac addresses and stuff and you'd have two firewalls, the little sync cable.

06:29
So that the backup firewall would know everything about what was going on, all the connections and everything. And then if there was a failure, then another firewall would take over and it would actually use the layer two address of the other one. So that the AJ pair was kind of, that was like a bread and butter. That was like a three legged firewall in an AJ configuration was like all you needed to know to make a pretty good living as a firewall admin for a very long time. Right? And then, so, you know, the next step beyond that is...

06:58
You know, in data centers, people started to recognize that you couldn't necessarily trust the interior traffic of your network, right? Sort of the insider threat thing started to evolve, right? You know, John Kinderberg wrote some, a couple of seminal papers about that. And so along came what I would call a big honking firewall. And that would be just these giant firewalls sticking to data center, right? Like these, so it's still an AJA pair, but now you have like these

07:27
really big, super expensive firewalls. And now not only are you controlling the traffic that's going in and out of the network, but also between your workloads, right? So you might have your web servers and your database servers and everybody lives on a different VLAN or whatever. And then you're passing them through these really expensive ass firewalls. And that was sort of the next iteration, right? And then you had clustering. So clustering. Hang on a second, I want to stop there for a second, Steve.

07:54
Sorry, one sec. I just want to stop there for a second because I know that everyone on this call probably has, when you're talking about the big honking firewall, I need to just for a second remember how awful the FWSM is in the 6500 chassis. Just have to like pour one out for the old FWSM module in the 6500 chassis. Yeah. Alex, Chris, did you guys use that one?

08:23
Did you guys use that one before? I think that was before. I think it's just you and me, Steve. Just the old timers. I didn't have to do that. Oh man, sorry. I didn't mean to interrupt you there. I do think it's funny that just calling that out, it kind of shows that RFC 19, was it RFC 1925? Yeah, like the it wasn't unique to just like

08:48
networking infrastructure. It was always the firewalls did the same thing. It was like, it's distributed, it's centralized. It's distributed, it's centralized. Like everyone just kind of kept rolling back and forth. Like each, it was a better idea each time. And when, you know, it's just funny that that's how, how the world works, but you know. Yeah. And it's, and it's always just renamed something else, right? North, south, east, west. Now is all people classify traffic as. And it's like, it's the same concepts, just called something different.

09:15
Sorry, you guys are too young to remember the horror that was the 6500. Yeah, I was just thinking about the distributed thing. For a while, people were building these wide area networks over the internet with IPsec to save money using the firewalls, right? And it seemed like a pretty good idea. And then when NGFW's next generation firewalls came along, suddenly firewalls became a subscription product.

09:41
And suddenly it was like super duper expensive to do that.

09:47
So that helps kind of feed your big honking firewall, right? Let's just fuck it. Let's just back-hile all the traffic to some gigantic firewalls in a data center and just sort it out. Oh yeah. I remember that. Uh, well, like, you know, we'll put these giant, uh, data center, or data center firewalls at the edge and then all of our branches will back-haul everything over. God knows what kind of lease line or, or, you know, if you're lucky, you had some sort of an MPLS circuit or something that was always full.

10:15
Oh man, I don't miss those days. Yeah, me neither. Yeah. I mean, even the model that you were talking about, right? Like having the DMZ and then the three-legged firewall. I mean, that's like, nobody even looks at things the same way anymore. It's just kind of funny how things change so much. Well, you know, I think it's interesting though. We, I think as we start to talk about cloud design patterns, you start to see some themes re-emerging though, and it's kind of funny.

10:43
You know, like as an old guy, I'm like, ha, see, you still got to do it that way. It's still a better way to solve the problem. So do you guys want to start talking about sort of that movement? Yeah, I was going to say, I think the difference now is it's, it's not so much, you know, five different components in a single firewall. Now it's five different firewalls, right? And, um, you know, you're kind of leveraging the, the, the topology that exists in the cloud to, to achieve the same goal really, right? But yeah, so let's, let's segue into that. So.

11:13
Tim, where do you want to take it from here? Actually, I think you're headed the right way, Steve. I would love to talk a little bit about, and now that we've covered 10,000 ways to do security on prem, although I really liked, and I didn't get a chance to, I wanted to let you finish. I like what you said about how security in general has evolved from the castle walls and the moat to actually like, okay, wow. Okay.

11:42
There's already people inside the the barbican here. We need to do something about that. And I'd love to take that forward into how we have to design the security piece in the cloud now that we're kind of trying. Actually, I think it actually makes it a little easier for us to do that kind of interior design, if you would call it that, in the cloud as well. So yeah, definitely. Let's talk about that, Steve. Sure. So yeah, so coming back to our initial idea of how traffic gets from workload to workload or whatever.

12:10
And point to end point, you know, that obviously everyone who's listening to this podcast, I'm assuming are 99.9% of the people listening to this podcast appreciate that there's no such thing as layer two in the cloud, right? Flood and learn is not a thing. And the other one of the cool aspects of that is you have a lot more control over who can talk to who in the first place. So you don't have a lot of problems around trying to constrain that flood and learn behavior. On the other hand, now we've lost all our layer two tricks.

12:40
Right? Suddenly having your HA firewall pair is not as simple anymore. And trying to create that, recreate that in the cloud ends up honestly being kind of an ugly hack. You know, it's not that you can do it, but it's kind of weird. So and the other thing, you know, the other thing that's really interesting too is stuff that's pretty expensive to do on prem, like in your on prem data center, like, for example, having just basic access control lists

13:10
you know, at various, like either at the interface level or the subnet level or whatever, is kind of a pain to do with on-prem tooling, but it's literally wired into cloud networking. Like by default, when you, for example, with any of these cloud providers, if you just use the GUI and just say, give me a Linux virtual machine or Windows virtual machine, it's gonna have like a little network security group attached to it right from day one. So that's a huge improvement. And I think in the beginning,

13:38
people thought, well, gosh, you know, that's all you need, right? You can just do a little programming and, you know, kind of automate this a little bit and everything's great, but, but not really, because you still have, you still have to look at the traffic that you're letting through, right? That's, that's the rub. And so now we have to get another section. Yeah. So I'd love to hear what you guys have. There's something there to that. Yeah. Steve, there's something there to that too, right? I mean, is that, what are, what are your thoughts around?

14:06
Do you think that's because there weren't a lot of network engineers that were using the cloud yet? And it was more, right, the Wild West and app dads were coming in and just standing this stuff up? Like, what are your thoughts on that? Well, I have some theories about that, but I don't know that they're reality, right? They're just my opinions. I think that, you know, and I say this as like an old school routing and switching guy, you know, that moved into, you know, security. And that is...

14:35
When you're in operations and you're doing the network, you get burned so many times by the application guys deploying like broken stuff and you got to support it and make it work in the middle of the night that you become very defensive and you become very conservative and you want to have, you know, and there becomes this conflict where the developers want to ship it and you're like, I don't want to, I don't want to get up at 2 30 in the morning on a Saturday to fix you, make, you know, make your crappy code work, please stop. And so I think when this, when the

15:04
When the developers got into the cloud, they could just whip out their credit card and just do whatever they wanted. They didn't invite the network guys to the party. And I can tell you, I've seen a lot of version one cloud deploys and they all, they're all, every single one of them is a mess, right? It's a typical thing that you see with organizations is their very first cloud deploy looks like crap and they realize it's not supportable. And so usually the version two cloud deploy, at least some network, someone with some networking knowledge gets invited.

15:33
party at that point. And I think that's kind of where we are. So that's my kind of my thought on that. I'd like to hit on this for a second, because you said something else that I don't know that was on the script, but I think is really cool. And it's going to kind of inform this whole future being hybrid thing and so on. So what I'm finding is that as enterprises move, so for the longest time, right, everybody's been, quote unquote, moving to the cloud.

16:03
But very few people, unless they were born there, have actually made it, like gotten all the way into the cloud. And what's happened is that, hey, we'll do cloud first with all our new apps, and we'll do cloud first with the stuff that we don't care about. But as we get more towards people putting their crown jewels in the cloud, this is when the sudden focus on repeatable networking and on networking that's supportable, and on security especially, that's not only supportable, but like baked in.

16:33
to the product, if you will, has become really, really important. So I just wanted to throw that out there because I think I'm seeing that more and more. Right, okay, so there's, now here's, I'm gonna, we are kind of going off, Chris, are you gonna say something? No, go ahead. Oh, okay, cool. So I'm gonna go a little farther off. I'm not gonna try to make this into a huge tangent, but so one thing I've seen, you know, that I do think is a big driver for that, Tim, is, you know,

17:01
It's no secret that ransomware has become big business and cybersecurity insurance has become big business and cybersecurity is now a board level concern. So one of the advantages of my job and the role that I have is I get to see a broad spectrum of organizations, just like y'all as a matter of fact, we're privileged in that aspect, right? We get to see how all these different companies do it and learn from them. And what you'll see is almost every one of them, the cybersecurity, that's a board level discussion.

17:29
And it's usually part of a company's risk management program. So what's happened is the risk management people are now making decisions about technology. And so if you're putting your crown jewels in the crowd, the risk management guys, they're going to want to see the table and they're going to want to see what's going on and they're going to want to analyze it through the lens of their framework and have some say about how things are going to be done. Right. That's great. I love it. You're absolutely right.

17:54
Yeah, that's the governance, the cloud governance framework, right? Like there's, they always have a say, security definitely has a big say now in building out those kinds of things. Sure. I mean, most, yeah, sorry. Oh, go ahead. No, no, no. Finish your thought. I'm sorry. I think we must have a little, just a tiny little bit of lag here, but yeah. Finish your thought please, Steve. Yeah. I get really eager about this stuff. I never get to talk about it. So yeah, with, you know, usually, you know, most organizations are going to use, if they're like a US based company.

18:25
You know, they're going to use like NIST cybersecurity framework is real popular because it's free. And then you've got companies that maybe if they're doing government subcontracting or anything like that, they're probably going to be using a 800-53. So that might be something to look up if you're interested. And for, for companies that are, that are doing business globally, they're going to use something like ISO 27002. So those are, those are, if you want to understand decisions that are getting made.

18:54
with how security is being deployed, knowing these frameworks was, it's kind of gives you the cheat codes because the risk management people are depending on these frameworks to make sure that they've got adequate coverage. So yeah, that's like a huge change. That's great man. Spinning up firewalls, right? So much easier back. All right, it's so true, man. And that actually does bring us back a little bit more on target because.

19:20
I mean, what you just said is exactly right, right? Like the risk management security, like the governance piece is gonna determine how we design our cloud security, right? So let's talk about a little bit about how cloud security looks, you know, not just from a fact that we can, you know, the cloud is almost like a carte blanche kind of environment where we can build stuff, but specifically how we're gonna build those kind of North, South, East, West, whatever you wanna call it this week, you know.

19:48
protections for our workloads and stuff. So, cool. So this is where I get an opportunity to kind of bring things full circle and we start to see maybe some older patterns emerge because they solve problems for us. And specifically, what I'm talking about here is if you're kind of look at things like in layers, right? You know, you have, you know, like the forwarding plane, right, and this is where like, traffic's moving around and all this is happening and you've got all these controls in there where you're saying who can talk to who and...

20:16
you're going to doing all this logging and all of that, right? You know, but sitting above that, you're going to have some kind of management layer, right? So the risk management people, that's where they live. And that's where the security operations team who helps risk management, they kind of live in that sort of analytics layer where they're getting all of this telemetry from all this stuff that's happening and they're trying to figure out, is everything does everything look like it's supposed to? Is there anything anomalous going on? You know, are we within our?

20:44
tolerable baselines and all of that. And so observability becomes very, very important, right? And that's now, that's where like network security groups and the sort of static, actual base controls no longer work. Now we need to look deeper, right? We wanna bring more advanced tooling, right? We wanna bring in analytics and machine learning. And you also wanna be able to bring in all these different information sources, you know, threat intelligence feeds and things like that to try and.

21:13
analyze what's happening. And so that means we need to get a look at that data stream. And so now we wanna be able to inspect that traffic at a very deep level. And the reality is that the cloud providers haven't caught up to that, right? So the CSPs, the cloud service providers, their native offerings just aren't gonna get the job done there. Just knowing these companies, they'll get there, but they're not there today, which means we're back, guess what, we're back to, you know, hi there, firewall guy, how you doing?

21:43
Let's talk. And that's a weakness of the hyperscalers, right? Cause they can't expose that level of telemetry to every customer, right? That's just a weakness in the model, if you will. Yeah. Well, no way they want to, right? Yeah. Yeah. Right. That's true. That's true. You know, that's absolutely true. So, you know, now we have to come back and now we're starting to get a look at that more of a centralized model again, right?

22:09
where you want to be able to efficiently look at that traffic. So maybe now we want to set up like inspection points. You know, maybe you want to for each, you might want to set up like, if you already have your cloud network architected properly based on other considerations like delay and high availability and stuff like that, you're probably going to have some kind of, you know, surprise spoken hub type of design, right? Where you're going to have some kind of some kind of transit hubs that you're wiring your applications into.

22:38
and that you're wiring your access from the outside world into. And so those are, that's a very logical and searching point for our security. And I can remember the first time I ever built something like that. And I'm gonna use a lot of Azure speak because that's the place I have the deepest knowledge in. Yeah, I know we have some AWS experts here who can maybe help me compare and contrast a little. I would love that. With respect to with Azure.

23:06
You know, the original design was like the spoken hub VNet was like the first real, really scalable type of design in Azure. But it's got a couple drawbacks and one of them is you just have this explosion of user defined routes. And so you have to use routing to get the traffic into the firewalls. They call them, by the way, I'm going to start using acronyms here. So I'm going to define this network virtual appliance is what a cloud service provider calls like a third party firewall. Right. So I'm going to use the term NVA.

23:34
just a shorthand so I can talk faster. So you have your hub VNet, all the traffic is transiting through the hub VNet, very useful for a couple of reasons. And you wanna be able to get the traffic into your NVAs. And so in order to do that, you have to use routing. So you have to override the cloud service providers routing with your own routes, but we don't get routing protocols really. We don't get at least interior gateway protocols with cloud service providers. So now you just have this. Can I cuss? Is that okay?

24:04
or not okay. I'm sure fucking hope so. All right. Yeah, absolutely. See, now what you end up with is a complete clusterfuck of static routes and it's like impossible to keep track of. You know? And it just becomes a mess, especially if you have like a lot of spokes, you have like a big deploy with like a lot of apps and stuff like that and a lot of things getting wired in. It just, it's like, and so I think that's part of it before you get the popularity of Terraform is because it allows you to wrangle that shit and kind of get it under control a little.

24:31
To manage it, definitely manage the lifecycle and deployment and all of that. Yeah. And, you know, and what you're talking about, that's even before you bring in things like Azure Private Link, which, you know, insert like underlay level routing into the, into the VNet that you can't even touch stuff like that. Right. Yeah. And also, I don't, I don't want to sidetrack you. Sorry, I knew this was going to be my fault. Yeah.

24:55
Yeah, so I think it's kind of circling back to the kind of traditional way that we thought about this thing in the on-prem world. Yeah, I think to the point you made, Steve, it was like everyone was used to this kind of like HA configuration where there's always, you know, you're maintaining state amongst all these NVAs, basically just the virtual form factor of that. And you know, a very concrete piece, a foundational element in the cloud is auto scaling.

25:25
That was something that a lot of the on-prem people struggled with is you don't need to maintain a lot of the state with the active and secondary firewalls. But with some of the NVAs, you did have to do that because the NVA vendors did not catch up and were able to support that auto scaling aspect of it that a lot of the cloud native stuff does, right?

25:53
Which is why I think you know kind of segueing into maybe what we were originally planning on talking about today is why the the CSP's introduced things like like gateway load balancers to kind of Let you leverage the NVAs without having to maintain a lot of that state, right? Yeah, that's the yeah, so the the end is the NVA came along because people were building these networks where they were they were putting

26:21
Like you said, you're putting in these firewalls as inspection points, like in your hub transit points. And so the original, I guess, which we call version one of that, right? So this is the cloud version of BigHonkin Firewall we're about to talk about. So the cloud version of BigHonkin Firewall is you have like a bunch of NVAs sitting in a subnet or V-net or whatever you want to call it. And then you end up having to have what's called a load balancer sandwich. It's the classic design.

26:50
for cloud network where you have an inside load balancer and outside load balancer. So for the benefit of the audience, just to make sure we have a level set here, there's a reason for why you have a load balancer sandwich. And that is because just what Chris talked about, you have this because of the statefulness of a firewall. A firewall has to keep track of states so we can analyze the connection, right? It has to see the traffic flowing in both ways. And if you have a auto scale group of virtual appliances, you have to make sure that that session, that traffic in both directions always goes to the same.

27:19
So guess what? We're going to dust off an old networking trick, which is NAT, network address translation. So the traffic comes in from the outside world, hits that outside firewall. Then when it addresses that firewall, it's going to use what's called source NAT. And then it's going to go out to the inside load balancer, into the workload. So why do we do that? So when the traffic comes back, the inside load balancer...

27:47
is going to direct the traffic to that source net, to that address of the correct firewall. That way we can guarantee traffic is always going to transit through the same firewall. And let me tell you something from an operations standpoint, think about that for a second. You've got like a bunch, you got a problem. It's 2.30 in the morning. You're trying to figure out what's going on, but all the client IP addresses are freaking getting changed inside the load balancer sandwich and you're like ripping your hair out. What's going on? That's just, that's not a good time. Right?

28:17
So load balancer sandwich sucks, but it was all we had. And I think what happened, and I'm just guessing, is that the cloud service providers looked at that and said, yeah, we've got to solve this. And enter our friend, gateway load balancer. We took a long time to get to it, but ta-da, gateway load balancers. Let's talk about them, how awesome they are, how they solve some problems for us. So I'm gonna take a breath for a second, and does anyone have any thoughts they wanna add to that?

28:43
Just one quick note, I think NAT should stand for never absent technology because I feel like it's in fucking everything. No matter what. I love it. And you're right. It doesn't matter what solution. It could be two computers talking on the same network and we'll get NAT in there somewhere. Yeah. It's like it's green tunnels and NAT all the way down. If your problem can't be solved by a G or A tunnel, it could probably solve.

29:10
be solved by NAT. If not, then you're not using enough NAT or enough GRE tunnels.

29:19
It's so true. That's why it's funny. I know because we're now we're talking about But to your point Steve, I guess about gateway load balancers. Let's maybe take a step back and talk about, you know, in what situations you need one, when you would use one and just what are the high level components there. So you want to you want to kick that off? Yeah, absolutely. So we just we just talked about how we have an inspection point, right? We've got some kind of transit hub for our traffic and then we've got a

29:48
a scaling group where you got like a whole bunch of NBAs. And we get a couple of problems to solve here. First, we're gonna get the traffic into the NBA. So we can do inspection. Second, we're gonna make sure that both that the connection for a session always travels over the same NBA so that we can get proper inspection in that traffic. So those are really the two big problems we need to solve here from a design perspective, right? So the traditional way of getting the traffic into the NBA is routing.

30:16
And I just talked about how that can quickly spiral into a mess, particularly with Azure. And so that's the first problem. And it's specific and unique to Azure. And I think it's a cool innovation. I would be shocked if AWS doesn't start doing the same thing. Is you can actually take your gateway load balancer, which has got all of your special load balancer that has all your firewalls sitting behind it. And you literally can just wire it right into the load balancer for an application.

30:44
So if you have like a web, like a load balancer sitting in front of a farm, a web server serving up a web app wiring in that firewall and getting the, and having it process all the traffic for you. It's just a simple matter of like a dropdown. You don't have to configure any routing. You don't have to get the routing guys involved. Operationally. It's amazing. Cause now you, now you don't have NAT anymore. You don't no longer require NAT. So the firewalls it's now you have a firewall of stick. It's not doing any routing and it's not doing any NAT.

31:12
It's acting like it just sits there and looks at the traffic. It's beautiful, right? It's just doing firewall stuff now. None of this other crap that makes life painful. It's amazing. Such a great idea. So the idea is you have traffic coming in and the gateway load balancer, in Azure at least, will actually take that traffic and it uses something, it uses an overlay called VXLAN. It actually, I'm sorry, the internal load balancer, when it goes to gateway load balancer in Azure, it actually uses a green tunnel.

31:41
They don't tell you that in the documentation, but that's what it really is. There's a green tunnel that gets wired into the Microsoft gateway load balancer. And then from there, the gateway load balancer has all these firewalls attached to it and that uses VXLAN. Aha, like on-prem data center stuff. Awesome. I understand this. This is easy. Cool. This is my world. I get this now. So if you're like a, an old school networking guy, now you're in your happy place, you know exactly what's going on. And the VXLAN...

32:11
Yeah. Sorry, just real quick. So the one thing that VXLens is doing for us then is it's letting us preserve our original source and destination IP for the inspection purpose. That's a huge one. And then, of course, because our firewall doesn't actually have to care about routing, we don't even need return IPs, return routes for the IPs that are getting past the firewall. So I just wanted to throw that in there real quick. Yep.

32:38
The only thing the firewall has to have is a route to the tunnel interface on the Gateway Load Balancer. That's it. It doesn't need a default route, it doesn't need a squat. All it needs to be able to do is talk to the Gateway Load Balancer. Oh, I take that back. It also has to be able to respond to probes. I found that out the hard way the first time I turned this up. There's actually a special IP address that Microsoft uses for the Load Balancer probe. A health check maybe. You know, wow that. Isn't it the APIPA?

33:04
Isn't it like one, I forget, is it the 169 addressing that they use the IP address for that? I watched everyone. Not completely. It doesn't make any sense. I sat there for four hours going, why isn't this working? Yeah. My firewall's alive. Why won't you? Yeah, just to go back a little bit. So kind of the point you originally made Steve was like it technically, or traditionally with a big honking firewall, you know, if that's not acting as, you know, kind of we'll come back to this, the, the, the bump in the wire behavior.

33:34
If it's, if it's, you know, your traditional firewall, you have to, you have to manipulate routing to get things, to get traffic, to hit that firewall, to be subject to inspection, right? Yep. So, and that, that, that adds so much more onto the firewall stack that you didn't maybe intend for it to do. Like it might have to participate in DGP. It might have to, you know, you know, completely, you know,

33:58
it's gonna influence all the traffic to come there. So maybe it's advertising default route and that can get very ugly if you don't want it to do that, right? So let's maybe kind of, because I know you talked about VXLAN in this situation, which is the encapsulation technology that gets the traffic to and from the gateway load balancer, which and then against it to the appropriate NVA, right? So let's maybe talk about that bump in the wire behavior and how that's actually accomplished

34:28
and encapsulation like that. Because I know in Azure, VXLAN is what gets used. Obviously there's the GRE tunnel before that, but VXLAN is ultimately what's encapsulating the packet, which lets you maintain that source and destination IP, right, so that doesn't get altered. You don't have NAT in this situation. And in AWS, I think they're using Geneve to do that. I believe you can add VXLAN on top of that. I don't know why you'd want to do that, but yeah. So let's have these. We need more fabrics. More fabrics. Yep.

34:57
Minimum payload, please. Yeah. So do we want to kind of talk about how that, how that is accomplished and what benefits you get from doing that? Well, sure. Okay. I'll take, I'll take like a high level stab at it. So would you really, what it really looks like is if you're unfamiliar with VXLAN, it's, it's actually kind of like a VPN, right? So you have what's called a VTEP or a VXLAN tunnel endpoint.

35:22
And that looks a lot like a virtual tunnel interface, like on a IPsec VPN. It's a similar concept. A VTI, yeah. Yep. So you have a VTEP. So you're going to have an IP address that's on your NVA. And then he's going to go talk to you. The Gateway Load Balancer has a VTEP too. So you're going to define what the VTEPs are. And then you're going to have what's called a VNI, or a virtual network identifier. So it's a lot like a VLAN. It's just like a big long number that lives inside the header.

35:52
that kind of tells the computer like, okay, this traffic belongs to like this virtual circuit thing, right? So you're going to have a V and I pair. So you're going to have one for the traffic that's coming towards the firewall or the NBA. You have one for the traffic coming towards the NBA and then one for the traffic that's egressing the NBA and going back out. So from the perspective of the firewall, what I'm seeing is firewall vendors are recognizing there's really no need to like overcomplicate this. So they're making it pretty easy where you just say, look, I want to create a...

36:21
I want to create a VXLAN connection and you just go in and you just define all that in one screen and hit go. It's real basic and you just have like one kind of, it looks like an interface construct, but it doesn't have an IP address or anything like that. And then it's just bound to a virtual interface that's the VTAP that actually sources the traffic. A virtual cable or something. Yeah. It's really straightforward. So, right. So we have this, we basically have this sort of virtual circuit, this tunnel.

36:51
And then the traffic is just, it's just encapsulated. And like Tim said, the beauty of this is everything is 100% intact, which can be like really important, right? So that's like all the header information, all the checksums, all the things try to, you know, it's not modified in any way, shape or form in the firewall, just firewalls. It doesn't have to route. It doesn't have to nat. It doesn't like all this other crap that we like Chris, so as Stooley pointed out big honking firewall, your firewall ended up being your router, right?

37:19
Which sucks because firewalls are not meant to route. And it's just, you know, so it solves it. It's just a beautiful and elegant way to solve a problem. You're using the firewall for what the firewall is designed to do. You're not routing with it. You're not NATing with it. You're strictly applying security policy and you're inspecting traffic with it. It's awesome. So hopefully that was a good answer. I just want to point out, like, we're all talking about this and we're kind of in amazement,

37:48
These are the things that cloud enables, that networking has been lacking for so long. Really, this solution is so elegant and just such a relief from the things. Everyone has heard all the things that Steve has been saying this whole time about how difficult it is to set this up manually. And now we have this model where so much of it is taken care of and transparent to you. And it's just incredible. These are the types of things that cloud can enable for networking. So I just want to call it out. But you don't have to follow the cable.

38:16
You don't have to follow the cable anymore, right? I say follow the cable a lot, but that's how on-prem networking ultimately is. Even if we do layer two tricks like Steve was talking about, ultimately we're constrained by the physical layout versus in the cloud or really any, I mean truly any fabric probably, but certainly in the cloud where we're talking about, you can create these architectures, bespoke architectures essentially that meet the need right off the bat and you don't have to follow the cable.

38:46
So that's one thing I love about it. You're one thing. I, one of the things I like about this is now we're innovating, but not only are we innovating, but we're using design patterns and we're using knowledge that us old guys already have, right? You can take all your skills of doing regular networking and they 100% apply here. You've got like a, you've got a huge headstart, you know, for, for doing these kinds of like really awesome design. So that's, I think that's really cool too. It just, it makes it really easy to pick up.

39:14
because we, you know, the cloud service providers were smart enough to just use things that already work, you know, like in terms of using encapsulation, right, setting up overlay networks to preserve the traffic information. And yeah, absolutely. It just, it's really great. Now, AWS just needs to fix it so we don't have to use routing for service insertion with their gateway load balancer, because you still have to, you still have to set up routing, right, from the, from the subnet with, with AWS. That's right. But it's still...

39:43
To be fair, AWS is a gateway load balancer that was like a lot more sophisticated than the Azure gateway load balancer. For example, it's more tunable. So for example, like say you have, like for example, we're talking about statefulness. So, and I'm gonna, this is not Cisco specific, but it's getting closer. So, one of the things that you can do, I mentioned it before, if you wanna go from HA, which is two,

40:13
You want to have more than two firewalls, right? To do that scale out design. So you have the concept of an auto scaling group, right? Like in the public cloud provider. But remember what Chris said about state, right? Because firewalls by their nature are firewall staple because it has to inspect the connection. So you have to make sure that traffic, remember that's one of the problems we have to solve. We have to make sure traffic's always going through the same firewall. But what if that firewall fails, right? So.

40:41
If you don't have, you don't have those firewalls sharing state information. When that fails, that means what happens is the Azure one is pretty slow to converge. Right. It takes a while for the gateway load balancer probe to figure out, okay, this guy's dead or whatever, right? It's like, there's the health check, right? There's a health check that needs to fail before it pulls it out at the load balancer, right? And it's, it's pretty basic. And so you lose like now you have like a, what, like a mini outage, right? So all the people that were going through that NVA, they're not happy right now.

41:09
They got the spinning circle of death and they're like WTF and whatever. Um, whereas, you know, like what you can do with Azure or with AWS on the other hand, and I'm sure Microsoft's going to fix it, but it doesn't work today. But with, with AWS, which you can do is we're going to again, take an older technology, which is clustering, right? So what, what is clustering? Clustering is a way that you have a auto scaling group, but now you have what's called like a, you have what's called like a cluster control link.

41:37
and you have a controller or a boss dog firewall that keeps track of who everybody is and what's going on, right? And he orchestrates, and so you actually can scale that sharing of state across N number of NDAs. So what happens is now, if you drop an NDA, right, the Azure Gateway Load Balancer is capable of doing something called a rebalance.

42:03
where now it's like, okay, we lost this NVA. Let's just take this connection and start reroute it through another one of these instances. And you actually don't get a loss of state. It's pretty sweet. So that's something- And this is an AWS, you said, Steve? Yeah, yeah. So you can use clustering in combination with the Gateway Load Balancer to scale your state across the auto scaling set of firewalls, absolutely. Oh man, I had not seen this. I gotta go look at it. We need to look at-

42:29
Yeah, let's find the link. I'm gonna find the link for that. We can throw it to show the show notes Yeah, back to Steve's there about that The NVA vendors are now enabling that option to add that kind of session sharing link Between these firewalls that get spun up and put behind the load balancer, right? but that's It's also kind of important to point out that this is this is unique to the gateway load balancer component This is unique to the NVA component

42:57
Because if you use something native like Azure Firewall or AWS Firewall, the CSPs have accounted for this, that all that is getting shared. They're just like not expressing that to the vendors, right? To let them do the same thing. So, I mean, naturally so. They're not going to let somebody get a leg up on them. I get it. But, you know, that's, that's, that is a unique problem tied to this specific use case. Yeah. The like AWS Firewall, Azure Firewall, they don't.

43:25
run into this issue because I mean, at the end of the day, we know that what's really happening under the fabric is that the CSPs are actually managing the same thing that you would manage, right? But they, of course, have home field advantage for all of that, right? And so what you said is spot on, Chris, is that all of that is there, but they're not going to expose that level. Multiple reasons. One is home field advantage. They don't want to give that up.

43:50
And second of all is probably there's some, there's probably a little bit of element of the hyper scale in there where they probably, you know, don't think they should or can't expose that level of control to everybody. It's hilarious on the diagrams that AWS puts out. If they'll show like an inspection VPC using gateway load balancer, and then they'll show the AWS firewall. And it's such a mess when they show the inspection one. And then it's just like one little thing. Right. One connection to the AWS firewall. It's a pretty little logo. It's the same thing.

44:18
Yeah, exactly right. Yeah, let's put that gateway load balancer packet walk diagram in the show notes because it's like, I know like at least 20 people that I know that have seen that and be like, this looks insane. And like when you break it down, it's not really that bad. But you know, at first glance, you're like, holy shit. This is like doing the packet walk is kind of a grudging thing. This is not a shill.

44:48
This is not a shill, but I will point out that I do actually cover the AWS gateway load balancer packet walk in my new book. Nice. Well, you know, I think- Go buy that shit. You know, what Chris was saying about it, you know, there's a lot of steps and it looks complicated, but the beauty of it is the cloud service provider takes care of so much of it for you, that from your perspective, you don't have a lot to do. It's pretty easy. They orchestrate that whole-

45:15
I mean, the actual gateway load balancer piece of it, and you just basically have to take care of the third party NVA configuration of the Geneva tunnels, I think, is it, right? Yeah, and at a very crude level of understanding for it, whenever a firewall vendor puts out a new image into the marketplace that says it supports gateway load balancer in respect of CSP, basically all they're saying is like, hey, we've...

45:41
built our NVA to where, as it gets spun up, if you spin up this particular machine image, it's automatically gonna build these tunnels to the game load balancer, right? So that's really mostly what's happening under the hood, right? No, you're absolutely right. I mean, and that's exactly right. Like as the, and it's a little bit of a race, right? The CSPs will come out with a feature, the NVA's vendors rush to basically support the feature.

46:11
And then there's a lot of this kind of back and forth between, you know, native and third party and, you know, who needs what and who supports what. So, and I don't think that's likely to change. I mean, eventually I'm sure there's some diminishing returns that will be reached, but I think that's probably for the foreseeable future, what we're going to continue to see. Yeah. The only thing I'd point out about that though, is it kind of, it's kind of going to get people stuck because it's like, if the CSPs are the ones that come out with the features, they're driving.

46:40
the features that need to be released. Like do you think they're listening to? The innovation, yeah. Right. And it's not like it's coming from the third party NVA providers and they're telling the CSPs, we need this, right? It's the opposite. So I don't know, there's something to that that's like maybe was just a little bit. Maybe there's a little bit of a push pull though, right? Alex, what I mean by that is the cloud service providers because of the way they do things, it's causing firewall vendors to have to increase their feature velocity.

47:10
right, in order to compete in that marketplace. That's great. That's a good point. Absolutely. So if somebody comes up with... Competing with each other specifically. Yeah, like if someone comes up with an innovation like Gateway Load Balancer in your firewall vendor, you better get to it, get that thing going, right? And on the other hand, right, they've got their cloud native firewall that doesn't quite have the features and functionality of maybe a more mature vendor, but...

47:38
they have the ability to kind of sit there and see what kind of usage patterns are developing and they can cherry pick the high impact use cases to put into their own products, right? So it's pretty interesting. They have the second mover advantage or whatever they call, I forget there's the fast follower advantage. There's a name for it, but I know exactly what you're talking about, Steve, and it's just the ability to not have jumped in and committed to something first and be able to kind of observe what people want and then, and are using and then build the product direction that way, you know?

48:08
So let's play Forecaster a little bit. Do you guys see, I guess I should say, how long do we expect before we see a next-gen firewall released by one of the CSPs that is completely cloud-native, that does a lot of this feature set that we're asking for? Or do you think they've kind of given up on that race and they're gonna let the experts do what they want? I mean, I think there's an obvious answer here. They're gonna try it, but how do you guys feel?

48:36
So you have a Microsoft in particular, they're spending enormous amounts of money on their security program and it's evolving pretty quickly. It's definitely a high priority item for them. And then if you're gonna, I'm gonna bring in another player we haven't talked about, which is Google pretty much invented the modern concept of zero trust, which is basically a reverse proxy that looks a lot like a next generation firewall quite honestly.

49:01
Right? Like it's, you have a reverse proxy that's just bringing in all this information from different sources and to figure out what's going on and do access control and all of that. So they've got a really interesting and cool take on things. It looks a lot like a firewall, but maybe they don't call it a firewall. And so, you know, as far as looking at the future, I'm not quite sure, you know, what AWS has cooking with that respect with respect to that, but that's not an area where I, it's an area where I need to develop my expertise a little bit more. But I.

49:29
I agree with you, Chris. I think they're going to, they may not be successful, but they're going to spend enormous amounts of money trying. Do you think they'll end up trying to take the market share and eat the lunch of the NVA vendors? Or is it just going to be kind of where it always has been, where the CSPs are really just trying to get a leg up on each other to get that kind of good enough or hyperscaler efficient offer out there? What do you guys think? With respect to Microsoft, that's been their brand since the days of MS-DOS, right?

49:58
That's what they've always done. That's never changed. It's always been that way. When innovation comes out, they commoditize it and they kill the market for it and they just roll it in and make it a feature in one of their products. They've always done it that way. So I don't see that changing. So I'm sure that's what they're trying to do. I think the key is if they're gonna do it, they need to make it easily adjustable for the firewall admins of today, right? It's gotta be the leverage.

50:25
it's got to be able to leverage tools that they're already using, automation that they're already already using or easily translatable to something like that. Because yeah, it's like the firewall vendors are where they are because they do it so well right now. Right? So the, the CSPs definitely need to play catch up. Obviously they are doing great in terms of innovation and a lot of spaces. But this one, I mean, you're going to have to convince some of the curmudgeons of the world that, that they should switch. Right? So it's, it's, it's not going to be

50:56
easy to do unless you make it as easy as possible for them. So I got one thing to say real quick is I can't believe we made it this far without mentioning Zero Trust. And the second is I actually see this different than all you guys. So I see the CSPs coming out with a lot of like the service mesh style, doing security through like a more service mesh style, right?

51:22
not necessarily competing with the firewall vendors, because a lot of the conversations from the app side of things seems to be more policy-based security, like using MTLS between different parts of the application. And I don't think the CSPs are interested in competing with the next-gen firewall vendors. I think they're just going to continue to build out these more server-smash style things. I mean, that's what Istio, like Google has Istio, and that they're making a lot

51:52
of updates to Istio all the time. And there's just a lot of push in that direction right now in the app space. And to me, it seems like the app space is really what drives a lot of this stuff most of the time. So you think the app space is gonna get most of the security kind of innovations? Yeah, it's not that like north south firewalls, I think aren't going anywhere. I just think a lot of the east west kind of traffic is being collapsed into more of these, more policy based security.

52:22
than traditional, the way it's done normally with traditional networking. But I mean, this is my take. The CSPs get to leverage as their built-in identity component, right? They can just kind of shoehorn it in, right? And if it all happens under the hood, kind of the same way we were talking about with Gateway Load Balancer, if a lot of it is automated for you and you can add that trust component to it, then yeah, I mean, that's, so I guess I should say, I don't think they're not gonna, you know.

52:50
I think they're going to make more money off service mesh type integrations. I just think they're going to do both because they have the money to do both. Right. So, yeah, but yeah, that's a great point, Alex. I mean, and think about for their managed service offers also, they'll be able to bake the security right into those managed service offers. Like you were talking about Alex. That's a, that's a great point. Yeah. What's crazy about those is, um, you know, like AWS is the first one to announce this and I can't believe this hasn't picked up more traction, but the VPC lattice thing that they announced at re-invent is.

53:20
It's a service mesh for Kubernetes infrastructure. It blows my mind when someone's talking about this, but it's all AWS products. Like every CSP is gonna do this because they can integrate so many different products that they sell. And that is, I'm sure it's ridiculously expensive. Yeah, that's so great. We should do, we should explore that in a future episode because I agree, people aren't really talking about it. I think maybe the problem with people talking about it or not talking about it is that nobody can quite figure out.

53:48
It's like one of those revolutionary change the game kind of like change the whole architecture kind of things that people announce and everybody's just kind of like scratching their head like what I don't even know like how to leverage this right now and definitely bear some exploration. I don't know Steve have you looked at any of this stuff? Not as much as Alex now I'm some I'm like I'm definitely going to be reading up on on Istio after this podcast. So I think it's interesting. I mean I've got it. I've got it.

54:15
I learned enough about Kubernetes to know what I didn't know, and I was fascinated by its architecture and how it takes a lot of the things that I associate with traditional network infrastructure and just automates it for you. And service mesh just seems to be really a continuation of that, right? Where it's just this programmatic automation of functionality. So yeah, it's really interesting. I would love to learn more about it. Awesome. We should definitely do an episode on service mesh. It's a really interesting topic.

54:44
I think a lot of networking people would grasp pretty intuitively if it was broken down into like actual basics and given a more networking kind of focus. Yep, absolutely. Well, I know we're coming up on an hour here and my cat is very much telling me she would like to leave. So, I don't know, is there any kind of final thoughts that you'd like to leave us with Steve and everybody else? We could start with Steve and we can close it out here.

55:13
Yeah, for me, you know, as a practitioner, the cloud service providers are making things fun. You know, I'm loving all of these innovations and the ability to do like cool stuff that would be like a huge pain in the ass to try and build in your own lab, but you can just go in there and click a few buttons and play around. It's amazing. What a time to live. Oh, I wanted to ask before you guys weigh in, I wanted to ask Steve, where can people find you? Oh.

55:41
Well, I have a blog which I haven't updated in a while, embarrassingly, which is, you know, densemode.com. And I'm sort of on Twitter. I haven't been active on there lately. I've just been kind of busy, but I'm at densemode with zero for him. But thanks for asking that, Tim. Appreciate it. All right, Alex, any closing thoughts? No, I mean, I love that Steve's closing thought was about how fun the cloud is because I, it really is. Like, it's, I think all of us are in agreement that.

56:09
It is quite the time to be alive and it's really brought a lot of joy back into the engineering and design. Oh, yeah Big time connecting things, you know, so I'm just really glad he called that out and I think that's awesome and definitely agree Well you Chris. Yeah wholeheartedly. Yeah, I agree. That's very fun. But I think One thing that we probably kind of covered in this episode that like security also needs to be a major component whenever you're you're learning the cloud and

56:36
And you probably got to think about it in a slightly different way than you were previous to this. But yeah, I mean, and you know, I'm really excited to see how the security space evolves for the CSPs. Cause that's obviously, you know, everyone's one, you know, what is, what's the saying? Everyone's one, one breach away from the, whatever it is. I can't remember what it is right now, but yeah, so it's, it's obviously a very, very important element.

57:03
to consider and I'm excited to see what they're going to do. So yeah, what about you Tim? No I have to echo all of that. I think security has to be one of the biggest things we talk about, especially, I mean we're talking about public cloud here too, right? So it's public cloud, it's not your data center, it doesn't have the same castles and walls. We've got workloads that are sitting right next to the internet, you know, in AWS Azure they've all got basically the ability for your workloads to go straight to the internet.

57:31
It's a different world and so it requires a different thinking about security. I think we'll close it out there. Thanks again, Steve, for joining us. I'm really glad we could have you on and I'm sure we'll have you back in the future. That's a wrap for the podcast. If you'd like to follow us on social media, it's on Twitter at Cable's 2 Clouds with the two, the number two, and our website, cablestoclouds.com. We have a YouTube channel as well, so you can find us there. Thanks for tuning in.

58:02
Hi everyone, it's Chris and this has been the Cables to Clouds podcast. Thanks for tuning in today. If you enjoyed our show, please subscribe to us in your favorite podcatcher as well as subscribe and turn on notifications for our YouTube channel to be notified of all our new episodes. Follow us on socials at Cables to Clouds. You can also visit our website for all of the show notes at cables to clouds.com. Thanks again for listening and see you next time.