DevOps Topeaks

#19 - IAM: Identity and Access Management

• Omer & Meir • Season 1 • Episode 19

Share episode

Send us a text

This week we discussed AWS IAM, on the infrastructure level, application level, what are users, roles, profiles, permission sets, temporary credentials and MORE!

Links promised:

  • SSO Sync for Google to AWS SSO: https://github.com/awslabs/ssosync

Tools / Experience of the week:

  • Omer mentioned "nocode": https://github.com/kelseyhightower/nocode

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops

I am ready and action action action welcome to DevOps topics I missed the process again fuck can I say fuck I can say fuck oh my god don't say don't say it's episode number 19 19 yes whoa that's every week it's like wow we've come this far okay so this week we are going to talk about talking of the week with the effect I am I am I am for identity access management exactly to anyone listening we did not practice that not even once no one will coordinate it and so so this is the today's topic this week's topic this episode topic and the email the first question as always what is it going to be what's the first thing that comes up to your mind when I say I am the most dangerous place on AWS by I can say fuck so not fucking far by far most dangerous place on AWS and I'm not saying that because that's the I don't know easiest place to hack into or whatever maybe it is sounds like it sounds like it but I think that is the most that's the place with with the most issues that are relating to security that's the most I think issues you have when you start to work with SOC 2 things like that start to work with all kinds of certificates you're trying to pass in the organization and basically the place I will I can't say focus most on my time on but I focus a lot on of effort of keeping it tidy and clean I'll just say two more sentences because that's a lot for just one question but it's the place where I manage profiles for users where I manage roles for machines instances stuff like that it's the place where you I don't know distribute credentials from also to users machine etc etc it's the place where you manage things like list privilege access when you try to prevent people from doing things they shouldn't either users or machines things like that and I started with talking about security because that's essentially where you can gain access into an AWS account you can you can perform things like privilege escalation which means I can take one role maybe that I managed to hack into an instance and then use that role to do certain stuff maybe that role doesn't have all the permission to do certain things that are looking to do for example delete an EC2 delete an ECS cluster but maybe I can assume a role and that happens many times maybe assume a role that's a specific action that I can do in the AWS and that means that with a specific set of credentials I can take on myself another role and then use new credentials and from there I can move laterally within the organization or upscale my own credentials and do certain stuff it's it's a lot it's a big topic it's a big topic yes okay okay I want to do okay I want to divide it now I feel that the need to divide it so as a beginner I think I remember as a beginner when I first started handling I am you know I think the the greatest issue that I had was realizing which is which so let's say I have my application and we'll talk about maybe two types of application the one that you also perform India and is your daily basis like this this application okay like an application of a cloud that works with a cloud provider and maybe my application is some I don't know some static website that takes information about soccer soccer results and has nothing to do with the cloud okay so two different applications one doesn't deal with AWS credentials or AWS at all any cloud provider and the other one I don't know it's like it's like this they said you know provide some cloud services so I think most of it is just assuming most of the listeners users all those who are developing you know regular applications not related to AWS any kind of application yeah and then the IAM is a bit confusing because you have credentials for AWS and then you have commissions in your application right so I think at the beginning I want us to make this like clear cut well my application stops and where IAM and AWS begins you know when it comes to roles and credentials right so I say let's do it like that I'll say the app and you say AWS okay so I'll start with the app if you don't have all those and permissions and log in okay users and base users like roles for your application so that say application has view mode edit mode administrator power user or anything like to your application to your soccer API football API application whatever you need to create your own user base maybe with some identity cloud provider like octa or maybe AWS cognitive to manage the user pool right this is for your application in the roles user actually do with the browser or maybe API course to your application and then putting that aside what is IAM OML so this is my roles and permissions for my application so where did the IAM comes into the application or maybe it's not even related to that so you added a different layer or aspect to the topic which I didn't even think about and it's totally relevant which is how I manage identity and access management for applications maybe I don't care about the permissions for AWS I don't care about being able to query in S3 bucket I do very much care about how my users are authenticating with the system and what they get to do within the system so to be honest I didn't have anything about that in mind yes you can use services like or zero and cognitive was it within AWS you asked a few things about IAM and when I speak about IAM specifically here and now I'm talking about AWS IAM which is the identity and access management of and this is your focus we will focus on that just saying some listeners are like well what's the difference I mean I have my roles and the AWS roles how do I separate this perfect so I think we can focus on first explaining what AWS IAM is which I did start to do and then everything that's not that and is related to your application isn't part of this when you manage things within your application yes you can use AWS services like we mentioned don't know what's the vast majority of users are doing but we're speaking about solely the access of developers and applications or resources that are running in the cloud to do stuff in the cloud again like if I have a process that runs within an easy to server and I need to query in history bucket or maybe I'm using some kind of queue that's actually an SQS queue that's a service by AWS I need to read stuff from that in order to read stuff from an AWS resource I need to have application needs to do that exactly and I okay that's that's a good point my user access my application now the application needs to do something to provide service for that user maybe that user needs to read something data from a pick my application and that's stored in history within a bucket that bucket is private the only entity that has access to that bucket would be the easy to server that has a role in the past I think roles were not that known about or used and we use the set of credentials we just created another duplication of credentials as in a user in the cloud and we provided that you can of course do that and AWS what do they call access key ID and then a secret access key once you have a set of these you can use that within an application many problems to that first it's not rotating if you don't do that manually they don't rotate it's not temporary it's constant and other problem you obviously if you're using it in one place it can leak to another place so again the temporary issue thing and for that you have roles roles are kind of those profiles you attach to a certain resource be that container a cluster a node and easy to instance and then once I have that role I don't have to use a set of credentials because by definition when I have a role I can access things on AWS does that make sense yes so let's talk about a full flow I think we talked about a lot of theory I say let's talk a flow okay you talk let's take your example we have an application this applications stores files for example public files private files and I want my user when they go to my website omel.com they enter the now they want to enter to omel.com slash private once they are there they need to get a static file in front of their face I love to manage static files in static split in and basically in S3 okay in Amazon store itself how do I so what's the flow from the moment the user sends the application maybe even logs in to the application to the moment they get a response from AWS and let's leave out all the DNS query you know just talking about the credit it's a good interview question yeah something like that you know okay so let's not go too deep into the application logic because it can be development in many ways but basically right the basic stuff so like you said a user reaches out to the application wherever is coming through yes DNS routed through some kind of a load balancer hopefully then reaches the back end I don't know where your authentication mechanism sits it can sit on top before the load balancer it can and then you authenticate with a service like octa of zero etc more committed by AWS or maybe that's so now I'm authenticated and I also like the user also has a bell talking with the portal authorization right right so that would usually sit in a cookie within your browser and then my backend gets the cookie probably reads it with the key understand that this user is valid because the cookie is valid he's authenticated and then maybe I manage different sets of permissions within my application maybe that user is an admin a normal user an organization manager whatever that's an application logic that I don't manage as the one that develops infrastructure in AWS that's for the developer to set but this user has a certain set of permissions within the application and then the application makes the logic within it to decide whether that user can access the data or not assuming it can now my application needs to reach somewhere and grab that information I think nine out of 10 times that would be a database the database by the way can be a service on AWS for example if I'm using DynamoDB as my database that would be a service you can only access that through API calls that you do with a set of permissions and now again regardless whether that would be Dynamo or S3 and just reading a text file I need to do that with a set of permissions and usually hopefully those set of permissions would be the role that attached to me me can be a pod on EKS it can be an ECS role it can be an easy to instance profile which is just another type of role that's attached to the machine and then I grab the file I send it back the user gets it okay yeah I like it let's dive even deeper okay now yeah so that was the basics it would like the very high and middle level I'd say let's dive into the ugly stuff all right when I say ugly I mean like web identity token like when you're doing Kubernetes all that I think it's going to be cool to say a few words about that you know what I mean I'm not sure then maybe you go ahead okay like okay so for example in Kubernetes so you said the user authenticated authorized maybe that mean maybe it's a viewer whatever now it reaches the back end so maybe if the back end is a container in Kubernetes right Kubernetes cluster which sits in AWS AWS EKS whatever no matter how you manage your Kubernetes cluster you can use I am all and attach it to the container so the container assumes that role with the web identity token you know that's a assume all in Kubernetes yeah right which is cool so the application itself goes and search searches from the environment variable AWS web identity token it has a specific path that is mounted to the container this path contains the temporal credentials that the application can use to assume a role in AWS and maybe fetch a file from S3 maybe get something from DynamoDB whatever according to the role now I'm thinking out loud OML I talked about two different types of roles and I hope I'm not you know lend mining myself let's say I have an admin user and if you will you know two sets of permissions one is view only and the other is view and edit right my application obviously can do everything you know view editor delete whatever so how do you like when you have a set of usual permissions so this is going to be like the connection between user permissions to AWS permissions if you think about it because I want admin users only to be able to you and edit and maybe to view else to only to view so it's two different roles so I don't think I'm going to hold two different containers so each container has different permissions so do you have any idea how to attack this specific thing I don't have any other idea but to stop it on the application level I would stop it there because I would usually for example nest which you've used nest provides that functionality exactly it lets you set different set of roles it not permissions but roles within the application and then based on that role you can then decide within the application logic whether this user can do something or can't for example he's an admin he's a manager he's a user he's a privileged user and then based off a set of roles that I've defined within the application I can then be I'm the backend and I now makes the decision whether this user can read a certain type of information for example information about the organization that I'm part of for example a certain data that maybe stored somewhere else on s3 dynamo like we talked before but that would be within the application and then assuming that any type of user would access this resource like you said we won't use two different or a set of resources for different type of roles this certain resource sorry has the entire set of permission that it would ever need to perform any kind of task and then the block edge would be within the application logic it's you have no idea how interesting it is what you just said because I think it's like it's hard you know it's like it's very I'm a bit surprised because on the other one one hand we always like to say list privilege and don't let something access it's not meant to be right it's not meant on the other hand you're saying the obvious I think that my application has to have all the access it needs to perform update delete whatever I'm not going to hold you know create instances of my application according to permissions you know that would be crazy so a single container can do any anything but you're saying I'm going to block the the API access to AWS maybe according to that it still sounds like a security risk I don't know why right because application can do everything and then okay so we're getting into application security but yes basically this what you're saying here is I'm assuming my application is going to be hacked and I want to stop it on the next level which is an AWS type of access and for that you can do certain things and add layers of security but what I want to say here if you if you don't trust your application security what you need to fix is your application security rather than the mechanisms on top of that that said there are additional solutions you can for example you mentioned something within an EKS cluster we talked about temporary credentials so let me branch out from this conversation to there what I wanted to say is you can put something like a vault server within the EKS cluster and then once I have a vault service I can talk to the vault service to obtain a set of credentials for AWS the nice thing about it that the vault's in the back end it's in its own back end has a set of permissions or a role that it can communicate with AWS I am in provision with using a service that's called STS simple token service I think on AWS and with that so you provide a temporary token and with with a temporary set of credentials so you would have the same AWS access KAD you would have the same AWS secret access key and you would now have a third addition additional string that would be AWS session ID or something it's a session session token I think back in exact day to with security token I remember I think they changed it something like that anyway that would define whether the set of credentials is valid and valid is not only whether it's correct it's also whether the time period that was extended to it is still relevant so I can generate a token for example with vault for an hour you can tell me what is vault you say vault what is vault okay vault is a secret management by HashiCorp the normal usage for the standard usage for it is to store secret data and then applications that need secret data for example passwords for databases and creates certain set of credentials regardless whether that AWS or anything else you can store there and then when an application needs it it communicates with vault within with based on a token or the I am role that's attached to that certain entity and then what decides is it relevant or not is it valid or not can this role read what it's asking for example is it actually running on production environment and is it this type of application and does this type of application can read the data that it's asking for and if it is I can unlock the data decrypted or send it back and it will be decrypted that's fault what I did want to say I want to take it to the temporary access that you mentioned so we talked about having a temporary access for applications the same thing exactly can be done for users yes you can have the user communicate with vault every time they want access AWS there's a better solution AWS provision or it sounds like a please say to what you said exactly there's a solution yeah I know something you don't yeah so AWS they're kind of encouraging to to migrate users from standalone users within AWS I am to what they now call the I am management center or whatever it's used to be called SSO essentially what this means is that instead of managing certain users and their credentials like specifically manually standalone as a user you now migrate them into SSO you can provision them manually but SSO is easily my great sorry integrateable with active directory or Google directory or Okta any kind of service that manages users and then once that's managed and sync you can configure different set of groups and users and when a user is logging in through SSO to AWS they get a temporary set of credentials so they have the server just to focus when you say SSO it's fault now we move the way from the application for I'd say um the customers okay and now we're talking about SSO of internal developers in the company okay so this is for internal services yeah right internal users and when a user wants to access something the reason a user would want to access something is for example to read this data from S3 or to upload something from S3 from his computer hopefully it's not to do stuff in production other than read them but yes you do need to interact both of them want to download S3 artifacts that the developer has uploaded from a CICP process not only that they want to access the console and start a new easy to server to test something or to do something I need to have permissions for that so when you log into the AC2 dashboard based on who you are as a user you authenticated in some kind of way Google Okta out zero wherever you're now authenticated and you have a certain set of permissions that was configured on SSO and then you have two things you can do with these set of credentials you can log into the console and do stuff or you can obtain those three uh those three sets of permission credentials that we discussed you can copy paste them use them within your console within your application and run them and you can configure for how long can they be provisioned for between something one hour and 12 hours I think it's the max and then once you work like that there's no longer the problem or the issue of credentials that were provisioned two years ago and nobody knows where they've licked and you can you only stop at 12 hours so it's constantly rotating and then you upgrade your security yeah also add that when you do because you the SSO so if someone let's say we use Google you know the google suite so if someone leave a google workspace so if someone someone leave the organization and they try to log into AWS they are either active to Google but if the user can't log into Google because they have the company so they get the activated and then they don't have access to it again talking about internal users all right it's important because we talked a lot in the beginning about application and whatever now we're talking about developers QA or that DevOps anyone in the organization that wants to access AWS this is the talk now okay yeah and one more thing about that by the way I think we talked about it a few times but specifically because you mentioned Google directory it is what I'm working with if you want to keep the two in sync AWS have their own tool that's called SSO sync if I'm not mistaken we'll leave the link below and it's essentially a tool that you probably want to run within a lambda function or periodically from wherever and it syncs between your google directory with a set of credentials by google and then with AWS SSO and it actually can sync the groups from google to the groups on SSO so you can decide I don't know a user has joined the company you automatically put them under rnd at company.com and now you know that all the rnd can have read permissions on staging environment so we automatically obtains that set of permissions and there's no need for you to do anything else that's it yes yes and also adding that SSO is a big solution okay so if you're a startup let's say up to 10 people you'll probably will create users manually in iam and you might even give them all yes it might sound real administrative access okay so oh my let's talk about that iam going flying iam going flying over let's not have i don't know something that delays us such as permissions so if that if that's something that delays my work i'd rather give administrator access to someone well not to anyone but i'm on client with that then starting to pick each permission each policy for everything because it delays my work okay and again as a small startup i'm not going to meet with compliance and everything okay so what do you see what do you have to say about this topic about permissions and you know delaying work because i don't have permission okay first of all there are paid solutions i can think of two Israeli companies i don't want to plug them here if you google good enough you'll find them that said you're speaking about a company with 10 people i'm not sure they have the finance you know they don't have the money to pay for that kind of a service i tend to agree with you i've learned in recent years that when you start doing that and you probably are start you just starting off by providing administrative credentials if you're just one two three four five maybe up to 10 people it quickly gets out of hand for two reasons first of all you may potentially you want to grow and if you grow all of a sudden from 10 people to i don't know 50 and you kind of you're probably even even the oil in 20 it might get crazy regardless of how many people it is each additional developer with access to AWS is an exponential set of problems that you create imagine that administrator can do almost anything he can add accounts to the organizations he can remove them he can read the bill he can destroy stuff provision stuff delete his food traps anything so i tend to i don't know where to put the line one two three people i think from day one i go maybe not automate the entire process and you know the shenanigans of at least privilege access i would create at least a set of credentials it doesn't have to be only the very limited set that they need but you know at least at least at least don't put administrator use power user access it's an AWS i AWS permission set that gives you everything aside from i am as long as you don't have as long as you don't have i am access you can't change your roles because once you have i am access you have everything you're basically root regardless of who you are because you can assume the role of the administrator but you just said the solution on to but what happens you know if all hell breaks loose you just said the solution root so explain the solution what happens if somebody starts getting crazy in the account how can i how can i get back you know access to my account how can i recover stock maybe how can i do there is okay so you said the solution there is a root user for any account that's basically a user and password that created the account hopefully this is not used at all at all ever unless this is why you should use root you know yes that and in any case yes and hopefully once you have a security systems in place you should be alerted when the root user is actually doing something you don't want to use it daily even if you're at the city or the city or the company use your even if it's your own private account by the way i have my own private account i don't use the root credentials i've created and i'm user for me to be alerted when something goes wrong it has an MFA on top of it and it's locked i'm not touching it i don't know the password MFA what is MFA on it right okay that's good that's a good question MFA MFA is a multi-factor authentication uh regardless of i am it's relevant to any security system when you what is to a fake i held MFA i held MFA it's a two-step authentication basically okay what it means is beyond the username and a password adds something else because a password can be leaked i might have sent it accidentally by a WhatsApp message or an SMS or leaked somewhere on Slack as long as i have something that protects me beyond that i'm kind of safe or safer than only having a password for example if i'm using to my Gmail account and i provided a username and a password and that password got leaked if the user if i've configured multi-factor authentication with my phone and the hacker or whoever got the leak is now trying to use it and they don't have my phone they can't access my my account right which brings me to another set of things hopefully are done within companies how do how do you set up MFA on your phone i mean not it's like how do we which apps do you know that can do that like how does it work can you explain maybe like a one one minute about the process of MFA like how to configure it and then use it yeah there are a few protocols i don't remember all of them all WhatsApp is the most general one uh forgot what it is sorry um essentially if you try to do it within just i am if you go to AWS i'm an ad multi-factor authentication you get this nice QR code that you can scan with an app on your phone the most used one is probably Google authenticator uh you can add it as a Chrome extension that can scan it i personally use one password it's a password manager private user i do yeah and it just fills it out on its own if you don't have to do anything it scans the QR it starts it it doesn't anything for you uh and then you have this uh another level of protection for us for example because i'm syncing Google directory to AWS and that's the way for me to control what users can access and how they access we enforce to factor authentication multi-factor authentication don't call it wherever you want uh on our Google users you cannot be part of the organization if you don't have multi-factor authentication set on your uh on your user in the company that's MFA thank you for the MFA yeah of course there's one more subject that you mentioned in this in the beginning of this episode which i think is important to touch you spoke about other services we consume as a company sometimes with a role for example maybe tomorrow i'm starting to use some kind of security service we have probably four or five names in our head right now and they can do one of two things they can either tell you okay get me a sense of credentials get me the access id and the secret access id that's probably a bad sign that means they're not used to modern work with AWS the better sign is when a company says okay this is the permission set or the iam policy that i want to attach to a role give me those permissions how do you do that you take the policy you attach it you create a role you attach a policy and when you configure the role you can say this role is for a third-party access third-party access would mean that this role will be used by a different AWS account probably it is something that is exactly it would be an external account that can control my account using those set of permissions back to MFA within that you can add another layer that's called an external id that's another type of security layer on top of that that's a basically a password so you can assume the role only if you're coming from the allowed account id but you need to know the password when you provide all of this you get a set of temporary credentials which you can use so if that's a security service that's what they do they'd assume a role they have an aim for that for the external id you know why they so that's like will be the last thing about the topic and then we move to the corner but they have an aim for the confused sheriff confused deputy you know you know this you remember i remember i remember i just the name rings a bell but i don't remember why the confused deputy is still in case so i think the external id with dimensions so again if they're third-party company wants to access your AWS account and they tell you give me i am user credentials so probably not doing it right but if they do create a user you create an i am user for them and grant the permission with external id maybe you have two AWS accounts one is staging one is production and i think the confused deputy is like you want to make sure that every time that they log in or use an account they know which account they're using so it's not like they're gonna do production stuff in staging or staging stuff in production you know very interesting we'll add it to the description there i need to google that i remember i read it about it the external id is for the confused deputy or something like that no shoot no okay okay i'm in 32 minutes oh my god 32 minutes okay so i'm going to go to the corner of the week because it's gonna be the almost the longest episode that we had we'll break right so records as we go yes amazing i am we can still talk about it more and more and more okay so corner of the week oh man i have nothing on me because i'm still doing the same thing so no breakthrough anything on your part yes uh do you know what date is today uh it doesn't even notice may it come on yeah not funny no that's uh april force for thinking that i didn't notice it it's april is that so today is april first and i have something for that um you know who kelsey hightower is you're not on twitter you don't know okay kelsey hightower is uh this google engineer or google senior uh advocate for ek or for gKE probably on google but is a really cool person to follow in the industry talking about golang uh kubernetes stuff like that anyway kelsey had pushed a repo that's called no code six years ago yeah six years ago uh this no code basically has nothing in it it has an empty docker file an empty rid of me almost empty rid me an empty license it has a style guide and it basically has nothing and the description is it's the best way to write secure and reliable applications just write nothing deploy nowhere okay so basically the best way to do things is just not do anything and that's how you have the best code in the world this repo has 56 000 stars it has been forked almost 5 000 times it has 460 pull requests and 4 000 issues i urge you to open the link go to the issues if you want to left for an hour or read the comments on the style guide that it pushed it's the funniest thing on earth people ask if it's working on windows uh why is it not compiling on lead knocks they didn't did not understand part of the example etc etc you have to read wow thank you for sharing that okay i actually love this even i didn't even know it now i know it i will get about it thank you for that wow that's it does it all Conlinos yeah wow amazing okay so this is for the week for this week we had the corner where everything uh almost even next week see you thank you for everyone who attended this meeting thank you cloud thank you crowd bye bye