Models

• RDBMS
  • Relational database management systems (RDBMS) data has formal and fixed relationships
    • data stored in rows -> individual attributes
    • Tables have schemas (define row layout)
  • RDBMS conforms ACID system (Atomicity, Consistency, Isolation, Durability)
    • High performance
    • Low scalability
  • Structured Query Language (SQL) is used for RDBMS
• Non-relational: No-SQL (e.g. for social media. data warehousing, analytics)
  • Elements
    • Key/value: fast queries, no relationships
    • Document: structure of key/value pairs. Operations are highly performant
    • Column: data is stored in columns rather than rows (Amazon Redshift)
    • Graph: designed for dynamic relationships. Data = nodes (Neo4j)

SQL — RDS

Definition

• RDS = Database as a Service (DBaS) -> fully functional DB without admin overhead
  • performs ar sclae
  • can be publicly accesible
  • can be configured for demanding availability and durability scenarios
• Engines
  • MySQL
  • PostgreSQL
  • MariaDB
  • Oracle
  • Microsoft SQL

graph LR

A(DB instance CNAME)

subgraph VPC1-region1
  B[Standby - DB storage -AZ1]
  C[Primary- DB storage -AZ2]
end

D[S3]

subgraph VPC2-region2
  E[Read replica - DB -AZ1]
end

A --> C;
B --> C;
C --> B;
B --> D;
C --> E;

• 1 or more AZs for resilliance
  • general purpose (DB.M4, DB.M5)
  • memory optimized (DB.R4, DB.R5, Oracle DB.X1 amd DB.X1e)
  • burstable (DB.T2 and DB.T3)
• Storage types
  • generally purpose SSD (gp2)
  • provisione IOPS SSD (io1): indepndent configuration
• Billing based on
  • instance size
  • provisioned storage (not used)
  • IOPS if using io1
  • Data transferred out
  • Any backups/snapshots beyond the 100% that is free with each DB instance
• RDS supports encryption with limitations
  • configured when creating DB instances
  • added by taking snapshots, or creating new instance from encrypted snapshot
  • encryption can not be removed
  • read replicas need to be the same state as theprimary instance (encrypted or not)
  • encrypted snapshots can be copied between regions, but a new destination region KMS CMK is used (as they are region specific)
• Network access to an RDS instance is controlled by a security group (SG) associated with RDS instance

Backups

• Automated backups: to S3 occur daily, retained for 0-35days
• Manual snapshots: exist until deleted
• Point in time log-based backups: stored on S3

RDS supports manual snapshot-based backups as well as automatic point-in-time recovery-capable backups with a 1- to 35-day retention period.

graph LR
A(Primary-A DB)
B(Standby FB)
C[S3]
D[S3]
E(Primary-B DB)

A -- synchronous data replication --> B;
B -- daily backup --> C;
B -- manual snapshot --> D;
D -- restore a new instance --> E;

Resiliency multi-AZ

• RDS can be provisioned in single or multi-AZ mode (standby instance in same or different) -> recover from failure
• Only primary can be accessed using instance CNAME
• No performace benefit, better RTO than storing snapshot
• Replication of data is asynchronous (copy in real time from primary to snadby)
  • Backups are taken from standby, to ensure no performance impact
  • Maintenance is performed in the standby first, then promoted to minimiza downtime

Read replicas

• Read replicas are read-only cpies of an RDS instance that can be created in the same or different region from the primary instance
  • can be adresseed independently (each having their DNS name)
  • used for read workloads, allowing scaling reads
  • 1 RDS instance -> 5 read replicas (whihc can be created from other read replicas)
  • read replicas can be promoted to primary instances and can be themselves multi-AZ
• Read replicas are eventually consistent (in seconds, but aplications need to support it)

SQL — Aurora

Essentials

• Aurora enhanced RDS, by Amazon, compatible with MySQL, PostgreSQL tools
  • base configuration = cluster
  • cluster contains a single primary instance and 0+ replicas
• Cluster storage: all instances use the same storage
  • read/write uses cluster endpoint
  • redas can use reader endpoint (balances connectionsover replicas)
  • volume = SSD based, can scale automatically up to 64TB, bill only consumed data
  • replicates data 6 times, across 3 AZs -> improve availability, be promoted to primary instance quickly
  • Aurora can tolerate 2 failures without writes being impacted and 3 failures without impacting reads
  • Aurora storage is auto-healing
  • Tier 0 has the highest priority in an Aurora failover
• Backtrack feature
  • allows you to roll back a database for up to 72 hours
  • yo don’t have to make a new cluster when using Aurora’s backtrack feature to restore a database.

Parallel queries and global

• Paralell queries: executed across all the nodes of the cluster at the same time
  • Activated when created the Aurora cluster
• Global: Aurora provisioning option which adds resiliency by allowing you to pick amongst all AWS regions as your secondary reader cluster
  • 1 primary region, 2n region for read workloads -> low latency
  • Activated when created the Aurora cluster, only for some verions

Serverless

• Aurora Serverless: based on the same DB engine as Aurora but wothout resource allocation
  • specify a minimum and maximum number of Aurora Capacity Units (ACUs) - measurement for processing (compute) and memory in Aurora Serverless.
  • can use the Data API to connect to it
  • billing: charges are based on database resources used per second
  • master exists in one Availability Zone
  • capable of rapid scaling because it uses proxy fleets to route the workload to “warm” resources that are always ready to service requests.
  • maximum amount of replicas: 15
  • when to use it
    • should be used when workloads are intermittent and unpredictable
    • slower failover time than Aurora Provisioned

graph LR

A[Applications]
B(Proxy fleet)
C[Aurora Serverless cluster DBs]
D[Pool DBs]

A --> B;
B --> C;
C --> D;

• Proxy Fleet: fleet of proxy instances who route an application’s query to a group of automatically scalable resources
• Query editor: web-based tool that allows you to log in to the Aurora Serverless cluster and executes queries

NoSQL: DynamoDB

Essentials

• Elements

  • DynamoDB: NoSQL DB service, 3 replicas of data
  • Table: collection of items that share teh same partition key (PK) or partition+sort key (SK) with other configuration and performance settings
  • Item: collections of attributes (up to 400KB in size) inside a table that shares the same key structure as every other in the table
  • Attribute: key-value pair

• Query: filter based on PK or SK, efficient

• Scan: checks all items, not efficient

  • Filters: applied to scan

Performance

• 2 read/write capacity modes

  • provisioned throughput (default)
    • each table is configured with Read Capacity Units (RCU) and Write Capacity Units (WCU)
    • every operation on Items
  • on-demand mode (automatically consumes at least 1 RCU or WCU - partial RCU/WCU cannot be consumed scales to handle performance demands)

• Consistency

  • 200 status code = write has been completed and is durable
  • ensures Dynamo DB returns the most up-to-date copy of data
  • Reads from leader node to use strongly consistent reads

• Capacity modes

  • On-demand
  • Provisioned
  • Provisioned with Auto Scaling

• Capacity Units How-To Guide – How to Calculate Read and Write Capacity for DynamoDB – Linux Academy

  • Read Capacity Units (RCU)
    • (ITEM SIZE (rounded up to the next 4KB multiplier) / 4KB) * # of items (Round up to the nearest 4 KB multiplier)
    • 4 KB of data read from a table per second in a strongly consistent way
    • Read 2kKB consumes 1 RCU
    • if eventually consistent reads are OK, 1 RCU can allow for 2x4 KB of data reads per second
    • atomic transactions require 2x the RCU to complete
  • Write Capacity Units (WCU)
    • (ITEM SIZE (rounded up to the next 1KB multiplier) / 1KB) * # of items (Round up to the nearest 1 KB multiplier)
    • 1 WCU is 1 KB of data or less written to a table
    • Write 200 bytes consumes 1 WCU
    • atomic transactions require 2x the WCU to complete

Elements

• Streams: provide ordered list of changes that occur to items with a DynamoDB table

  • rolls 24 hour window of changes
  • enabled per table (only data from the point of being enabled)
  • has a ARN that identifies it globally across all tables, accounts and regions
  • View types
    • KEYS_ONLY: whenever an item is added, updated or deleted, the keys of the item are added to the strea,
    • NEW_IMAGE: the entire item is added to the stream “post-change”
    • OLD_IMAGE: the entire item is added to the stream “pre-change”
    • NEW_AND_OLD_IMAGES: both the new and old versions of the item are added to the stream

• Triggers:

  • steams can be integrated with AWS lambda, invoking a function whenever items are changed in a DynamoDB table (a DB ttrigger)

graph LR

A[Terminal]
B[Dynamo DB table]
C[Dynamo DB Stream Records]
D((AWS lambda))

A --> B;
B --> C;
C --> D;

• Indexes: provide n alternative represntation of data in a table, which is useful for applications with varying query demansa
  • 2 forms
    • Local Secoundary Indexes (LSI)
      • created at the same time as the table
      • same PK, as alternative SK
      • share the RCU and WCU values for the main table
      • maximum: 5
    • Global Secoundary Indexes (GSI)
      • created after table was created, asynchronous data from the table
      • different PK and SK
      • have their own RCU and WCU values
      • maximum amount (without logging a support ticket) of GSIs per table: 30
  • interated with as though they are table (alternative representation of it)

In-memory caching

• DynamoDB Accelerator (DAX): in-memory cache designed specifically ofr DynamoDB

  • results delivered from DAX are available in microseconds rather than in the single-digit milliseconds available from DynamoDB
  • can use a cluster architecture, run inside VPC, applications use a DAX client
  • 2 distinct caches
    • item cache
      • stores results from GetItem and BatchGetItem
      • has a 5-minutes default TTL
    • query cache
      • stores results from Query and Scan
      • caches based on the parameters specified

• ElastiCache: managed in-memory data store supporting the Redis or Memcached engines, for lyarge sets of data with repeated read patterns

  • 2 use cases
    • offloading database
      • reads by caching responses
      • improving application speed and reducing costs
    • storing user session database
      • allowing for stateless compute instances (used for fault tolerant architectures)
  • is used with key-value databases or to store simple session data, but it can be used with SQL database engines

S3 architecture

Permissions

• Bucket authorization within S3 is controlled using
  • identity policies on AWS identities
  • bucket policies in the form of resource policies on the bucket
  • bucket or object ACLs
• Final authorization is acombination of all applicable policies
  • Priority order
    1. explicit deny
    2. explicit allow
    3. implicit deny

Transferring data to S3

• Uploads to S3 are generally done using the S3 console, the CLI or the APIs
• Uploads either use a single operation (known as a single PUT upload) or multipart upload
• Single PUT upload
  • Object is uploaded in a single stream of data
  • Limit of 5 GB -> can cause performance issues, if it fails the whole upload fails
• Multipart upload
  • Object is broken up into parts (up to 10.000)
  • Each part is %MB-5GB, and the last part can be less (remaining data)
  • Faster (parallel uploads), and the individual parts can fail and be retried individually
  • AWS recommends multipart for anything over 100MB, but it is required for anything beyond 5GB

Serve content

• Static Websites

  • Amazon S3 buckets can be configured to host websites -> content can be uploaded to the bucket and when enabled, static web hosting will provide a unique endpoint URL that can be accessed by any web brwser
  • S3 can be used to host front-end code for serveless applications or an offload location for static content
  • Cloudfront can also be added to improve the speed and efficiency of content delivery for global users or to add SSL for custom domains
  • Route53 and alias records can also be used t add human-friendly names to buckets
  • bucket policy example (trailing /* -> applies policy to all objects in bucket)

    1 2 3 4 5 6 7 8 9 10 11

    { "Version":"2012-10-17", "Statement":[{ "Sid":"PublicReadGetObject", "Effect":"Allow", "Principal": "*", "Action":["s3:GetObject"], "Resource":["arn:aws:s3:::YOUR_BUCKET_NAME/*"] } ] }

• Cross Origin Resource Sharing (CORS)

  • CORS is a security measure allowing a web application running in one domain to reference resources in another

Encryption

• Data is encrypted on transit/at rest (on per-object basis)
  • Client side
  • Server side with Customer-managed keys (SSE-C)
  • Server side with S3 managed keys (SSE-S2)
  • Server side with AWS KMS-managed keys (SSE-KMS)
• Bucket default encryption
  • objects, not buckets, are encrypted
  • each PUT operation needs to specify encyption (and type) or not
  • a bucket default captures any PU operations where no encryption method/directive is specified
  • it doesn’t enforce what type can and can’t be used. Buckets policies can enforce

Versioning

• Objects versioning: enabled on a bucket
  • once enabled, any operations that would otherwise modify objects generate new versions of that original object
  • once a bucket is version enabled, it can never be switched-off, only suspended
  • with versioning enabled, an AWS account is billed for all versions of all objects
    • objects deletion by default does not delete an object - delete marker added
    • older versions of an object can be accessed using name + verionID
    • specific veriosn can be deleted
• MFA delete: feature designed to prevent accidental deleteion of objects
  • a 1 time password is required to delete an object version or when changing the versioning state of a bucket

Presigned URLs

• Preassigned URL: can be created by an identity in AWS, providing access to an object using the creators permissions

  • When preassigned URL is used, AWs verifies the creator’s access to the object, not yours
  • The URL is encoded with authentication built in and has an expiry time
  • Preassigned URLs can be used to download or upload objects

• Any identity can create a preassigned URL - even if that identity doesn’t have access to the object

• When used preassigned URLs, you may get an error. Some common situations include

  • URL has expired - 7 days maximum
  • creator’s permissions of the URL changed
  • URL was created using a role (36-hours max)and the role’s temporary credentials have expired (aim to never create preasigned URLs using roles)

S3 performance and resilience

Resiliance = High Availability (HA)

Storage tiers/classes

Lifecycle policies and intelligent-tiering

• Lifecycle rules control storage classes, allow for the automated transition of objects between storage classes, or expiration of objects that are no lobger required
  • rules are added at bucket level
  • rules can be enabled or disabled based on business rules
  • objects can be archieved using lifecycle configurations
  • objects can be restored into S3 for temporary periods of time, after which they are deleted
  • objects encrypted remain encryted during the transitions
• INTELLIGENT_TIERING
  • Objects smaller than 128KB cannot be transitioned into it
  • Objects must be in the original storage for a minimum of 30 days before transitioning
  • At the point of expiry, ojects are deleted from the bucket

graph LR

subgraph S3
  A[Standard]
  B[Standard IA]
  C[One Zone IA]
end

D[Glacier]

A --> B;
B --> C;
C --> D;

CRR

• S3 cross region replication (CRR): on buckets, allow one way replication of data from a source bucket to a destination bucket in another region
  • Replicas keep
    • storage class
    • object name (key)
    • owner
    • object permissions
  • Replication configuration
    • applied to the source bucket
    • versioning must be enabled both on orgin and destination
    • requires an IAM role with permissions.
  • excluded from replication
    • system actions (lifecycle events)
    • any existing objects from before replication is enabled
    • SSE-C encrypted objects- only SSE-S3 and (if enabled) KMS encrypted objects are supported

CloudFront

CloudFront is a content delivery network (CDN): global cache that stores copies of your data on edge caches, which are positioned as close to your customers as possible

• lower latency
• higher transfer speeds
• reduced load on the content server

CloudFront architecture

• CloudFront components
  • origin: server/service to host content
  • distribution: configuration entity in CloudFront (CloudFront implementation)
  • edge location: local infrastructure (150 locations over 30 countries)
  • regional edge caches: larger veriosn of edge locations (more capacity, larger areas)
• Caching process
  1. create a distribution and point at one or more origins. A distribution has DNS address that is used to access it
  2. DNS address directs clients at the closes avialable edge location
  3. of edege location has cached copy of your data, it’s delivered locally from edge location
  4. if it¡s not cached, the edge location attempts to downloadit from either a regional cache or from the origin (known as an origin fetch)
  5. as the edge location receives the data, it immediatly begins forwarding it and caches it for the next visitor

graph TD

A(customer)

subgraph distribution
  B[Edge location]
  C[Regional cache]
  D[S3]
end

A -- object delivery --> B;
B --> C;
D -. origin fetch .-> B;
D -. transfer to regional cache .-> C;

OAI

• CloudFront is publicly accesible by default (anyone with the DNS endpoint address can access it
  • Distribution can be configured to be private (access requires a signed URL cookie) via trusted signers on the distributions ->can be bypassed by going straight to the origin
• Origin Access Identity (OAI) is a virtual identity that can be associated with a distribution
  • S3 bucket can then be restricted to only allow this OAI to access it, all other identities can be denied

Network File Systems

• Amazon EFS: implementation of the Network File System (NFSv4) delivered as a service. Files can be create and mounted on multiple Linux instances at the same time
  • base entity of a file system
  • accessed via mount targets
  • file system is mounted on Linux instances (the only one supported on Linux)
  • file systems are accessible from a VPC or from on-premises locations via VPN or Direct Connect

graph TD

subgraph VPC
  A[EFS]
  B(POSIX permissions)
  C[AZ-1 instances]
  D[mount target]
end

E[Direct Connect]

subgraph corpo. datacenter
  F[Server]
end

A -.- B;
A --> C;
C --> D;
E --> D;
F --> E;

• Performance modes

  • General purpose (default, 99% of needs)
  • Max I/O (larger number of instances (>100) need to access the file system)

• Throughput modes

  • Bursting throughput: 100MiB/s base burst
  • Provisioned troughput: allows control over throughput independently of file system size

• Security groups are used to control access to NFS mount targets

• EFS supports 2 storage classes (with lifecycle managament)

  • Standard
  • Infrequent access (IA)

Networking Fundamentals

OSI Model

Encapsulation model

IP Addressing Basics

• IPv4 -> communication at layer 4 and above on OSI model

  • Dotted decimal notation: 192.168,10.5

  • IPs are split in network part and node/host part, the netmask or suffix show where it happens (network part: 1s, node part = 0s)

    IP	192	168	10	5
    Binary	11000000	10101000	00001010	00000101
    Subnet	255	255	2555	0
    Prefix/24	11111111	11111111	11111111	__

• Reserved addresses

  Address (range)	Reserved for
  0.0.0.0	all IP addresses
  255.255.255.255	broadcast reveryone
  127.0.0.1	localhost
  169.254.0.1-168.254.255.254	autoconfigure with DHCP

• Classes (original, had ranges for private networking only)

  • A for large organizations

  • B for medium

  • Cfor small business

    Class	Range IP	Networks	Nodes in network
    Class A (/8)	1.0.0.0 - 126.255.255.255	126	17.777.214 (2 reserved)
    Class B (/16)	128.0.0.0 - 191.255.255.255	16.382	65.534 (2 reserved)
    Class C (/24)	192.0.0.0 - 223.255.255.255	2.097.150	254 (2 reserved)

• CIDS (Classless Inter-Domain Routing), used in IPv4 trather than class system

Subnetting

• Subnetting : process of breaking a network down into smaller subnetworks
• Adapt it to VPC, split into 2 smaller networks (increment the prefix, example …/16 can be split into to …/17 subnets)

IP Routing

• Local device-to-device communication takes place using L1 (physical) and L2 (data-link) using MAC addresses and physical 0’s and 1’s. This does not scale across LANs, so a method of network-to-network transit is needed.
• IP-routing: method of network-to-network transit is needed. IP-routing provides a scaling method, which depends on if the 2 devices are local, in a known remote network, or unknown network.

1. Local:

• IP-IP locally doesn’t use a router

• ARP translates IP addresses to MAC addresses

      graph LR
  A[Network1]
  B[Network2]
  
  A -- L3 package --> B;

2. Known:

• Use its IP mask to know if it is local, if not, use the default gateway (router), which will send it to the known network

      graph LR
  A[Network1]
  B(Router)
  C[Network2]
  
  A -- L2 connection --> B;
  B -- L2 connection --> C;
  A -- single L3 package --> C;

3. Unknown:

• If default gateway for A knows it is not local, then send it to next network

• Unchnaged packets being passed around form router to router each time using a new L” connection

      graph LR
  A[Network1]
  B(RouterA)
  C[Network2]
  D(RouterB)
  E((Internet))
  F(RouterC)
  G[Network3]
  
  A --> B;
  B --> C;
  C --> D;
  D --> E;
  E --> F;
  F --> G;

Firewalls

• Firewall: device which historically sits at different networks, and monitors traffic flowing between them

  • it is capable of reading packet data and either allow or deny traffic
  • stablish barrier between networks of different security levels

• What a data firewall can do depends on the OSI layer it operates at

  #Layer	Layer name	Actions
  3	Network	source/destination IP addresses or ranges
  4	Transport	protocol (TCP/UDP) and port numbers
  5	Session	as layer 4, but understand response traffic
  7	Application	application specifics (e.g. HTML paths, images)

Proxy Servers

• Proxy server: gateway that sists between a private and public network (e.g. the Internet)

  • needs application support, configured on the OS, web browser…

  • proxy servers can choose to pass on traffic or not based on things anetwork layer appliance can’t (username, company identity…)

        graph LR
    
    subgraph SubNet
      A[customer unauthorized]
      B[proxy server]
      C[cache]
      D[customer auhtorized]
    end
    
    subgraph PublicInternet
      E((Cloud))
      F[Server]
    end
    
    A -- deny --> B;
    B -- allow --> D;
    B --> C;
    B --> E;
    E --> B;
    E --> F;
    F --> E;

Private networking

VPCs and subnets

• Virtual Private Cloud
  • private network within AWS (“your private data center”)
  • can be configured to be public/private or mixture
  • regional (can’t span regions), highly available, can be cnnected to your datacenter or corporate metwork
  • isolated from otherVPCs by default
  • VPC and subnet: max/16, min/28
  • VPC subnets can’t span AZs (1:1 mapping)
  • certain IPs are reserved in subnets
• Region default VPC
  • required for some services, used as a default for most
  • pre-configured with all required networking/security
  • configured using a /16 CIDS block
  • a /20 public subnet in each AZ, allocating a public IP by default
  • attached internet gateway with a “main” route table sending all IPv4 traffic to the Internet gateway using a 0.0.0.0/route
  • a default DHCP option set attached
  • SG:default- all from itself, all outbound
  • NACL:default - allow all inbound and outbound
• Custom VPC

  • can be designed and configured in any valid way

  • you need to allocate IP ranges, create subnets and provision gateways and networking, as well as design and implement security

  • when you need multiple tiers or a more complex set of networking

  • best practice is not use default for most production things

    Reserved IPs	Value
    .0	Network
    .1	Router
    .2	DNS
    .3	Future
    .X	Broadcast

Routing and Internet Gateway

graph TD

subgraph us-east-1a
  A[Bastion]
  B(Gateway - publicSubnet 10.0.1.1)
end

C((Router))

subgraph us-east-1b
  D(Gateway - publicSubnet 10.0.2.1)
end

E((Internet gateway))

A --> B;
B --> C;
C --> D;
C --> E;

• VPC routing
  • every VPC has a virtual routing device called VPC router (choose default tenancy on the console)
  • it has an interface in any VPC subnet known as the “subnet+1” address (e.g. for 10.0.1.0/24 -> 10.0.1.1/32)
  • router = highly available, scalable and controls data entering and leaving the VPC
• Routes
  • a RT is a collection of routes that are used when traffic from a subnet arrives at the VPC router
  • every route table has a local route, which matches the CIDR if the VPC and lets traffic be routed between subnets
  • a route contains a destination and a target. Traffic is forwarded to the target if its destination matches the route destination
  • if multiple routes apply, the most specific is chosen (/32 before /24 before /16)
  • deafult routes (0.0.0.0 v4 and ::/0 v6) can be added that match any traffic not already matched
  • targets can be IPs or AWS networking gateways/objects
  • a subnet is a public subnet if (AND conditions)
    • it is configured to allocate public IPs
    • tha default VPC has an associated internet gateway
    • that subnet has a default route to that internet gateway

graph LR

A((some network))

subgraph public network
  B[Bastion]
end

C((private instances))
D((private instances))

A -- from trusted IPs --> B;
B -- from bastion --> C;
B -- from bastion --> D;

Bastion Host/JumpBox

• Bastion: a host that sits at the parameter of a VPC
• it functions as an entry point to the VPC for trusted admins
• allows for updates or configuration tweaks remotely while allowing the VPC to stay private and protected
• generally connected to via SSH (Linux) or RDP (Windows)
• bastion hosts must be kept updated and security hardened and audited regularly
• multifactor authentication, ID federation, and/or IP blocks

NAT, NAT Instance, and NAT Gateway

• NAT: Network Address Translation: method to remap IPs or destination IPs from packets

  • static NAT: private IP is mapped to public IP (what IGW do)
  • dynamic NAT: a range of private addresses are mapped onto 1 or more public (your home router and NAT gateways)

• Public IP addressing (EIP)

      graph LR
  
    A[Instance]
    B((NATGW))
    C((IGW))
    D(Internet site)
  
    A -- src --> B;
    B -- src natgwpriv --> C;
    C -- src natgwpub --> D;

NACLs

• Network Access Control Lists (NACL)

  • operate at layer 4 of the OSI model (TCP/UDP and below)
  • a subnet has to be associated with a NACL - either the VPC default or a custom NACL (Security Groups are for resources)
  • NACLs only impact traffic crossing the boundary of a subnet
  • NACLs are collections of rules that can explicitly allow or deny traffic based on its protocol, port range and source/destination
  • rules are processed in number order lowest first. When a match is found, that action is taken and processing stops
  • the * rule is processed last and is an implicit deny
  • NACLs have 2 sets of rules: inbound and outbound

• ephemeral ports (response traffic)

  • When a client initiates communications with a server, it is to a well-known port number on that server
  • the response is from that wel-known port to an ephemeral port on the client (the client decides the port)
  • NACLs are stateless, they have to consider both initating and response traffic - state a session-layer concept

Advanced VPC

VPC Peering

• VPC peeering: allows direct communication between VPCs

  • services can communicate iusing privaye IPs from VPC to VPC
  • VPC peers can span AW accounts and even regions (with some limitations)
  • Data is encrypted amd transists via the AWS global backbone
  • VPC peersare used to link 2 VPCs at layer 3: company mergers, shared services, company and vendor, auditing

• Important limits and considerations

  • VPC CIDR blocks cannot overlap
  • VPC peers connect 2 VPCs - routing is not transitive
  • Routes are required at both sides, so update the Route Tables (remote CIDR -> peer connection)
  • NACL and SGs can be used to control access
  • SGs can be referenced but not* cross-region
  • IPv6 support is not available cross region
  • DNS resolution to private IPs can be enabled, but it’s a setting needed at both sides

• ⚠️ Transistive routing

      graph LR
  
    subgraph VPC A
      A[Instance 10.0.1.10/32]
    end
  
    B((peering connection))
  
    subgraph VPC B
      C[Instance 10.1.10.27/32]
    end
  
    A --> B;
    B --> C;

VPC Endpoints

• VPC endpoints: gateway objects created within a VPC. They can be used t connect to AWS public services without the need for the VPC to have an attached Internet gateway and be public

  • Types
    • gateway endpoints: for DynamoDB and S3
    • interface endpoints: for everything else
  • When to use them
    • entire private VPC (no IGW)
    • an specific instance has no public IP/NATGW and needs to access public services
    • access resources restricted to specific VPCs or endpoints (private S3 bucket)

• Limitations and considerations

  • gateway endpoints are used vua route table entries - they are gateway devices. Prefix lists for a service are used in the destination field with the gateway as the target
  • gateway endpoints can be restricted via policies
  • gateway endpoints are HA across AZ in a region
  • interface endpoints are interfaces in a specific subnet (may override DNSwith private DNS, powered by AWs PrivateLink). For HA you need to add multiple AZ
  • interace endpoints are controlled via SGs on thet interface. NACLs also impact traffic
  • code changes to use the endpoint DNS, or enable private DNS to override the default service DNS

List public endpoints

IPv6 within AWS

• IPv6 is supported, but not across every product and not with every feature

  • 2001:db8:1234:1a00::/56
  • Route tables can cointain IPv6 routes with the dafult route being ::/0 (all 0s)
  • VPCs have optionally have a fixed /56 range allocated by AWS. Each subnets uses a fixed /64. The /64 can be chosen from the /56, but the VPC range cannot be adjusted
  • Instacne can be allocated IPv6 addresses at launch in teh same way as IPv4 - they are static by default

• IPV6 VPC setup

  1. Disabled by default (opt-in)
  2. Request an IPv6 allocation, each VPC is allocated a /56 CIDS from the AWS pool (can’t be adjusted)
  3. With the VPC IPv6 ranbge allocated, subnets can be allocated a /64 CIDR from within the /56 range
  4. Resources launched into a subnet with an IPv6 range can be allocated a IPv6 address via DHCP6

• Limitations and considerations

  • DNS names are not allocated to IPv6 addresses
  • IPv6 addresses are all publicly routable - there is no concept of private vs public with IPv6 (unlike IPv4 addresses)
  • With IPv6, the OS is ocnfigured with this public address via DHCP6
  • Elastic IPs aren’t relevant with IPv6
  • Not concurrently supported for VPNs, customer gateways, and VPC endpoints

• Egress-Only Gateway

  • Egress only internet gateway: provide IPv6 instances with outgoing access to the public internet using IPv6 but prevent the instances from being accessed from the Internet

    • NAT isn’t required with IPv6, and so NATGW’s aren’t copatible with IPv6. Egress-only gateways provide the outgoing-only access of a NATGW but do so without adjusting bay IP addresses
    • Architecturally, they are otherwise the same as IGW

        graph LR
    
    A(incoming)
    
    subgraph VPC
      B[Instance IPv6]
      C((VPC router))
      D[Route table]
      E((Internet gateway))
    end
    
    F(outgoing)
    
    A -- disallowed --> E;
    B --> C;
    D --> C;
    C --> E;
    E -- egress only --> F;

DNS fundamentals

Global DNS (Route 53) Fundamentals: terms

• DNS root servers: trust servers: authoritative on root zone
• Top-Level Domain (TLD): geographical and generic (.es, .us, .com, .org)
• Subdomain: organization, anything between host and TLD (example.com)
• Zone and Zone File: mapping of IPs and hosts for a given subdomain (record of www for example.com)
• Records: DNS has severla record types (A, MX…)
• Name Server: server that runs a DNS service to store/cache info from DNS platform
• Authoritative: root servers ar authoritative for root zone = trusted by every OS and networking stack globally
• Hosts: record in a zone file (www, mail…)
• FQDN: Fully Qualified Domain Name (the host and domain: www.example.com)
  • Absolute domains have a period at the end e.g. www.example.com.)

DNS 101

• Domain Name System (DNS): turn DNS names into IP addresses. It is a distributed system (no part knows all)
• Steps (asking for example.com)
  1. Query your ISP, if it doesn’t know, it handles it for you
  2. The ISP queries the DNS root servers. I they don’t know, they help by providing servers *authoritative for .com
  3. The .com servers are queries. If they don’t have an IP, they provide the example.com authoritative servers
  4. The servers are run by LA, they know and return one or more IPs

Domain registration

• Register a domain in DNS
  1. Check if the domain is available (generally against a database of the TLD subdomain operator)
  2. Purchase the domain via registrar
  3. Hosting the domain (manage/pay for DNS hosting or name servers configured for your domain)
     • Route 53 allows you to registasr and host, or just host it, or just register it.
  4. Records in the zone file (on name servers that authoritative/host the domian, you need to add records into the zone file)

Hosted Zones

• Hosted Zone: container for DNS records relating to aparticular domain
• Route 53 supports public hosted zones (influence the visible domain)
• Visibility
  • Public Zones
    • created when you register a domain with Route53, when you transfer a domain into Route53, or if you create one manually
    • a hosted zone has the same nme as the domain it relates to (e.g. example.com will have a hosted zone called example.com)
    • public zone is accesible either from Internet-based DNS clients
    • a hosted one will have “name servers” - these are the IP addresses you can give to a domain operator, so Route 53 becomes “authoritative” for a domain
  • Private Zones
    • created manually and associated with one or more VPCs - they are only accessible from those VPCs
    • need enableDnsHostnames and enableDnsSupport enabled on a VPC
    • note all Route53 features supported - limits on health checks
    • split-view DNS is supported, using the same zone name ofr public and private zones - providing VPC resources with different reocrds
      • with split view, private is preferred - if no matches, public is used

Record Set Types

• DNS supports different types of records, each providing different functionality
  • A Record: provides an IPv4 address for given host
  • AAAA Record: provides an IPv6 address for given host
  • CNAME Record: allows aliases to be created
  • MX Record: provide the mail serves a given domain
  • NS Record: used to set authoritative servers for a subdomain (e.g. .com would have NS servs for example.com)
  • TXT Record: used for descriptive text ina domain (to verify domain ownership)
  • Alias Records: extension of CNAME - can be used like a A record, with the functionality of a CNAME and none of the limitations
    • can referto AWS logial services (load balancers, S3)
    • AWS doesn’t charge for queries iof alias records against AWS resources
• Default records of a zone - SOA, NS

Health checks

• Health checks can be created with Route53, used to influence Route53 routing decisions
• Classification
  • monitor endpoint
  • monitor another health check
  • monitor CloudWatch alarms
• Route53 health checkers
  • global health chekc system that checks an endpoint in a agreed way of frequency
  • values
    • >18% of checks report healty -> healthy
    • <18% of checks report healty -> unhealthy
• Types of health check
  • HTTP and HTTPS check: tcp/80 or tcp/443 connection checks in less than 4 seconds (reposrt 2xx or 3xx in 2 seconds)
  • TCP check: tcp connection within 10 seconds
  • HTTP/S check: all checks from HTTP and HTTPS but the body is checked for a String match -> 10, 30s
• Route53 and decisions via health checks
  • Records can be linked to health checks. If unhealthy -> record is not used
  • can be used to failover and other routing arcitectures

DNS advanced

Global DNS (Route 53): Routing policy

• Simple: single record within a hosted zone that contains 1 or more values

  • When queried - returns all the values in random order
  • • simple, default, even spread of requests
  • • no performace control, no granular health checks for alias type (only a single AWS resource)

• Failover: allows to create 2 records with the same name (primary and secoundary, in case the 1st is unhealthy)

  • Can be combinedwith other types to allow multiple primary and secoundary records

• Weightened: to control the ammount of traffic that reaches specific resources (!0% for 1st, 90% for 2nd)

  • Useful to test new software, or when resources are being added or removed from a configuration taht doesn’t use a load balancer
  • Records are returned on a ratio of their weight to the total weight, assuming records are healthy

• Latency based: Route53 consults a latency database each time a request occurs to give a latency-based host in DNS from a resolver server

  • Record sets with the same name are considered part of the same latency-set
  • Each is allocated to a region
  • The record set returned is the 1 with the lowest latency to the resolver server

• Geolocation (geoproximity): lets you choose the sresources that serve your trafic based on the geographic region from which queries originate

  • A record set is configured for a continent or a country
  • The record set is used for queries in that same region, with more specificmatches taking priority
  • Record sets can be set as the default that gets return if the IP matching process fails or if no record set is configured for the originating query region

Server-Based Compute (EC2) Fundamentals

Architecture

graph LR

A[EC2 host]
B[Instance Store volume]
C[EC2 instance]
D[EBS]
E[AMI]
F[CloudWatch]
G[Bucket]
H(ENI - Elastic Network interface)

A --> B;
B --> C;
C --> A;
C --> D;
E --> C;
C --> F;
D --> G;
C -- security group--> H;

• EC2 instances states
  • main states
    • run (you are only billed for this one)
    • stop
  • intermeduate states (you are not billed for these)
    • pending
    • stoping
    • terminate (deleting)

Types/sizes

• EC2 instances are gruped into families
  • general purpose
  • compute optimized
  • memory optimized
  • accelerated computing
• Types
  • T2 and T3: low-cost, rovide burst capability
  • M5: for general workloads
  • C4: provides more capable CPU
  • X1 and R4: optimize large ammounts of fast memory
  • I3: deliver fast IO
  • P2, G3 and F1: deliver GPU and FPGAs
• sizes
  • nano
  • micro
  • small
  • medium
  • large
  • x.large
  • 2x.large
  • larger
• special cases
  • ‘a’: use AMD CPUs
  • ‘A’: arm based
  • ‘n’: higher speed networking
  • ‘d’: NVMe storage

Storage architecture

• Elastic Block Service (EBS): storage service that creates abd manages volumes based on 4 underlying storage types
• Volumes: presistent, can be attached and removed from EC2 instances, and are replicated within a single AZ.
  • types
    • mechanical
      • sc1: lowest cost, infrequent access, can’t be boot volume
      • st1: low cost, throughput intensive, can’t be boot volume
    • solid state
      • gp2: default, balance of IOPS/MiB/s - burst pool IOPS perGB
      • io1: highst performance, can adjust size and IOPS separately
• to protect against AZ failure, EBS snapshots (to S3) can be used. Data is replicated accross AZs in the region and (optionally) internationally

EBS Snapshots

• EBS snapshots: a pint-in-time backup of an EBS volume stored in S3
• Initial snapshot: full copy of the volume
• Future snapshots only strore data changed since last snapshot
• Uses
  • move/copy instances between AZs
  • It is recommended to power of instance to create snapshot, or flush disk
  • snapshots can be copied between regiosn, shared and automated using DLM (Data Lifecycle Manager)

Security Groups

• Security groups are fostware firewalls, that can be attached to network interfaces
• Each have inbound and outbound rules (rules, from-to)
• They have a hidden implicit/default deny rule, but can not explicitly deny traffic
• Stateful
• Can reference AWS resoures, other security groups and even thenselves

Instance metadata

• Instance metadata: data relating to te instance taht can be accessed from within the instance itself using a utility capable of accessing HTTP and using the URL

• A way that scripts and applications running onEC2 can get visibility of data they would normally need API calls for
• The metadata can provide the currentexternal IPv4 address for the instace, which is not configured on the instance itself but provided by the Internet gateway in the VPC, It provides the AZ the instance was launched in and the security groups applied to the instance. In the case of spot instances, it will also provide the approximate time the instance will terminate

Server-Based Compute (EC2) Intermediate

AMI

• AMI (amazon Machine Image): used to build instacnes
• Store snapshots of EBS volumes, permissions and a block of device mapping, which configures how the instance OS sees the attacheed volumes
• AMIs can be shared, free or paid, and can be copied to other AWS
• Steps
  1. Configure instance (soure instance amd attached EBS volums are configured with any required software and configuration)
  2. Create image (snapshots are created from volumes, AMI references snapshots, permissions and block device mappings)
  3. Launch instance -> new instance
• With appropiate launch permissions, instances can be created from an AMI. EBS volumes are created using snapshots as the source, and an EC2 instance is created using the blk device mapping to reference its new volumes

Bootstrap

• Bootstrapping is a process where instructions are executed on an instance during its launch process. Bootstraping is used to configure the instance, perform software installation, and add application configuration.
• In EC”, user data can be used to run shell scripts (bash or powershell) or sun cloud-init directives

graph TD

A(S3 or Github)
B(OS repository)
C(User script)
D[AMI - OS + baked components]
E[Instance + user datascript]
F[Final instance]

A --> C;
B --> C;
C --> E;
D --> E;
E --> F;

ENI, IP, and DNS

• private instance (only communicates inside the VPC)

  • allocated an ip-x.x.x.x.ec2.internal DNS name - only works inside AWS
  • private IP allocated when launching instances
  • An ENI is assigned an IP

• public instance (has a public IP address)

  • Elastic IPs are static, when allocated, they replace the normal public IP, which is deallocated
  • The public DNS resolves to the public address externally, but the private address internally
  • a public IPv4 address can be allocated. This is allocated when the machine starts and deallocated when it stops

Instance roles

• EC2 instance roles: IAM roles that can be assumed by EC2 using and intermediary called an instance profile. An instance profile is either created automatically when using the console UI or manually when using the CLI. It’s a container for the role taht is associated with an EC2 instance.
• The instance profile allows applications on the EC2 instance to access the credentials from the role using instance metadata

graph TD

A[Permission policy]
B[IAM role]
C[EC2]
D[Credentials]

A -- STS assume role --> B;
B -- instance metadata provide temp access --> C;
C -- credentials can be used to access AWS --> D;

• Order of credentials
  1. Command line options: aws [command] --profile [profile name]
  2. Environment variables: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY_ID, AWS_SESSION_TOKEN
  3. AWS CLI credentials file: aws configure, not recommended for production environments
  4. Container credentials: IAM roles associated with AWS Elastic Container Service (ECS) Task Definitions. Recommended for ECS environments
  5. Instance profile credential: IAM roles associated with Amazon Elastic Compute Cloud (EC”) via Instance Profiles -> temporary credentials, recommended for EC2 environments

Server-Based Compute (EC2) advanced

EBS Volume and Snapshot Encryption

• Volume encryptin uses EC2 host hardware to encrypt data at rest and in transit beteen EBS and EC2 instances
• Encryption generates data encryption key (DEK) from a customer master hey (CMK) in each region
• Snapshots of that voluem are encrypted with the same DEK, as are any volumes created from that snapshot

graph TD

A[KMD]
B[EC2 host]
C[Encrypted DEKs]
D[EC2 instance]

A -- plaintext DEKs in EC2 memory --> B;
B --> D;
C -- decrypted via CMK from host--> A;
C --> D;

EC2 instance and OS see plaintext data as normal -> no performance impact

EBS optimization, enhanced networking, placement groups

• EBS optimization

  • Legacy non-EBS-optimized used a shared networking path for data and storage communications
  • EBS-optimized mode, which was historically optinal and is now the default, adds optimization paths for storage and traditional data networking (separated).
    • Consistent utilization
    • Required feature to support higher performance storage

• Enhanced networking

  • Traditional virtual networking: an EC2 arranges access for n virtual machinesto access 1 physical network card (software multitasking = slow)
  • Enhanced networking: uses SR-IOV, allows a single physical network card appear as multiple physical devices. Each instance can be given 1 of these (fake) physical devices (faster transfer rates, lower CPU usage, lower latency).
    • EC2 delivers via Elastic Network Adapter (ENA)
    • EC2 delivers via Intel 82599 Virtual Function (VF) Interface

• Placement Groups

  • Cluster PG: 1 next to each other, just 1 availability zone, all clusters on the same place
  • Partition PG: 1 next to each other, 2 availability zones, part 1 one AZ, part in the other
  • Spread PG: similar to partition, max of 7 instances per AZ.
    • each instance occupies a partition
    • each instance has an isolated fault domain
    • great on mail servers, domain controllers, file servers and application HA pairs.

EC2 billing models

• Spot and Spot Fleet

  • Spot instances allow consumption of spare AWS capacity for a given instance type and size in a specific AZ
    • provided as long as yur bid price is above the spot price
    • if you bis is exceeded, instances are terminated in a 2 minute warning
  • Spot fleets are container for “capacity needs”
    • you aim specify pools of certain types/sizes aiming for a given “capacity”. A minimum percentage of non-demand can be set to ensure the fleet is always active.
    • perfect for non-critical workloads, burst workloas, consistent non-critical jobs that can tolerate interrumptions without impacting functionality
    • not suitable for lonniing workloads that require stability and cannot tolerate interrumptions

• Reserved Instances

  • Reserved instances: lock in a reduced rate for 1-3 years
  • Zonal reserved instances: include capacity reservation
  • Commitment: cost even if instances aren’t launched
  • Reserved = long running, understood and consistent workloads

Dedicated Hosts

• Dedicated hosts: EC2 hosts for a given type and size that can be dedicated to you
• The number of instances that can run the host is fixed, depending on the type and size
• An on-demand or reserved fee is charged for the dedicated host
• Used when software is licensed per core/CPU and not compatible with running within a shared cloud environment

Serverless compute (Lambda)

APIs and microservices

• Microservices architecture: inverse of monolithic: components are separated into microservicesand operate independently.

  • A microservice does 1 thing - and does it well
  • Operations, updates and scaling can be done on a per-microservice basis
  • Inflexible scaling: either increasing or decreasing the instance size or duplicating the instance
  • Microservices operate as independent applications: allow direct communication between compoennts and users

• API (Application Programming Interface) is and interface accessed (consumed) by another service (rest/soap: json/xml)

  • API endpoints re locations taht allow API interaction, hosts 1 or more APIs and makes them available for on a network (public or private)
  • static (abstracted ffrom whta the code is doing) -> lower risk changes

Serverless and event-driven architectures

• Serverless compute
  • when using an event driven architecture, a system operates around “events” that represent an action or a change of stae
  • efficient beacuse events are generated and pushed, rathr than been polled (traditional polls = compute on, scale poorly)
  • principles
    • Back-end as a Service (BaaS) -> 3rd party services where possible rather tahn running your own (auth, cognito, dynamoDB)
    • Function as a Service (FaaS) -> application logic

Lambda essentials

• Lambda: FaaS -> Functions = code which run at runtime
  • Functions are invoked by events, perform actions for up to 15 minutes, and terminate
  • Stateless (each run is clean, lack of persistance)
  • Parts
    • Runtime environment (prebuilt, e.g. Python)
    • Funciton code
    • Execution role (access to AWS -> temproary security credentials available via STS)
• Lambda free tier: 100ms

API gateway essentials

• API ateway: managed API endpoint service
• Can be used to create, publish, monitor and secure APIs “as a service”
• Can use other AWS services for compute (FaaS, IaaS) as well to store recall data

Step functions

Container-based compute and microservices

• Step functions: serverless visual workflow service, that provides state machines. A state machine can orchestrate other AWS services with simple logic, branching and parallel execution, and it mantains a state.
• Workflow steps are known as states, and they can perform work via tasks
• Step functions allows ofr long-running serverless workflows
• A state machine can be defined using Amazon States Language (ASL) -> similar to json
• Longest runtime allowable on a state machine: up to 1 year

graph LR

A[state machine is executed from another service or component]
B[Lambda]

subgraph AWS seo functions
  C(Start)
  D[Lambda]
  E[Lambda]
  F(Manual approval)
  G(approved)
  H(rejected)
end

I[API gateway]
J[Email]

A --> B;
B --> C;
C --> D;
D -- generate email --> J;
J -- send email --> I;
I --> E;
E -- update states --> F;
F --> G;
F --> H;

Docker

• Container: package that contains an application, libraries, and file systems equired to tun it
  • Run on a container engine that generally runs within a single OS, such as Linux
  • Povide isolation benefits of virtualization but more lightweight, allowing faster starts and more dense packing with a host
• Example from EC2 machine

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

  sudo amazon-linux-extras install docker sudo service docker start sudo usermod -a -G docker ec2-user sudo yum install git git clone https://github.com/linuxacademy/content-aws-csa2019.git cd content-aws-csa2019/lesson_files/03_compute/Topic5_Containers/Docker/ docker build -t containercat . docker images --filter reference=containercat docker run -t -i -p 80:80 containercat docker login --username YOUR_USER docker images docker tag IMAGEID YOUR_USER/containercat docker push YOUR_USER/containercat

ECS

• A managed container engine: it allows dicker containers to be deployed and managed within AWS environments
• EC2 can use infrastructure clusters (for backing infrastructure):

1. Based on EC2

graph TD

subgraph ECS
  A[Cluster manager]
  B[Placement engine]
end

subgraph EC2
  C[EC2-1]
  D[EC2-2]
end

E(Image)

A --> C;
B -- task definition is used to create ECS task--> D;
E -- stored and rertieved from registry --> D;

2. Based on Fargate (service that encapsulates clusters)

graph TD

subgraph ECS
  A[Cluster manager]
  B[Placement engine]
end

C[Fargate service]

D(Image)

A --> C;
B -- task definition is used to create ECS task--> C;
D -- stored and rertieved from registry --> C;

• Tips
  • Cluster: logicaal collection of ECS resources
  • Task definition: defines your application, similar to a dockerfile but for running containers in ECS
  • Container definition: inside a task definition, a container definition defines the individual containers used by a Task. It controls the CPU and memory each container has, in addition to port mappings for the container
  • Task: a single running copy of any containers defined by a task definition. 1 working copy of an application (e.g. DB and web containers)
  • Service: services allow task definitions to be scaled by adding additional tasks. Defined Minimum and Maximum values
  • Registry: storage for container images (e.g ECS Container Registry, DockerHub). Used to download image to create containers

IAM (Identity and Access Management)

IAM essentials

• Identity an Access Management: primary servcies that handles authentication and authorization within AWs environments.
  • Systems architecture is incomplete without being able to control access in a granular way.
  • IAM controls access to AWS services via policies that can be attached to users, groups, and roles. Users are gien long term credentials to access AWS resources (username, pàssword, access keys).
  • Roles allow for short term access to resources when assumed using temporary access credentials.
• Amazon Resource Name (ARN):
  • always begin with arm:partition:service:region:account-id
    • parition = aws or aws-cn (China)
  • depending on the service,finish with:
    • resource
    • resourcetype/resource
    • resourcetype/resource/qualifier
    • resourcetype/resource:qualifier
    • resourcetype:resource
    • resourcetype:resource:qualifier

5 different types of access

graph LR

A(1. External principal)
B[2. AWS service]
C{IAM role}
D{IAM policy}
E{IAM credentials}
F[Bucket]
G(3. Root user)
H{IAM policy}
I{4. IAM group}
J(5. IAM user)

A --> C;
B --> C;
C --> D;
D --> E;
E --> F;
G -- Full access --> F;
H --> I;
H --> J;
I -- AWS CLI --> E;
J -- Console UI --> F;

IAM policies

• IAM policy (policy document) is known as identiy policy when attached or and identity or a resource policy when attached to a resource
• Document

  1 2 3 4

  { "Version": "2020-03-30" "Statement": [{...},{...}.,{...}] }

• Each statement makes a request to AWS. Requests are matched based on their action (or actions), which the API calls or operations being attempted and the repurce (or resources) the request is against. A given statement results in Allow or Deny for the request

  1 2 3 4 5 6 7 8 9 10

  { "Sid": "SpecificTable", "Effect: "Allow", "Action": [ "dynamodb:BatchGet", "dynamodb:CreateTable" ], "Resource": "arn:ws:dynamodb:*:*:table/TestPictures" }

• Tips
  • Managed policies allow the same policy to impact many identities
  • Inline policies allow exceptions to be applied to identities
  • AWs amnaged policies are low overhead but lack flexibility
  • Customer managed policies are flexible but require administration
  • Inline and managed policies can apply to users, groups and roles

IAM users

• IAM users: a type of IAM identity for long-term access for a known entity (human, service, application)
• Principal autehticate to IAM either with a username and a password, or using access keys

graph LR

A(User)
B((MFA))
C[IAM]
D[AWS SDK & CLI]
E(IAM user identity)
F(Resource)

A --> B
B --> C
A -- auth username & password --> C
D -- access keys --> C
C --> E
E --> F

• Tips
  • Hard limit: 5000 IAM user per account
  • 10 group memeberships per IAM users
  • Default maximum of 10 management policies per user
  • No inline limit, but no more tahn 2048 chars for all inline policies on IAM users
  • 1 MFA per user
  • 2 access keys per user

IAM groups

• IAM group: collection of IAM users. Groups allow easier administration over sets of IAM users. Inline and managed policies can be applied to groups that flow to members of that group
• Groups are not a true identity - they can not be the principal in a policy, so they can not be used in resource policies
• Tips
  • groups = admin feature
  • groups can contain many IAM users, and users can be in many groups
  • IAM inline policies can be aded to IAM groups - and these flow on to IAM users who are members
  • managed IAM policies can be attached and flow on to IAM users who are members
  • groups are not “true” identities (because it cannot be identified as a Principal in a permission policy), and they can be referenced from resource policies
  • groups have no credentials

IAM access keys

• Access keys: a pair of values used by aplications, SDKs, or the AWS CLI to authenticate to AWS
• Parts
  • access key id -> public, stored by AWS once generated
  • secret access key -> private, avaiabel only once the access key is fully generated. Stored by the owner
• Only 2 sets allowed, that can be created, deleted and disabled
• They can’t be used to log on console, and don’t expire

Securing your account — Creating an IAM user and setting up the CLI

• Linux install

  1 2 3 4 5

  sudo yum install epel-release sudo yum install python-pip sudo pip install awscli aws configure # enter the access keys

IAM Roles

• IAM roles: are assumed by another identity allowed in thetrust policy (IAM user AWS service, another AWS account, web identity or even an anonymous identity)
• When arole is assumed, the security token (STS) generated a time limited set of access keys (temporary security credentials). These keys have permissios defined in the permissions policy
• IAM roles have no long-term credentials

Multi-account management and organizations

AWS organizations

• AWs organizations: service for managing multiple accounts within a single business

• Allow consolidation

• All accountswithing an AWS Organizain can consolidate bills into a single account (1 bill covering all business usage)

• Organizations can share bulk discounts and even easily manage accounts and permisssions and limit account usage using service control policies

• Tips

  • Service control policies (SCP) enables permission controls
  • Root user of an Organization Unit account can be restricted by creating and attaching a SCP
  • Benefits
    • Consolidated billing
    • Reduced admin overhead
    • Reserved purchases can be used by member accounts.
  • SCP FullAWSAccess: allows access to all AWS services within an attached member account
  • SCP affect any account and organizational unit under the master account
  • Attach SCP:
    • Log in to the master account and create the SCP
    • Attach the SCP to the member account within the Organizational Unit
    • Enable the SCP for the Organizational Unit
    • Select the Organizational Unit

Role switching between accounts

• Role switching is a method of accessing one account from another inly using a set of credentials
• Used both in organizations and between 2 unconnected accounts
• Information needed for role switching
  • OrganizationAccountAccessRole as the role
  • Account ID of the member account
  • The display name of the role that will be seen in the linked account

graph LR

subgraph Account1
A(Account A identity)
end

subgraph Account2
B(Account B identity)
C[IAM role]
D[Trust policy]
end

A -- sts:AsumeRole --> C
C --> B
D --> A