AWS Certified Developer Associate 자격증 준비 - EC2편

AWS Certified Developer Associate 자격증을 준비하면서, 알게된 내용을 정리하였습니다. ( + 영어 문서 익숙해지기)

강의(적극 추천드립니다): https://www.udemy.com/course/aws-certified-developer-associate-dva-c01/

 

 

Amazon EC2 - Basics

 

Amozon EC2

• EC2 = Elastic Compute Cloud = Instructure as a Service
• It mainly consists in the capability of:
  • Renting virtual machines (EC2)
  • Storing data on virtual drives (EBS)
  • Distributing load across machines (ELB)
  • Scaling the services using an auto-scalig group (ASG)

 

EC2 sizing & configuration options

• Operating System (OS): Linux, Windows or Mac OS
• How much compute power & cores (CPU)
• How much random-access memory (RAM)
• How much storage space:
  • Network-attached (EBS & EFS)
  • hardware (EC2 Instance Store)
• Network card:speed of the card, Public IP address
• Firewall rules: security group
• Bootstrap script (configure at first launch): EC2 User Data

 

EC2 User Data

• It is possible to bootstrap our instances using an EC2 User data script
• bootstrapping means launching commands when a machine starts
• That script is only run once at the instance first start
• EC2 user data is used to automate boot tasks such as:
  • Installing updates
  • Installing software
  • Downloading common files from the internet
  • Anything you can think of
• The EC2 User Data Script runs with the root user

 

 

EC2 Instance Types - Overview

• You can use different types of EC2 instances that are optimised for diffent use cases
  (https://aws.amazon.com/ko/ec2/instance-types/)
• AWS has the following naming convention:
  • m: instance class
  • 5: generation (AWS improves them over time)
  • 2xlarge: size within the instance class

 

 

EC2 Instance Types - General Purpose (범용)

• Great for a diversity of workloads such as web servers or code repositories
  웹 서버 또는 코드 저장소와 같은 다양한 워크로드에 적합
• Balance between:
  • Compute
  • Memory
  • Networking

 

EC2 Instance Types -Compute Otimized (컴퓨터 최적화)

• Great for compute-intensive tasks that require high performance processors:
  고성능 프로세서가 필요한 컴퓨팅 집약적인 작업에 적합:
  • Batch processing workloads
  • Media transcoding
  • High performance web servers
  • High performance computing (HPC)
  • Scientific modeling & machine learning
  • Dedicate gaming servers

 

EC2 Instance Types - Memory Optimized

• Fast performance for workloads that process large data sets in memory
  메모리의 대용량 데이터 세트를 처리하는 워크로드를 위한 빠른 성능 제공
• Use cases:
  • High performance, relational/non-relational databases
  • Distributed web scale cache stores
  • In-memory databases optimized for BI (business intelligence)
  • Applications performing real-time processing of big unstructured data

 

EC2 Instance Types - Storage Optimized

• Great for storage-intensive tasks that require high, sequential read and write access to large data sets on local storage
  로컬 스토리지의 대용량 데이터 세트에 대한 높은 순차적 읽기 및 쓰기 액세스가 필요한 스토리지 집약적인 작업에 적합
• Use cases:
  • High frequency online transaction processing (OLTP) systems
  • Relational & NoSQL databases
  • Cache for in-memory databases (for example, Redis)
  • Data warehousing applications
  • Distributed file systems

 

EC2 Instance Types: example

 

 

 

Introduction to Security Groups

• Security Groups are the fundamental of network security in AWS
  보안 그룹은 AWS의 네트워크 보안의 기본 요소입니다
• They control how traffic is allowed into or out of our EC2 Instances.
• Security groups only contain allow rules
• Security groups rules can reference by IP or by security group

 

 

 

Security Groups Deeper Dive

• Security groups are acting as a "firewall" on EC2 instances
• They regulate:
  • Access to Ports
  • Authorised IP ranges - IPv4 and IPv6
  • Control of inbound network (from other to the instance)
  • Control of outbound network (from the instance to other)

 

 

Security Groups Good to know

• Can be attached to multiple instances
• Locked down to a region VPC combination
  영역 VPC 조합으로 잠금 설정
• Does live "outside" the EC2 - if traffic is blocked the EC2 instance won't see it
  EC2의 "외부"에 거주하는지 여부 - 트래픽이 차단되면 EC2 인스턴스가 이를 볼 수 없음
• It's good to maintain one separate security group for SSH access
  SSH 액세스를 위해 하나의 보안 그룹을 별도로 유지하는 것이 좋습니다
  
• If your application is not accessible (time out), then it's security group issue
• If your application gives a "connection refused" error, then it's an applicaion error or it's not launched
• All inbound traffic is blocked by default
• All outbound traffic is authorised by default

 

Referencing security groups Diagram

 

 

Classic Ports to know

• 22 = SSH (Secure Shell) - log into a Linux instance
• 21 = FTP (File Transfer Protocol) - upload files into a file share
• 22 = SFTP (Secure File Transfer Protocol) - upload files using SSH
• 80 = HTTP
• 443 = HTTPS
• 3389 = RDP (Remote Desktop Protocol) - log into a Winodows instance

 

SSH Summary Table

 

 

 

EC2 Instances Purchasing Options

• On-Demand Instances - short workload, predictable pricing, pay by second
• Reserved (1 & 3 years)
  • Reserved Instances - long workloads
  • Convertible Reserved Instances - long workloads with flexible instances
• Savings Plans ( 1 & 3 years) - commitment to an amount of usage, long workload
• Spot Instances - short workloads, cheap, can lose instances (less reliable)
• Dedicated Hosts - book an entire physical server, control instance placement
• Dedicated Instances - no other customers will share your hardware
• Capacity Reservations - reserve capacity in a specific AZ for any duration

 

EC2 On Demand

• Pay for what you use:
  • Linux or Windows - billing per second, after the first minute
  • All other operating sysyems - billing per hour
• Has the highest cost but no upfront payment
• No long-term commitment
• Recommended for short-term and un-interrupted workloads, where you can't predict how the application will behave
  애플리케이션의 동작을 예측할 수 없는 단기적이고 중단되지 않는 워크로드에 권장

 

EC2 Reserved Instances

• Up to 72% discount compared to On-demand
• You reserve a specific instance attributes (Instance Type, Region, Tenancy, OS)
• Reservation Period - 1 Year (+discount) or 3 years ( +++discount)
• Payment Options - No Upfront (+), Partial Upfront (++), All Upfront (+++)
• Reserved Instance's Scope - Regional or Zonal (reserve capacity in an AZ)
• Recommended for steady-state usage applications (think database)
• You can buy and sell in the Reserved Instance Marketplace

 

• Convertible Reserved Instance
  • can change the EC2 intance type, instance family, OS, scope and tenancy
  • Up to 66% discount

 

EC2 Savings Plans

• Get a discount based on long-term usage (up to 72% - same as Rls)
• Commit to a certain type of usage ($ 10/hour for 1 or 3 years)
  특정 유형의 사용(1년 또는 3년 동안 시간당 10달러)에 커밋
• Usage beyond EC2 Saving Plans is billed at the On-Demand price
  EC2 절약 요금제를 초과한 사용량은 주문형 가격으로 청구됩니다

 

• Locked to a specific instance family & AWS region (e.g., M5 in us-east-l)
• Flexible across:
  • Instance Size (e.g., m5.xlarge, m5.2xlarge)
  • OS (e.g., Linux, Windows)
  • Tenancy (Host, Dedicated, Default)

 

 

EC2 Spot Instances

• Can get a discount of up to 90% compared to On-demand
• Instances that you can "lose" at any point of time if your max price is less than the current spot price
  최대 가격이 현재 현물 가격보다 낮은 경우 언제든지 "손실"할 수 있는 경우
• The MOST cost-efficient instances in AWS
• Useful for workloads that are resilient to failure
  장애에 강한 워크로드에 유용
  
  • Batch jobs
  • Data analysis
  • Image processing
  • Any distributed workloads
  • Workloads with a flexible start and time
• Not suitable for critical jobs or databases
  중요한 작업 또는 데이터베이스에 적합하지 않음

 

EC2 Dedicated Hosts

• A physical server with EC2 instance capacity fully dedicated to your use
  EC2 인스턴스 용량을 완벽하게 사용할 수 있는 물리적 서버
• Allows you address compliance requirements and use your existing server-bound software licenses (per-socket, per-core, pe-VM software licenses)
• Purchasing Options:
  • On-demand - pay per second for active Dedicated Host
  • Reserved - 1 or 3 years (No Upfront, Partial Upfront, All Upfront)
• The most expensive option

 

• Useful for software that have complicated licesing model (BYOL - Bring Your Own License)
  복잡한 라이센스 모델(BYOL - Bride Your Own License)을 가진 소프트웨어에 유용합니다
• Or for companies that have strong regulatory or compliace needs
  또는 강력한 규제 또는 규정 준수 요구가 있는 회사의 경우

 

EC2 Dedicated Instances

 

• Instances run on hardware that's dedicated to you
• May share hardware with other instances in same account
• No control over instance placement
  (can move hardware after Stop / Start)

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

EC2 Capacity Reservations (EC2 용량 예약)

• Reserve On-Demand instances capacity in a specific AZ for any duration
• You always have access to EC2 capacity when you need it
• No time commitment (create/cancel anytime), no billing discounts
• Combine with Regional Reserved Instances and Savings Plans to benefit fron billing discounts
• You're charged at On-Demand rate whether you run instances or not

 

• Suitable for short-term, uninterrupted workloads that needs to be in a specific AZ(Availability Zones)
  특정 AZ에 있어야 하는 단기간의 중단 없는 워크로드에 적합

 

Which purchasing option is right for me?

• On demand: coming and staying in resort whenever we like, we pay the full price
• Reserved: like planning ahead and if we plan to stay for a long time, we may get a good discount.
• Savings Plans: pay a certain amount per hour for certain period and stay in any room type
• Spot instances: the hotel allows people to bid for the emptu rooms and the highest bidder keeps the rooms. You can get kicked out at any time
• Dedicated Hosts: We book an entire building of the resort
• Capacity Reservations: you book a room for a period with full price even you don't stay in it

 

 

Amazon EC2 - Instance Storage

 

What's an EBS Volume?

 

• An EBS (Elastic Block Store)Volume is a network drive you can attach to your instances while they run
• It allows your instances to persist data, even after their termination
• They can only be mounted to one instance at a time (at the CCP level)
• They are bound to a specific availability zone
  특정 가용성 영역에 속합니다

 

• Analogy:Think of them as a "network USB stick"
• Free tier:30GB of free EBS storage of type General Purpose (SSD) or Magnetic per month

 

EBS Volume

• It's network drive (i.e. not a physical drive)
  • It uses the network to communicate the instance, which means there might be a bit of latency
    네트워크를 사용하여 인스턴스를 통신합니다. 이는 약간의 지연 시간이 있을 수 있음을 의미합니다
  • It can be detached fron an EC2 instance and attached to another one quickly
• It's locked to an Availability Zone (AZ)
  • An EBS Volume in use-east-1 a cannot be attached to us-east-1 b
    사용 중인 EBS 볼륨 - east-1 a는 us-east-1 b에 첨부할 수 없습니다
  • To move a volume across, you first need to snapshot it
• Have a provisioned capacity (size in GBs, and IOPS)
  • You get billed for all the provisioned capacity
  • You can increase the capacity of the drive over time

 

EBS - Delete on Termination attribute

• Controls the EBS behaviour when an EC2 instance terminates
  • By default, the root EBS volume is deleted (attribute enabled)
  • By default, any other attached EBS volume is not deleted (atrribute disabeld)
• This can be controlled by the AWS console / AWS CLI
• Use case:preserve root volume when instance is terminated

 

 

EBS Snapshot

• Make a backup (snapshot) of your EBS volume at a point in time
• Not necessary to detach volume to do snapshot, but recommended
• Can copy snapshots across AZ or Region

 

 

EBS Snapshots Features

 

 

• EBS Snapshot Archive
  • Move a Snapshot to an "archive tier" that is 75% cheaper
  • Takes within 24 to 72 hours for restoring the archive
• Recycle Bin for EBS Snapshots
  • Setup rules to retain deleted snapshots so you can recover them after an accidental deletion
    실수로 삭제된 스냅샷을 복구할 수 있도록 삭제된 스냅샷을 보존하는 규칙 설정
    보존(1일 ~ 1년) 지정
  • Specify retention (from 1 day to 1 year)
• Fast Snapshot Restore (FSR)
  • Force full initialization of snapshot to have no latency on the first use ($$$)

 

 

 

AMI = Amazon Machine Image

• AMI are a customization of an EC2 instance
  
  • You add your own software, configuration, operating system, monitoring...
  • Faster boot / configuration time because all your software is pre-packaged
• AMI are built for a specific region (and can be copied across regions)
  AMI는 특정 지역에 대해 구축되어 있으며 여러 지역에 걸쳐 복사할 수 있습니다
• You can launch EC2 instances from:
  • A Pubilc AMI: AWS provided
  • Your own AMI: you make and maintain them yourself
  • An AWS Marketplace AMI: an AMI someone else made (and potentially sells)

 

AMI Process ( from an EC2 instacne)

• Start an EC2 instacne and customize. it
• Stop the instance (for data integrity)
• Build an AMI - this will also create EBS snapshots
• Launch instances from other AMIs

 

 

EC2 Instance Store

• EBS volumes are network drives with good but "limited" performance
• If you need a high-performance hardware disk, use EC2 Instance Store

 

• Better I/O performance
• EC2 Instance Store lose their storage if they're stopped (ephemeral)
• Good for buffer / cache / scratch data / temporary content
• Risk of data loss if hardware fails
• Backups and Replication are your responsibility

 

Local EC2 Instance Store

 

 

 

EBS Volume Types

• EBS Volumes come in 6 types
  • gp2 / gp3 (SSD): General purpose SSD volume that balances price and performance for a wide variety of workloads
  • io1 / io2 (SSD): Highest-performance SSD volume for mission-critical low-latency or high-throughput workloads
  • st1 (HDD): Low cost HDD volume designed for frequently accessed, throughput-intensive workloads
  • sc1 (HDD): Lowest cost HDD volume designed for less frequently accessed workloads
• EBS Volumes are characterized in Size | Throughput | IOPS (I/O Ops Per Sec)
• Only gp2/gp3 and io1/io2 can be used as boot volumes

 

 

EBS Volume Types Use cases General Purpose SSD

• Cost effective storage, low-latency
• System boot volumes, Virtual desktops, Development and test environments
• 1Gib - 16Tib
• gp3:
  • Baseline of 3,000 IOPS and throughput of 125 MiB/s
  • Can increase IOPS up to 16,000 and throughput up to 1000 Mib/s independently
• gp2:
  • Small gp2 volumes can burst IOPS to 3,000
  • Size of the volume and IOPS are linked, max IOPS is 16,000
  • 3 IOPS per GB, means at 5,334 GB are at the max IOPS

 

 

 

EBS Volume Types Use cases Provisioned IOPS (PIOPS) SSD

• Critical business applications with sustained IOPS performance
• Or applications that need more than 16,000 IOPS
• Great for databases workloads (sensitive to storage perf an consistency)
• io1/io2 (4GiB - 16TiB):
  • Max PIOPS: 64,000 for Nitro EC2 instances & 32,000 for other
  • Can increase PIOPS independently from storage size
  • io2 have more durability and more IOPS per GiB (at the same price as io 1)
• io2 Block Express (4GiB - 64TiB):
  • Sub-millisecond latency
  • Max PIOPS:256,000 with an IOPS:GIB ratio of 1,000:1
• Supports EBS Multi-attach

 

EBS Volume Types Use cases Hard Disk Drives (HDD)

• Cannot be a boot volume
• 125 GiB to 16TiB
• Throughput Optimized HDD (st1)
  • Big Data, Data Warehouse, Log Processing
  • Max throughput 500 MiB/s - max IOPS 500
• Cold HDD (sc1):
  • For data that is infrequently accessed
  • Scenarios where lowest const is important
  • Max throughput 250 MiB/s - max IOPS 250

 

EBS - Volume Types Summary

 

 

 

EBS Multi-Attach - io1/io2 familysa

• Attach the same EBS volume to multiple EC2 instances in the same AZ
• Each instance has full read & write permissions to the high-performance volume
• Use case:
  • Achieve higher application availability in clustered Linux applications(ex:Teradata)
  • Applications must manage concurrent write operations
• Up to 16 EC2 Instances at a time
• Must use a file system that's cluster-aware (not XFS, EXT4, etc...)

 

 

 

 

 

 

EFS - Elastic File System

• Managed NFS (network file system) that can be mounted on many EC2
• EFS works with EC2 instances in multi-AZ
• Highly available, scalable, expensive (3x gp2), pay per use

 

 

 

 

EFS - Elastic File System

• Use cases: content management, web serving, data sharing, Wordpress
• Uses NFSv4.1 protocol
• Uses security group to control access to EFS
• Compatible with Linux based AMI (now Windows)
• Encryption at rest using KMS

 

• POSIX file sysyem (~Linux) that has a standard file API
• File sysyem scales automatically, pay-per-use, no capacity planning!

 

 

EFS - Performance & Storage Classes

• EFS Scale
  • 1000s of concurrent NFS clients, 10GB + /s throughput
  • Grow to Petabyte-scale network file system, automatically
• Performance Mode (set at EFS creation time)
  • General Purpose (default) - latency-sensitive use cases (web server, CMS, etc...)
  • Max I/O - higher latency, throughput, highly parallel (big data, media, processing)
• Throughput Mode
  • Bursting - 1TB = 50MiB/s + burst of up to 100MiB/s
  • Provisioned - set your throughput regardless of storage size, ex: 1GiB/s for 1TB storage
  • Elastic - automatically scales throughput up or down based on your workloads
    • Up to 3GiB/s for reads and 1GiB/s for writes
    • Used for unpredicatable workloads

 

 

 

EFS - Storage Classes

• Storage Ties (lifecycle management feature - move file after N days)
  • Standard:for frequently accessed files
  • Infrequent access (EFS-IA): cost to retrieve files, lower price to store. Enable EFS-IA with a Lifecycle Policy
• Availability and durability
  • Standard: Multi-AZ, great for prod
  • One Zone: One AZ, great for dev, backup enabled by default, compatible with IA (EFS One Zone-IA)
• Over 90% in cost savings

 

 

 

 

 

 

 

 

 

 

 

EBS vs EFS - Elastic Block Storage

• EBS volumes...
  • one instance (except multi-attach io1/io2)
  • are locked at the Availiability Zone (AZ) lebel
  • gp2: IO increase IO independently
  • io1: can increase IO independently
• To migrate an EBS volume across AZ
  • Take a snapshot
  • Restore the snapshot to another AZ
  • EBS backups use IO and you shouldn't run them while youre application is handling a lot of traffic
• Root EBs Volumes of instances get terminated by default if the EC2 instance gets terminated. (you can disable that)

 

 

 

EBS vs EFS - Elastic File

 

 

• Mounting 100s of instances across AZ
• EFS share website files (WordPress)
• Only for Linux Instances (POSIX)

 

• EFS has a higher price point than EBS
• Can leverage EFS-IA for cost savings

 

• Remember: EFS vs EBS vs Instance Store