Overview
Let's look at the costs that arise when using multiple Ingress resources on Amazon EKS and how to share an Application Load Balancer (ALB) efficiently with the AWS Load Balancer Controller's Ingress Grouping feature.
The Problem
Default Ingress Behavior
In Kubernetes, an Ingress is an API object that provides HTTP/HTTPS routing from outside the cluster to internal services. With the AWS Load Balancer Controller, an ALB is automatically provisioned whenever an Ingress resource is created.
yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-app-ingress
annotations:
kubernetes.io/ingress.class: alb
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
spec:
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-app-service
port:
number: 80Cost Issues
In a microservices architecture, each service gets its own Ingress, which creates the following issues.
- A separate ALB for each Ingress: One ALB is created per Ingress
- High operating costs: ALBs are billed hourly, so costs rise as unnecessary ALBs accumulate
- Management complexity: The burden of managing multiple ALBs individually
What Is Ingress Grouping?
Ingress Grouping is an AWS Load Balancer Controller feature that allows multiple Ingress resources to share a single ALB.
Ingress Grouping으로 두 Namespace의 Ingress가 ALB 하나를 공유
Ingress Grouping으로 두 Namespace의 Ingress가 ALB 하나를 공유
Read the diagram as text
- blue-green-ns의 blue-green-ingress가 group.name app-color-lb를 쓴다.
- orange-purple-ns의 orange-purple-ingress도 같은 group.name을 쓴다.
- AWS Load Balancer Controller가 두 Ingress 규칙을 자동 병합해 ALB 하나에 적용한다. 두 Ingress의 ADDRESS가 같다.
- ALB의 /blue, /green 규칙은 blue-green-ns의 blue-service, green-service로 간다.
- ALB의 /orange, /purple 규칙은 orange-purple-ns의 orange-service, purple-service로 간다.
Key Features
- Shared ALB: Multiple Ingress resources share one ALB
- Cross-namespace sharing: Ingress resources in different namespaces can be grouped
- Automatic rule merging: The controller automatically merges Ingress rules and applies them to the ALB
- Cost savings: Fewer ALBs directly reduce costs
Prerequisites and Installation
To use Ingress Grouping, you need to install and configure a few required components first.
1. EKS Cluster Requirements
Basic Requirements
- EKS cluster: Kubernetes 1.19 or later
- Node groups: EC2 or Fargate (or a mix)
- VPC configuration: Subnets in at least 2 Availability Zones
bash
# EKS 클러스터 생성 (eksctl 사용 예시)
eksctl create cluster \
--name my-cluster \
--region us-west-2 \
--nodegroup-name standard-workers \
--node-type m5.large \
--nodes 3 \
--nodes-min 1 \
--nodes-max 4 \
--managed2. Install the AWS Load Balancer Controller
Ingress Grouping is supported in AWS Load Balancer Controller v2.0 and later.
2-1. Create an IAM Policy
First, create the IAM policy for the AWS Load Balancer Controller.
bash
# IAM 정책 다운로드
curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.7.2/docs/install/iam_policy.json
# IAM 정책 생성
aws iam create-policy \
--policy-name AWSLoadBalancerControllerIAMPolicy \
--policy-document file://iam_policy.json2-2. Create an IAM Role and Service Account
bash
# OIDC 공급자 생성 (한 번만 실행)
eksctl utils associate-iam-oidc-provider --region=us-west-2 --cluster=my-cluster --approve
# 서비스 계정 생성 및 IAM 역할 연결
eksctl create iamserviceaccount \
--cluster=my-cluster \
--namespace=kube-system \
--name=aws-load-balancer-controller \
--role-name AmazonEKSLoadBalancerControllerRole \
--attach-policy-arn=arn:aws:iam::ACCOUNT-ID:policy/AWSLoadBalancerControllerIAMPolicy \
--approve2-3. Install the Controller with Helm
bash
# Helm 차트 레포지토리 추가
helm repo add eks https://aws.github.io/eks-charts
helm repo update
# AWS Load Balancer Controller 설치
helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
-n kube-system \
--set clusterName=my-cluster \
--set serviceAccount.create=false \
--set serviceAccount.name=aws-load-balancer-controller \
--set region=us-west-2 \
--set vpcId=vpc-xxxxxxxx2-4. Verify the Installation
bash
# Controller 팟 상태 확인
kubectl get deployment -n kube-system aws-load-balancer-controller
# 로그 확인
kubectl logs -n kube-system -l app.kubernetes.io/name=aws-load-balancer-controller3. Configure the VPC and Subnets
3-1. Tag the Subnets
Tag the subnets so that the ALB is created in the correct ones.
bash
# 퍼블릭 서브넷 태깅 (인터넷 대면 ALB용)
aws ec2 create-tags \
--resources subnet-12345678 subnet-87654321 \
--tags Key=kubernetes.io/role/elb,Value=1
# 프라이빗 서브넷 태깅 (내부 ALB용)
aws ec2 create-tags \
--resources subnet-abcdefgh subnet-hgfedcba \
--tags Key=kubernetes.io/role/internal-elb,Value=1
# 클러스터별 태깅 (여러 클러스터 사용 시)
aws ec2 create-tags \
--resources subnet-12345678 subnet-87654321 \
--tags Key=kubernetes.io/cluster/my-cluster,Value=shared3-2. Check the Security Group Configuration
bash
# 클러스터 보안 그룹에 태그 추가 (필요시)
aws ec2 create-tags \
--resources sg-xxxxxxxxx \
--tags Key=kubernetes.io/cluster/my-cluster,Value=owned4. Required Annotations and IngressClass Configuration
4-1. Create an IngressClass (Kubernetes 1.18+)
yaml
# ingress-class.yaml
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
name: alb
spec:
controller: ingress.k8s.aws/albbash
kubectl apply -f ingress-class.yaml4-2. Validate the Basic Configuration
bash
# AWS Load Balancer Controller 버전 확인
kubectl get deployment aws-load-balancer-controller -n kube-system -o yaml | grep image:
# 서비스 계정 확인
kubectl get serviceaccount aws-load-balancer-controller -n kube-system -o yaml
# IAM 역할 확인
kubectl describe serviceaccount aws-load-balancer-controller -n kube-system5. Test the Installation
5-1. Deploy a Simple Test Application
yaml
# test-app.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: test-app
namespace: default
spec:
replicas: 2
selector:
matchLabels:
app: test-app
template:
metadata:
labels:
app: test-app
spec:
containers:
- name: app
image: nginx:latest
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: test-service
namespace: default
spec:
type: NodePort
selector:
app: test-app
ports:
- port: 80
targetPort: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test-ingress
namespace: default
annotations:
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
spec:
ingressClassName: alb
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: test-service
port:
number: 80bash
# 테스트 앱 배포
kubectl apply -f test-app.yaml
# Ingress 상태 확인
kubectl get ingress test-ingress
# ALB 생성 확인
kubectl describe ingress test-ingress6. Troubleshooting Guide
6-1. Common Problems
Problem 1: The Controller Pod does not start
bash
# 서비스 계정 권한 확인
kubectl describe serviceaccount aws-load-balancer-controller -n kube-system
# IAM 역할 정책 확인
aws iam list-attached-role-policies --role-name AmazonEKSLoadBalancerControllerRoleProblem 2: No ALB is created
bash
# Controller 로그 확인
kubectl logs -n kube-system -l app.kubernetes.io/name=aws-load-balancer-controller
# 서브넷 태그 확인
aws ec2 describe-subnets --subnet-ids subnet-12345678 --query 'Subnets[*].Tags'Problem 3: Insufficient permissions error
bash
# IAM 정책이 최신인지 확인
aws iam get-policy-version \
--policy-arn arn:aws:iam::ACCOUNT-ID:policy/AWSLoadBalancerControllerIAMPolicy \
--version-id v16-2. Installation Checklist
- [ ] EKS cluster is operating normally
- [ ] AWS Load Balancer Controller v2.0+ is installed
- [ ] IAM policy and service account are configured
- [ ] Subnets have the appropriate tags
- [ ] IngressClass resource is created
- [ ] ALB creation is verified with a test Ingress
Hands-on: Implementing Ingress Grouping
With the prerequisites complete, let's implement Ingress Grouping.
1. Prepare the Basic Environment
First, deploy an application in each of two namespaces.
bash
# 네임스페이스 생성
kubectl create namespace blue-green-ns
kubectl create namespace orange-purple-ns2. Deploy the Applications
Deploy the color-specific applications in each namespace.
yaml
# blue-green-ns 네임스페이스의 애플리케이션
apiVersion: apps/v1
kind: Deployment
metadata:
name: blue-app
namespace: blue-green-ns
spec:
replicas: 2
selector:
matchLabels:
app: blue-app
template:
metadata:
labels:
app: blue-app
spec:
containers:
- name: blue-app
image: nginx:latest
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: blue-service
namespace: blue-green-ns
spec:
type: NodePort
selector:
app: blue-app
ports:
- port: 80
targetPort: 803. Configure Ingress Grouping
This is the key step: configuring Ingress Grouping with the alb.ingress.kubernetes.io/group.name annotation.
yaml
# blue-green-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: blue-green-ingress
namespace: blue-green-ns
annotations:
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
alb.ingress.kubernetes.io/group.name: app-color-lb # 그룹 이름 지정
spec:
ingressClassName: alb
rules:
- http:
paths:
- path: /blue
pathType: Prefix
backend:
service:
name: blue-service
port:
number: 80
- path: /green
pathType: Prefix
backend:
service:
name: green-service
port:
number: 80yaml
# orange-purple-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: orange-purple-ingress
namespace: orange-purple-ns
annotations:
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
alb.ingress.kubernetes.io/group.name: app-color-lb # 동일한 그룹 이름
spec:
ingressClassName: alb
rules:
- http:
paths:
- path: /orange
pathType: Prefix
backend:
service:
name: orange-service
port:
number: 80
- path: /purple
pathType: Prefix
backend:
service:
name: purple-service
port:
number: 804. Check the Results
After deployment, checking the Ingress status shows that both Ingress resources share the same ALB address.
bash
kubectl get ingress -A
NAMESPACE NAME ADDRESS
blue-green-ns blue-green-ingress k8s-appcolorlb-xxx.elb.amazonaws.com
orange-purple-ns orange-purple-ingress k8s-appcolorlb-xxx.elb.amazonaws.comAdvanced Configuration
1. Control Group Order
Use the group.order annotation to control the order in which Ingress rules are evaluated.
yaml
metadata:
annotations:
alb.ingress.kubernetes.io/group.name: my-group
alb.ingress.kubernetes.io/group.order: '10' # 낮은 숫자가 우선순위 높음2. Group Naming Rules
- Length: 63 characters or fewer
- Characters: Only lowercase letters, digits,
-, and.are allowed - Start/end: Must begin and end with a letter or digit
yaml
# 올바른 예시
alb.ingress.kubernetes.io/group.name: my-app-group
alb.ingress.kubernetes.io/group.name: production.web-services
# 잘못된 예시
alb.ingress.kubernetes.io/group.name: -my-group- # 잘못된 시작/끝
alb.ingress.kubernetes.io/group.name: My_Group # 대문자와 언더스코어 사용 불가Security Considerations
1. Trust Boundary
All users in the same Ingress Group must be within the same trust boundary. If another user creates an Ingress using the same group name, they can overwrite existing rules.
2. Prevent Group Name Collisions
yaml
# 권장: 명확하고 고유한 이름 사용
alb.ingress.kubernetes.io/group.name: team-a-production-services
# 비권장: 일반적인 이름 사용
alb.ingress.kubernetes.io/group.name: my-group3. Configure RBAC
Configure appropriate RBAC policies for Ingress resources so that only authorized users can access specific groups.
Limitations and Precautions
1. ALB Rule Limits
- Default limit: Up to 100 rules per ALB listener
- Performance impact: More rules may affect performance
- Monitoring: Monitor the rule count continuously
2. Conflict Resolution
By default, rule order across Ingress resources is determined by the lexicographic order of namespace/name. Use group.order to control the order explicitly.
3. Debugging Complexity
Because multiple Ingress resources share one ALB, identifying the cause of a problem can be complicated.
Monitoring and Troubleshooting
1. Check AWS Load Balancer Controller Logs
bash
kubectl logs -f -n kube-system -l app.kubernetes.io/instance=aws-load-balancer-controller2. Check ALB Rules
bash
# ALB ARN 확인
ALB_ARN=$(aws elbv2 describe-load-balancers --query 'LoadBalancers[?contains(LoadBalancerName, `k8s-mygroup`)].LoadBalancerArn' --output text)
# 리스너 규칙 확인
LISTENER_ARN=$(aws elbv2 describe-listeners --load-balancer-arn $ALB_ARN --query 'Listeners[0].ListenerArn' --output text)
aws elbv2 describe-rules --listener-arn $LISTENER_ARN3. Check Ingress Status
bash
kubectl describe ingress <ingress-name> -n <namespace>Practical Use Cases
1. Microservices Environments
yaml
# API Gateway 패턴
alb.ingress.kubernetes.io/group.name: api-gateway
# /api/users -> user-service
# /api/orders -> order-service
# /api/products -> product-service2. Separation by Environment
yaml
# 개발 환경
alb.ingress.kubernetes.io/group.name: dev-services
# 스테이징 환경
alb.ingress.kubernetes.io/group.name: staging-services
# 프로덕션 환경
alb.ingress.kubernetes.io/group.name: prod-servicesCost Savings
Example Calculation
- Existing approach: 10 services = 10 ALBs = $200+ per month (ALB at $0.0225 per hour × 24 hours × 30 days × 10)
- Ingress Grouping: 10 services = 2 ALBs = $40+ per month (split into groups)
- Savings: $160+ per month (approximately 80% cost reduction)
Closing Notes
Ingress Grouping can significantly reduce ALB costs on Amazon EKS. It is particularly worth considering in environments using a microservices architecture.
Key points:
- Simple configuration with the
alb.ingress.kubernetes.io/group.nameannotation - Ingress sharing across namespaces
- Substantial cost savings
- Understand the security considerations and limitations
- Appropriate monitoring and management are needed
Next time, I will also cover advanced load balancing patterns using Target Group Binding.