Terraform 华为云实践
一、基本配置
1. 华为云控制台
访问华为云控制台，进行以下工作
创建用户 terraform
创建用户组 tf-admin
配置组授权，允许操作所用到的各类资源
下载认证文件（保存 AccessKey、SecretKey）
2. 系统环境变量
设置 AK、SK
export HW_REGION_NAME="cn-north-1"
export HW_ACCESS_KEY="xxx"
export HW_SECRET_KEY="xxxxxx"二、模块化资源定义
1. global 配置
obs backend
使用 华为云 OBS 对象存储保存 tfstate 状态文件，以目录区分不同环境、组件的状态文件
初始化目录结构
terraform-huawei-demo
└── global
└── backend
├── backend.tf
├── main.tf
├── outputs.tf
├── variables.tf
└── versions.tf创建 versions.tf 定义 provider 版本
terraform {
required_providers {
huaweicloud = {
source = "huaweicloud/huaweicloud"
version = ">=1.36.0"
}
}
}创建 variables.tf 定义 region
variable "region" {
type = string
default = "cn-north-1"
}创建 main.tf 引用 provider
provider "huaweicloud" {
region = var.region
}创建 obs.tf 定义存储 backend state 文件的 obs 存储
resource "huaweicloud_obs_bucket" "bucket" {
bucket = "tf-dayo-backend-bucket"
acl = "private"
tags = {
type = "bucket"
}
}初始化 backend，这一步主要是为了确认 provider 配置，以及拉取 provider 文件（注意 GFW 问题）
$ tf init创建 obs backend 资源
创建 obs 资源
$ tf apply -auto-approve由于 obs 资源的名称在平台是具有唯一性的，所以，可能会出现重名
│ Error: Error creating bucket tf-backend-bucket: BucketAlreadyExists,
│ Reason: The requested bucket name is not available. The bucket namespace is shared by all users of the system. Please select a different name and try again.
│
│ with huaweicloud_obs_bucket.bucket,
│ on main.tf line 5, in resource "huaweicloud_obs_bucket" "bucket":
│ 5: resource "huaweicloud_obs_bucket" "bucket" {重新修改使用一个新名称即可
$ terraform.exe apply -auto-approve
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# huaweicloud_obs_bucket.bucket will be created
+ resource "huaweicloud_obs_bucket" "bucket" {
+ acl = "private"
+ bucket = "tf-dayo-backend-bucket"
+ bucket_domain_name = (known after apply)
+ bucket_version = (known after apply)
+ encryption = false
+ enterprise_project_id = (known after apply)
+ force_destroy = false
+ id = (known after apply)
+ kms_key_project_id = (known after apply)
+ multi_az = (known after apply)
+ policy = (known after apply)
+ policy_format = "obs"
+ quota = 0
+ region = (known after apply)
+ storage_class = "STANDARD"
+ storage_info = (known after apply)
+ tags = {
+ "type" = "bucket"
}
+ versioning = false
}
Plan: 1 to add, 0 to change, 0 to destroy.
Changes to Outputs:
+ bucket_domain_name = (known after apply)
~ bucket_name = "tf-backend-bucket" -> "tf-dayo-backend-bucket"
huaweicloud_obs_bucket.bucket: Creating...
huaweicloud_obs_bucket.bucket: Creation complete after 1s [id=tf-dayo-backend-bucket]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Outputs:
bucket_domain_name = "tf-dayo-backend-bucket.obs.cn-north-1.myhuaweicloud.com"
bucket_name = "tf-dayo-backend-bucket"访问控制台，检查确认
使用 obs 作为 Backend
创建 backend.tf
terraform {
backend "s3" {
# obs bucket 名称
bucket = "tf-dayo-backend-bucket"
# 配置 obs 中状态文件的路径
key = "global/backend/terraform-global-backend.tfstate"
region = "cn-north-1"
# endpoint = "tf-dayo-backend-bucket.obs.cn-north-1.myhuaweicloud.com"
endpoint = "obs.cn-north-1.myhuaweicloud.com"
skip_credentials_validation = true
# skip_metadata_api_check = true
# skip_region_validation = true
}
}这个报错就是没有配置 AWS_ACCESS_KEY_ID 与 AWS_SECRET_ACCESS_KEY 的错误信息，是个小坑
Initializing the backend...
╷
│ Error: error configuring S3 Backend: no valid credential sources for S3 Backend found.
│
│ Please see https://www.terraform.io/docs/language/settings/backends/s3.html
│ for more information about providing credentials.
│
│ Error: NoCredentialProviders: no valid providers in chain. Deprecated.
│ For verbose messaging see aws.Config.CredentialsChainVerboseErrors配置下
# 仍旧是 华为云 terraform 用户的 AK、SK
# 华为云 oss 弄得有点怪，非得搞个 AWS 开头的名变量
export AWS_ACCESS_KEY_ID="xxxxx"
export AWS_SECRET_ACCESS_KEY="xxxxx"还有一个坑，配置完 AWK_* 认证变量后，可能还是会报错，这里还需要在做几项配置
│ Error: error configuring S3 Backend: error validating provider credentials: error calling sts:GetCallerIdentity: InvalidClientTokenId: The security token included in the request is invalid.完整配置如下：
terraform {
backend "s3" {
# obs bucket 名称
bucket = "tf-dayo-backend-bucket"
# 配置 obs 中状态文件的路径
key = "global/backend/terraform-global-backend.tfstate"
region = "cn-north-1"
# endpoint = "tf-dayo-backend-bucket.obs.cn-north-1.myhuaweicloud.com"
endpoint = "obs.cn-north-1.myhuaweicloud.com"
skip_credentials_validation = true
# skip_metadata_api_check = true
# skip_region_validation = true
}
}尝试 init
Initializing the backend...
Do you want to copy existing state to the new backend?
Pre-existing state was found while migrating the previous "local" backend to the
newly configured "s3" backend. No existing state was found in the newly
configured "s3" backend. Do you want to copy this state to the new "s3"
backend? Enter "yes" to copy and "no" to start with an empty state.
Enter a value: yes
Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing provider plugins...
- Reusing previous version of huaweicloud/huaweicloud from the dependency lock file
- Using previously-installed huaweicloud/huaweicloud v1.47.1
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.访问控制台，检查确认
上传上去后，就可以删除掉本地 terraform.tfstate 与 terraform.tfstate.backup 文件
2. network 资源
创建目录 env/dev/network，存放所有网络相关资源的定义
创建以下文件：
versions.tf 定义 provider
veriables.tf 定义 region 变量
backend.tf 配置 tfstate 文件路径（这个很重要）
main.tf 调用后面的各种模块创建资源
outputs.tf：对外输出模块的返回值，供以其他资源使用
目录结构
$ tree terraform-huawei-demo
terraform-huawei-demo
├── env
│ └── dev
│ └── network
│ ├── backend.tf
│ ├── main.tf
│ ├── outputs.tf
│ ├── variables.tf
│ └── versions.tf
└── global
└── backend
├── backend.tf
├── main.tf
├── outputs.tf
├── variables.tf
└── versions.tftf init 初始化
Initializing the backend...
Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing provider plugins...
- Finding huaweicloud/huaweicloud versions matching ">= 1.36.0"...
- Using huaweicloud/huaweicloud v1.47.1 from the shared cache directory
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.vpc 资源
官方文档：https://registry.terraform.io/providers/huaweicloud/huaweicloud/latest/docs/resources/vpc
定义模块
模块化设计，将 vpc 资源抽象到 modules 目录，以便于复用
terraform-huawei-demo
├── env
│ └── dev
│ └── network
│ ├── backend.tf
│ ├── main.tf
│ ├── outputs.tf
│ ├── variables.tf
│ ├── versions.tf
│ └── vpc.tf
├── global
│ └── backend
│ ├── backend.tf
│ ├── main.tf
│ ├── outputs.tf
│ ├── variables.tf
│ └── versions.tf
└── modules
└── vpc
├── main.tf
├── outputs.tf
├── variables.tf
└── versions.tfversions.tf 定义 provider
# modules/vpc/versions.tf
terraform {
required_providers {
huaweicloud = {
source = "huaweicloud/huaweicloud"
version = ">=1.36.0"
}
}
}variables.tf 定义模块传参
# modules/vpc/versions.tf
variable "vpc_name" {}
variable "vpc_cidr" {}
variable "subnet_cidr" {}
variable "subnet_name" {}
variable "subnet_gateway_ip" {}
variable "availability_zone" {}outputs.tf 定义返回值
# modules/vpc/outputs.tf
output "vpc_id" {
value = huaweicloud_vpc.vpc.id
}
output "subnet_id" {
value = huaweicloud_vpc_subnet.subnet.id
}
output "subnet_subnet_id" {
value = huaweicloud_vpc_subnet.subnet.subnet_id
}main.tf 定义模块管理资源
# modules/vpc/main.tf
resource "huaweicloud_vpc" "vpc" {
name = var.vpc_name
cidr = var.vpc_cidr
}
resource "huaweicloud_vpc_subnet" "subnet" {
name = var.subnet_name
cidr = var.subnet_cidr
gateway_ip = var.subnet_gateway_ip
vpc_id = huaweicloud_vpc.vpc.id
availability_zone = var.availability_zone
}定义资源
创建 env/dev/network/vpc.tf，调用 vpc module，传递参数，创建资源
# env/dev/network/vpc.tf
locals {
vpc_cidr = "172.16.0.0/12"
vpc_name = "dev-vpc"
subnet_name = "dev-subnet"
subnet_cidr = "172.16.0.0/24"
subnet_gateway_ip = "172.16.0.1"
availability_zone = "cn-north-1a"
}
module "dev-vpc" {
source = "../../../modules/vpc"
vpc_name = local.vpc_name
vpc_cidr = local.vpc_cidr
subnet_name = local.subnet_name
subnet_cidr = local.subnet_cidr
subnet_gateway_ip = local.subnet_gateway_ip
availability_zone = local.availability_zone
}执行 tf init 拉取模块
terraform.exe init
Initializing modules...
- dev-vpc in ..\..\..\modules\vpc
# ...执行 tf apply 创建资源
terraform.exe apply -auto-approve
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# module.dev-vpc.huaweicloud_vpc.vpc will be created
+ resource "huaweicloud_vpc" "vpc" {
+ cidr = "172.16.0.0/12"
+ enterprise_project_id = (known after apply)
+ id = (known after apply)
+ name = "dev-vpc"
+ region = (known after apply)
+ routes = (known after apply)
+ status = (known after apply)
}
# module.dev-vpc.huaweicloud_vpc_subnet.subnet will be created
+ resource "huaweicloud_vpc_subnet" "subnet" {
+ availability_zone = "cn-north-1a"
+ cidr = "172.16.0.0/24"
+ dhcp_enable = true
+ dns_list = (known after apply)
+ gateway_ip = "172.16.0.1"
+ id = (known after apply)
+ ipv4_subnet_id = (known after apply)
+ ipv6_cidr = (known after apply)
+ ipv6_gateway = (known after apply)
+ ipv6_subnet_id = (known after apply)
+ name = "dev-subnet"
+ primary_dns = (known after apply)
+ region = (known after apply)
+ secondary_dns = (known after apply)
+ subnet_id = (known after apply)
+ vpc_id = (known after apply)
}
Plan: 2 to add, 0 to change, 0 to destroy.
module.dev-vpc.huaweicloud_vpc.vpc: Creating...
module.dev-vpc.huaweicloud_vpc.vpc: Creation complete after 7s [id=506afa4d-7a05-4f29-88b7-80d14ffd62d9]
module.dev-vpc.huaweicloud_vpc_subnet.subnet: Creating...
module.dev-vpc.huaweicloud_vpc_subnet.subnet: Creation complete after 8s [id=09e3c780-2469-4faa-9638-b9f2e76496e8]
Apply complete! Resources: 2 added, 0 changed, 0 destroyed.控制台确认
secgroup 资源
定义模块
目录结构
# ...
terraform-huawei-demo
├── env
│ └── dev
│ └── network
│ ├── backend.tf
│ ├── main.tf
│ ├── outputs.tf
│ ├── secgroup.tf
│ ├── variables.tf
│ ├── versions.tf
│ └── vpc.tf
├── global
│ └── backend
│ ├── backend.tf
│ ├── main.tf
│ ├── outputs.tf
│ ├── variables.tf
│ └── versions.tf
└── modules
├── secgroup
│ ├── main.tf
│ ├── outputs.tf
│ ├── variables.tf
│ └── versions.tf
└── vpc
├── main.tf
├── outputs.tf
├── variables.tf
└── versions.tfversions.tf 定义 provider
# modules/secgroup/versions.tf
terraform {
required_providers {
huaweicloud = {
source = "huaweicloud/huaweicloud"
version = ">=1.36.0"
}
}
}variables.tf 定义模块传参
# modules/secgroup/variables.tf
variable "sec_group_name" {}
variable "sec_group_desc" {}outputs.tf 定义返回值
# modules/secgroup/outputs.tf
# 返回 sec_group_id，给 ecs 关联使用
output "sec_group_id" {
value = huaweicloud_networking_secgroup.secgroup.id
}main.tf 定义模块管理资源
# modules/secgroup/main.tf
resource "huaweicloud_networking_secgroup" "secgroup" {
name = var.sec_group_name
description = var.sec_group_desc
}
# 出站规则：开放 80 端口给所有客户端访问
resource "huaweicloud_networking_secgroup_rule" "secgroup_rule_in_80" {
security_group_id = huaweicloud_networking_secgroup.secgroup.id
direction = "ingress"
ethertype = "IPv4"
protocol = "tcp"
# 开放 80 端口给所有客户端访问
port_range_min = 80
port_range_max = 80
remote_ip_prefix = "0.0.0.0/0"
}
# 出站规则：开放 22 端口给所有客户端访问
resource "huaweicloud_networking_secgroup_rule" "secgroup_rule_in_22" {
security_group_id = huaweicloud_networking_secgroup.secgroup.id
direction = "ingress"
ethertype = "IPv4"
protocol = "tcp"
port_range_min = 22
port_range_max = 22
remote_ip_prefix = "0.0.0.0/0"
}
# 出站规则：开放所有 TCP 出向端口访问
resource "huaweicloud_networking_secgroup_rule" "secgroup_rule_out_all" {
security_group_id = huaweicloud_networking_secgroup.secgroup.id
direction = "egress"
ethertype = "IPv4"
protocol = "tcp"
port_range_min = 1
port_range_max = 65535
remote_ip_prefix = "0.0.0.0/0"
}定义资源
env/dev/network/secgroup.tf
# env/dev/network/secgroup.tf
locals {
sec_group_name = "dev_secgroup"
sec_group_desc = "dev secgroup for terraform"
}
module "dev-secgroup" {
source = "../../../modules/secgroup"
sec_group_name = local.sec_group_name
sec_group_desc = local.sec_group_desc
}tf init 拉取模块
terraform.exe init
Initializing modules...
- dev-secgroup in ..\..\..\modules\secgroup
# ...tf apply 创建资源
terraform.exe apply -auto-approve
module.dev-vpc.huaweicloud_vpc.vpc: Refreshing state... [id=506afa4d-7a05-4f29-88b7-80d14ffd62d9]
module.dev-vpc.huaweicloud_vpc_subnet.subnet: Refreshing state... [id=09e3c780-2469-4faa-9638-b9f2e76496e8]
Note: Objects have changed outside of Terraform
Terraform detected the following changes made outside of Terraform since the
last "terraform apply":
# module.dev-vpc.huaweicloud_vpc.vpc has changed
~ resource "huaweicloud_vpc" "vpc" {
id = "506afa4d-7a05-4f29-88b7-80d14ffd62d9"
name = "dev-vpc"
+ tags = {}
# (5 unchanged attributes hidden)
}
# module.dev-vpc.huaweicloud_vpc_subnet.subnet has changed
~ resource "huaweicloud_vpc_subnet" "subnet" {
id = "09e3c780-2469-4faa-9638-b9f2e76496e8"
name = "dev-subnet"
+ tags = {}
# (12 unchanged attributes hidden)
}
Unless you have made equivalent changes to your configuration, or ignored the
relevant attributes using ignore_changes, the following plan may include
actions to undo or respond to these changes.
─────────────────────────────────────────────────────────────────────────────
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# module.dev-secgroup.huaweicloud_networking_secgroup.secgroup will be created
+ resource "huaweicloud_networking_secgroup" "secgroup" {
+ created_at = (known after apply)
+ description = "dev secgroup for terraform"
+ enterprise_project_id = (known after apply)
+ id = (known after apply)
+ name = "dev_secgroup"
+ region = (known after apply)
+ rules = (known after apply)
+ updated_at = (known after apply)
}
# module.dev-secgroup.huaweicloud_networking_secgroup_rule.secgroup_rule_in_22 will be created
+ resource "huaweicloud_networking_secgroup_rule" "secgroup_rule_in_22" {
+ action = (known after apply)
+ direction = "ingress"
+ ethertype = "IPv4"
+ id = (known after apply)
+ port_range_max = 22
+ port_range_min = 22
+ ports = (known after apply)
+ priority = (known after apply)
+ protocol = "tcp"
+ region = (known after apply)
+ remote_address_group_id = (known after apply)
+ remote_group_id = (known after apply)
+ remote_ip_prefix = "0.0.0.0/0"
+ security_group_id = (known after apply)
}
# module.dev-secgroup.huaweicloud_networking_secgroup_rule.secgroup_rule_in_80 will be created
+ resource "huaweicloud_networking_secgroup_rule" "secgroup_rule_in_80" {
+ action = (known after apply)
+ direction = "ingress"
+ ethertype = "IPv4"
+ id = (known after apply)
+ port_range_max = 80
+ port_range_min = 80
+ ports = (known after apply)
+ priority = (known after apply)
+ protocol = "tcp"
+ region = (known after apply)
+ remote_address_group_id = (known after apply)
+ remote_group_id = (known after apply)
+ remote_ip_prefix = "0.0.0.0/0"
+ security_group_id = (known after apply)
}
# module.dev-secgroup.huaweicloud_networking_secgroup_rule.secgroup_rule_out_all will be created
+ resource "huaweicloud_networking_secgroup_rule" "secgroup_rule_out_all" {
+ action = (known after apply)
+ direction = "egress"
+ ethertype = "IPv4"
+ id = (known after apply)
+ port_range_max = 65535
+ port_range_min = 1
+ ports = (known after apply)
+ priority = (known after apply)
+ protocol = "tcp"
+ region = (known after apply)
+ remote_address_group_id = (known after apply)
+ remote_group_id = (known after apply)
+ remote_ip_prefix = "0.0.0.0/0"
+ security_group_id = (known after apply)
}
Plan: 4 to add, 0 to change, 0 to destroy.
module.dev-secgroup.huaweicloud_networking_secgroup.secgroup: Creating...
module.dev-secgroup.huaweicloud_networking_secgroup.secgroup: Creation complete after 0s [id=071cf562-a52c-4914-9cdc-c5b6f95aa595]
module.dev-secgroup.huaweicloud_networking_secgroup_rule.secgroup_rule_out_all: Creating...
module.dev-secgroup.huaweicloud_networking_secgroup_rule.secgroup_rule_in_22: Creating...
module.dev-secgroup.huaweicloud_networking_secgroup_rule.secgroup_rule_in_80: Creating...
module.dev-secgroup.huaweicloud_networking_secgroup_rule.secgroup_rule_in_22: Creation complete after 1s [id=6f313dcd-b4e3-49c1-8927-522f07458c04]
module.dev-secgroup.huaweicloud_networking_secgroup_rule.secgroup_rule_out_all: Creation complete after 1s [id=56d9ab47-97f7-4353-ab29-cd82a5f270a5]
module.dev-secgroup.huaweicloud_networking_secgroup_rule.secgroup_rule_in_80: Creation complete after 2s [id=cc7063ba-d7a5-426f-ba30-1b9dea762db9]
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.控制台确认，VPC → 访问控制 → 安全组
OK，network 基本已经完成，最后一步，把上述资源的关键信息，通过 outputs.tf 输出出去，以便于后续 ecs 资源调用
# network/outputs.tf
# 返回 sec_group_id，给 ecs 关联使用
output "vpc_id" {
value = module.dev-vpc.vpc_id
}
output "subnet_id" {
value = module.dev-vpc.subnet_id
}
output "subnet_subnet_id" {
value = module.dev-vpc.subnet_subnet_id
}
# 返回 sec_group_id，给 ecs 关联使用
output "sec_group_id" {
value = module.dev-secgroup.sec_group_id
}重新 apply 一下，这一次不会做任何变更，但是会多出几条输出信息
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Outputs:
sec_group_id = "071cf562-a52c-4914-9cdc-c5b6f95aa595"
subnet_id = "09e3c780-2469-4faa-9638-b9f2e76496e8"
subnet_subnet_id = "df7bdd8b-11e1-4ee3-8c11-e0ec20394fad"
vpc_id = "506afa4d-7a05-4f29-88b7-80d14ffd62d9"3. service 资源
service 资源目录中会包含：ecs、eip、elb、dns等资源的定义调用（主要就是调用 modules 中的各个模块，通过传参自定义创建资源）
ecs
创建 env/dev/service 目录，照例创建以下文件，也可以直接从 network 复制
versions.tf：与 network/version 一致，因为里面只有 provider 的定义
variables.tf：与 network/version 一致，因为里面只有 region 的定义
outputs.tf：清空，后面这里要修改返回值
backend.tf：修改 tfstate 文件路径
terraform { backend "s3" { # obs bucket 名称 bucket = "tf-dayo-backend-bucket" # 配置 obs 中状态文件的路径 key = "env/dev/service/terraform-dev-service-backend.tfstate" region = "cn-north-1" # endpoint = "tf-dayo-backend-bucket.obs.cn-north-1.myhuaweicloud.com" endpoint = "obs.cn-north-1.myhuaweicloud.com" skip_credentials_validation = true # skip_metadata_api_check = true # skip_region_validation = true } }main.tf：清空，由于 ecs 模块会用到 secgroup、vpc_id 等数据，所以这里需要配置读取远程 tfstate 文件
provider "huaweicloud" { region = var.region } # 通过 data 查询远程存储中的 secgroup、vpc 等资源的数据，供以 ecs 模块使用 data "terraform_remote_state" "network" { backend = "s3" config = { # config 配置中的参数基本为 network backend 各参数值 bucket = "tf-dayo-backend-bucket" key = "env/dev/network/terraform-dev-network-backend.tfstate" region = "cn-north-1" endpoint = "obs.cn-north-1.myhuaweicloud.com" skip_credentials_validation = true } }
定义模块
老规矩，创建 modules/ecs 模块
versions.tf 保持不变，只定义 provider 配置
variables.tf 定义 ecs 模块创建资源时所需变量
# modules/ecs/variables.tf
variable "instance_name" {}
variable "image_id" {}
variable "flavor_id" {}
variable "secgroup_id" {}
variable "availability_zone" {}
variable "os_init_script" {}
variable "admin_password" {}
variable "subnet_id" {}
variable "vpc_id" {}outputs.tf 定义模块返回值
# modules/ecs/outputs.tf
# 实例 ID
output "instance_id" {
value = huaweicloud_compute_instance.basic.id
}
# 镜像 ID
output "instance_image_name" {
value = huaweicloud_compute_instance.basic.image_name
}
# 外网 IP
output "instance_public_ip" {
value = huaweicloud_compute_instance.basic.public_ip
}
# 内网 IP
output "instance_ip_ipv4" {
value = huaweicloud_compute_instance.basic.access_ip_v4
}main.tf
resource "huaweicloud_compute_instance" "basic" {
name = var.instance_name
image_id = var.image_id
flavor_id = var.flavor_id
security_group_ids = [var.secgroup_id]
availability_zone = var.availability_zone
# 官方文档中提示：user_data 设置时会导致 admin_pass 失效
user_data = data.template_file.shell.rendered
# 所以配置使用 私钥登录
key_pair = var.keypair_name
network {
uuid = var.subnet_id
}
lifecycle {
# 先创建新的资源实例，再销毁旧资源
create_before_destroy = true
}
}
# 系统初始化脚本
data "template_file" "shell" {
template = file(var.os_init_script)
}定义资源
创建 env/dev/service/ecs.tf 文件，调用模块
locals {
availability_zone = "cn-north-1a"
# 从查询结果列表中选出第一个符合条件的实例规则
# flavor_id = data.huaweicloud_compute_flavors.flavor_1C1G.ids[0]
}
data "huaweicloud_compute_flavors" "flavor_1C1G" {
region = var.region
availability_zone = local.availability_zone
/* 规格类型：
normal 通用计算型
computingv3 通用计算增强型
highmem 内存优化型
saphana 超大内存型
diskintensive 磁盘密集型
*/
performance_type = "normal"
cpu_core_count = 1
memory_size = 1
}
output "ecs_flavor" {
value = data.huaweicloud_compute_flavors.flavor_1C1G.ids
}这里可以先通过 data 查询出符合自己需要的实例规格
执行 tc init 初始化
terraform.exe init
Initializing the backend...
Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing provider plugins...
- terraform.io/builtin/terraform is built in to Terraform
- Finding huaweicloud/huaweicloud versions matching ">= 1.36.0"...
- Using huaweicloud/huaweicloud v1.47.1 from the shared cache directory
# ...执行 tc plan
terraform.exe plan
Changes to Outputs:
+ ecs_flavor = [
+ "s2.small.1",
+ "s3.small.1",
]
You can apply this plan to save these new output values to the Terraform
state, without changing any real infrastructure.在实际环境中建议去华为云官网核验下费用，衡量性价比
还有镜像也需要注意下
data "huaweicloud_images_image" "centos_image" {
# 企业使用建议用 name 固定镜像名称，避免受 most_recent 影响，
# name_regex = "^CentOS 7"
name_regex = "CentOS 7.9 64bit"
visibility = "public"
architecture = "x86"
most_recent = true
}
# 受上述疑似 bug 的影响，华为云似乎无法获取镜像列表，只能从符合规则镜像中选择最近的一个
output "huaweicloud_images_image" {
value = data.huaweicloud_images_image.centos_image
}most_recent 配置为 false 会导致操作，估计是 bug，又或者是现阶段就是不支持以列表形式返回多个结果（不理解他们怎么想的）
data "huaweicloud_images_image" "centos_image" {
name_regex = "^CentOS"
visibility = "public"
architecture = "x86"
most_recent = false
}
output "huaweicloud_images_image" {
value = data.huaweicloud_images_image.centos_image
}报错如下
╷
│ Error: Your query returned more than one result. Please try a more specific search criteria, or set `most_recent` attribute to true.
│
│ with data.huaweicloud_images_image.centos_image,
│ on ecs.tf line 30, in data "huaweicloud_images_image" "centos_image":
│ 30: data "huaweicloud_images_image" "centos_image" {完整配置如下
# env/dev/service/ecs.tf
locals {
availability_zone = "cn-north-1a"
counts = 2
# 从查询结果列表中选出第一个符合条件的实例规则
flavor_id = data.huaweicloud_compute_flavors.flavor_1C1G.ids[0]
image_id = data.huaweicloud_images_image.centos_image.id
vpc_id = data.terraform_remote_state.network.outputs.vpc_id
subnet_id = data.terraform_remote_state.network.outputs.subnet_id
secgroup_id = data.terraform_remote_state.network.outputs.secgroup_id
instance_name = "node"
os_init_script = "system-init-script.sh"
keypair_name = "tf-node-keypair"
# SSH 公钥，公钥内容会写入 ecs 实例 /root/.ssh/authorized_keys 中，后续使用私钥登录节点即可
public_key = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCTrVri2QWeZZPEdS+PvxG9bdHswybJjrljA6FvsfVB4ji5dwT6jrBLXt+auhEHz0F7YO5esQcJ3x+j1trgE1adZ4wU9tYvQekUO48imJOwTVnbJzdCpYGAoYqHMbrTaRiY3ryeqabbq4KdKJdy7ooHYtENTkAVU31wduq2xpEGBl9V6A2Z0QIgkCIJIUDh5rXakVOFTvKZrdA8yxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxx@pc"
}
# 基于公钥创建密钥对
resource "huaweicloud_compute_keypair" "keypair" {
name = local.keypair_name
public_key = local.public_key
}
module "dev-ecs" {
source = "../../../modules/ecs"
count = local.counts
availability_zone = local.availability_zone
flavor_id = local.flavor_id
image_id = local.image_id
vpc_id = local.vpc_id
subnet_id = local.subnet_id
secgroup_id = local.secgroup_id
instance_name = "${local.instance_name}-${count.index}"
os_init_script = local.os_init_script
# 为 ecs 节点绑定关联密钥
keypair_name = local.keypair_name
}
data "huaweicloud_images_image" "centos_image" {
# 企业使用建议用 name 固定镜像名称，避免受 most_recent 影响，
# name_regex = "^CentOS 7.9 64bit$"
name = "CentOS 7.9 64bit"
visibility = "public"
architecture = "x86"
# most_recent 配置为 false 会导致操作，估计是 bug
most_recent = true
}
data "huaweicloud_compute_flavors" "flavor_1C1G" {
region = var.region
availability_zone = local.availability_zone
/* 规格类型：
normal 通用计算型
computingv3 通用计算增强型
highmem 内存优化型
saphana 超大内存型
diskintensive 磁盘密集型
*/
performance_type = "normal"
cpu_core_count = 1
memory_size = 1
}
# 这里 华为云似乎无法获取镜像列表，只能
#output "huaweicloud_images_image" {
# value = data.huaweicloud_images_image.centos_image
#}
# 输出符合条件的实例规格列表，从中选择满足需求且性价比高的实例
#output "ecs_flavor" {
# value = data.huaweicloud_compute_flavors.flavor_1C1G.ids
#}执行 tf init 拉取模块
terraform.exe init
Initializing modules...
- dev-ecs in ..\..\..\modules\ecs
# ...再次执行 tf apply 创建资源
控制台确认
eip
官方文档：https://registry.terraform.io/providers/huaweicloud/huaweicloud/latest/docs/resources/vpc_eip
上面的 ecs 虽然创建出，但是是没有公网访问能力的，如果节点本身就运行内网，而非 DMZ 之类需要访问公网的环境，那么可以直接跳过不申请 EIP
定义模块
versions.tf 不变
variables.tf
# modules/eip/variables.tf
variable "bandwidth_name" {}
variable "bandwidth_size" {}
variable "instances" {}outputs.tf
# modules/eip/outputs.tf
# 返回 bandwidth_id 供以 ELB 负载均衡资源使用
output "bandwidth_id" {
value = huaweicloud_vpc_bandwidth.bandwidth_1.id
}
# 返回 publicip 供以输出展示给用户查看
output "eip_public_ip" {
value = huaweicloud_vpc_eip.eip.publicip
}main.tf
# modules/eip/main.tf
# 共享带宽资源，后续 EIP 关联使用这个共享带宽
resource "huaweicloud_vpc_bandwidth" "bandwidth_1" {
name = var.bandwidth_name
# 5 5 Mbit/s to 2000 Mbit/s
size = var.bandwidth_size
}
# EIP 资源，count 循环绑定到 共享带宽资源
resource "huaweicloud_vpc_eip" "eip" {
# 使用 count 创建 ecs 实例数量相同的 eip 资源
count = length(var.instances)
publicip {
# 5_bgp 动态 bgp、5sbgp 静态 bgp
type = "5_bgp"
}
# 将 eip 关联绑定到共享带宽资源上
bandwidth {
# WHOLE 共享带宽、PER 专享带宽
share_type = "WHOLE"
id = huaweicloud_vpc_bandwidth.bandwidth_1.id
}
}
# 循环绑定 EIP <----> ECS
resource "huaweicloud_compute_eip_associate" "eip_associate" {
# 创建 ecs eip_associate 绑定关系资源
count = length(var.instances)
# huaweicloud_vpc_eip.eip[*].address 获取所有 eip 对象的地址列表（公网），按照迭代索引选择 eip 公网地址
public_ip = huaweicloud_vpc_eip.eip[*].address[count.index]
# 按照迭代索引选择获取实例 id
instance_id = var.instances[count.index]
}定义资源
创建 env/dev/service/eip.tf
# env/dev/service/eip.tf
locals {
bandwidth_name = "dev-bandwidth"
bandwidth_size = 5
# 获取所有 ecs实例 ID
instances = module.dev-ecs[*].instance_id
}
module "dev-eip" {
source = "../../../modules/eip"
bandwidth_name = local.bandwidth_name
bandwidth_size = local.bandwidth_size
instances = local.instances
}初始化 tf init
terraform.exe init
Initializing modules...
- dev-eip in ..\..\..\modules\eip
Initializing the backend...
Initializing provider plugins...
- terraform.io/builtin/terraform is built in to Terraform
- Reusing previous version of huaweicloud/huaweicloud from the dependency lock file
- Reusing previous version of hashicorp/template from the dependency lock file
- Using previously-installed huaweicloud/huaweicloud v1.47.1
- Using previously-installed hashicorp/template v2.2.0
# ...创建资源 tf apply，请确保 ecs 实例已创建完毕
terraform.exe apply -auto-approve
huaweicloud_compute_keypair.keypair: Refreshing state... [id=tf-node-keypair]
module.dev-ecs[0].huaweicloud_compute_instance.basic: Refreshing state... [id=645ae0f7-704a-4097-9003-cce2020084f9]
module.dev-ecs[1].huaweicloud_compute_instance.basic: Refreshing state... [id=b052e963-9890-43f9-b33c-18622344b79d]
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[0] will be created
+ resource "huaweicloud_compute_eip_associate" "eip_associate" {
+ fixed_ip = (known after apply)
+ id = (known after apply)
+ instance_id = "645ae0f7-704a-4097-9003-cce2020084f9"
+ port_id = (known after apply)
+ public_ip = (known after apply)
+ region = (known after apply)
}
# module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[1] will be created
+ resource "huaweicloud_compute_eip_associate" "eip_associate" {
+ fixed_ip = (known after apply)
+ id = (known after apply)
+ instance_id = "b052e963-9890-43f9-b33c-18622344b79d"
+ port_id = (known after apply)
+ public_ip = (known after apply)
+ region = (known after apply)
}
# module.dev-eip.huaweicloud_vpc_bandwidth.bandwidth_1 will be created
+ resource "huaweicloud_vpc_bandwidth" "bandwidth_1" {
+ bandwidth_type = (known after apply)
+ charge_mode = (known after apply)
+ enterprise_project_id = (known after apply)
+ id = (known after apply)
+ name = "dev-bandwidth"
+ publicips = (known after apply)
+ region = (known after apply)
+ share_type = (known after apply)
+ size = 5
+ status = (known after apply)
}
# module.dev-eip.huaweicloud_vpc_eip.eip[0] will be created
+ resource "huaweicloud_vpc_eip" "eip" {
+ address = (known after apply)
+ charging_mode = (known after apply)
+ enterprise_project_id = (known after apply)
+ id = (known after apply)
+ ipv6_address = (known after apply)
+ port_id = (known after apply)
+ private_ip = (known after apply)
+ region = (known after apply)
+ status = (known after apply)
+ bandwidth {
+ charge_mode = (known after apply)
+ id = (known after apply)
+ name = (known after apply)
+ share_type = "WHOLE"
+ size = (known after apply)
}
+ publicip {
+ ip_address = (known after apply)
+ ip_version = (known after apply)
+ port_id = (known after apply)
+ type = "5_bgp"
}
}
# module.dev-eip.huaweicloud_vpc_eip.eip[1] will be created
+ resource "huaweicloud_vpc_eip" "eip" {
+ address = (known after apply)
+ charging_mode = (known after apply)
+ enterprise_project_id = (known after apply)
+ id = (known after apply)
+ ipv6_address = (known after apply)
+ port_id = (known after apply)
+ private_ip = (known after apply)
+ region = (known after apply)
+ status = (known after apply)
+ bandwidth {
+ charge_mode = (known after apply)
+ id = (known after apply)
+ name = (known after apply)
+ share_type = "WHOLE"
+ size = (known after apply)
}
+ publicip {
+ ip_address = (known after apply)
+ ip_version = (known after apply)
+ port_id = (known after apply)
+ type = "5_bgp"
}
}
Plan: 5 to add, 0 to change, 0 to destroy.
module.dev-eip.huaweicloud_vpc_bandwidth.bandwidth_1: Creating...
module.dev-eip.huaweicloud_vpc_bandwidth.bandwidth_1: Creation complete after 4s [id=f9999a46-ed3e-40a1-a9a7-52ea725c4274]
module.dev-eip.huaweicloud_vpc_eip.eip[1]: Creating...
module.dev-eip.huaweicloud_vpc_eip.eip[0]: Creating...
module.dev-eip.huaweicloud_vpc_eip.eip[0]: Creation complete after 7s [id=c7925fae-7a38-4c8c-af07-1186e9f7ca1a]
module.dev-eip.huaweicloud_vpc_eip.eip[1]: Creation complete after 7s [id=078da343-959c-4992-ba1e-98447bb45f30]
module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[0]: Creating...
module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[1]: Creating...
module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[0]: Creation complete after 8s [id=49.4.48.245/645ae0f7-704a-4097-9003-cce2020084f9/172.16.0.149]
module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[1]: Creation complete after 8s [id=49.4.48.144/b052e963-9890-43f9-b33c-18622344b79d/172.16.0.92]
Apply complete! Resources: 5 added, 0 changed, 0 destroyed.
Outputs:
ecs_ids = [
"645ae0f7-704a-4097-9003-cce2020084f9",
"b052e963-9890-43f9-b33c-18622344b79d",
]
ecs_image = [
"CentOS 7.9 64bit",
"CentOS 7.9 64bit",
]
ip_ipv4 = [
"172.16.0.149",
"172.16.0.92",
]
public_ip = [
tostring(null),
tostring(null),
]控制台确认
稍微修饰下 outputs 信息
# env/dev/service/outputs.tf
output "ecs_ids" {
value = module.dev-ecs[*].instance_id
}
output "ecs_image" {
value = module.dev-ecs[*].instance_image_name
}
output "ip_ipv4" {
value = module.dev-ecs[*].instance_ip_ipv4
}
output "eip_public_ip" {
value = module.dev-eip.*.eip_public_ip
}再次执行 tf plan
Changes to Outputs:
+ eip_public_ip = [
+ [
+ "49.4.48.245",
+ "49.4.48.144",
],
]
# ...使用证书登录 node 安装配置 nginx 服务，因为之前无法访问外网，所以 user-data 脚本执行失败，这里手动处理下，也可以登录到节点重新执行 user-data 脚本，如下
$ cd /var/lib/cloud/instances/<instance_id>/scripts/
$ ls
part-001
$ cat part-001
#!/bin/sh
yum -y install nginx
systemctl enable nginx --now
echo `hostname` > /usr/share/nginx/html/index.html
$ ./part-001 两个节点配置完后，测试外网访问
$ curl 49.4.48.245
node-0
$ curl 49.4.48.144
node-1elb
定义模块
variables.tf
variable "subnet_id" {}
variable "instance_ip_list" {}
variable "lb_listener_port" {}
variable "lb_member_port" {}outputs.tf
# 输出 elb_vip_port_id，用以创建 eip 与 elb 关联
output "elb_vip_port_id" {
value = huaweicloud_lb_loadbalancer.lb.vip_port_id
}
# 输出 elb 外网地址，用以让用户查看
output "elb_vip_address" {
value = huaweicloud_lb_loadbalancer.lb.vip_address
}main.tf
# 负载均衡资源
resource "huaweicloud_lb_loadbalancer" "lb" {
vip_subnet_id = var.subnet_id
}
# 负载均衡监听器资源
resource "huaweicloud_lb_listener" "listener" {
loadbalancer_id = huaweicloud_lb_loadbalancer.lb.id
protocol = "HTTP"
# ELB 负载均衡监听端口
protocol_port = var.lb_listener_port
}
# 后端服务器组
resource "huaweicloud_lb_pool" "pool" {
lb_method = "ROUND_ROBIN"
protocol = "HTTP"
listener_id = huaweicloud_lb_listener.listener.id
# 一致性规则
persistence {
type = "APP_COOKIE"
cookie_name = "dev-cookie"
}
}
resource "huaweicloud_lb_member" "member" {
count = length(var.instance_ip_list)
subnet_id = var.subnet_id
address = var.instance_ip_list[count.index]
pool_id = huaweicloud_lb_pool.pool.id
# ECS 服务器监听端口
protocol_port = var.lb_member_port
}定义资源
elb.tf
locals {
# 获取 实例内网 IP
instance_ip_list = module.dev-ecs.*.instance_ip_ipv4
lb_listener_port = 80
lb_member_port = 80
}
module "dev-elb" {
source = "../../../modules/elb"
/* 区分说明下
subnet.id 网络 ID
subnet_id VPC 子网 ID
*/
subnet_id = data.terraform_remote_state.network.outputs.subnet_subnet_id
instance_ip_list = local.instance_ip_list
lb_listener_port = local.lb_listener_port
lb_member_port = local.lb_member_port
}
# 为 EIB 创建 EIP，关联共享带宽
resource "huaweicloud_vpc_eip" "elb_eip" {
publicip {
type = "5_bgp"
}
bandwidth {
share_type = "WHOLE"
# 这里用的还是共享带宽
id = module.dev-eip.bandwidth_id
}
}
# 绑定 eip <---> 负载均衡 vip 监听端口
resource "huaweicloud_networking_eip_associate" "eip_elb_associate" {
port_id = module.dev-elb.elb_vip_port_id
public_ip = huaweicloud_vpc_eip.elb_eip.address
}初始化
$ tf init创建资源
terraform.exe apply -auto-approve
module.dev-eip.huaweicloud_vpc_bandwidth.bandwidth_1: Refreshing state... [id=f9999a46-ed3e-40a1-a9a7-52ea725c4274]
huaweicloud_compute_keypair.keypair: Refreshing state... [id=tf-node-keypair]
module.dev-elb.huaweicloud_lb_loadbalancer.lb: Refreshing state... [id=229e123b-0b68-4278-86ef-713d0670488f]
module.dev-elb.huaweicloud_lb_listener.listener: Refreshing state... [id=09949f34-bb2d-4137-bb6a-f56f27dac201]
module.dev-ecs[1].huaweicloud_compute_instance.basic: Refreshing state... [id=b052e963-9890-43f9-b33c-18622344b79d]
module.dev-ecs[0].huaweicloud_compute_instance.basic: Refreshing state... [id=645ae0f7-704a-4097-9003-cce2020084f9]
module.dev-elb.huaweicloud_lb_pool.pool: Refreshing state... [id=76997134-ab81-466f-8472-eba9036b05ae]
module.dev-eip.huaweicloud_vpc_eip.eip[0]: Refreshing state... [id=c7925fae-7a38-4c8c-af07-1186e9f7ca1a]
module.dev-eip.huaweicloud_vpc_eip.eip[1]: Refreshing state... [id=078da343-959c-4992-ba1e-98447bb45f30]
module.dev-elb.huaweicloud_lb_member.member[1]: Refreshing state... [id=d0fc9a8c-6e47-41ed-9f50-04e9f648c848]
module.dev-elb.huaweicloud_lb_member.member[0]: Refreshing state... [id=f8d61812-abe4-4719-b801-f0654701c45d]
module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[1]: Refreshing state... [id=49.4.48.144/b052e963-9890-43f9-b33c-18622344b79d/172.16.0.92]
module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[0]: Refreshing state... [id=49.4.48.245/645ae0f7-704a-4097-9003-cce2020084f9/172.16.0.149]
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# huaweicloud_networking_eip_associate.eip_elb_associate will be created
+ resource "huaweicloud_networking_eip_associate" "eip_elb_associate" {
+ fixed_ip = (known after apply)
+ id = (known after apply)
+ mac_address = (known after apply)
+ network_id = (known after apply)
+ port_id = "35e91d3b-15d4-4d72-967d-84bcec6aa5f6"
+ public_ip = (known after apply)
+ region = (known after apply)
+ status = (known after apply)
}
# huaweicloud_vpc_eip.elb_eip will be created
+ resource "huaweicloud_vpc_eip" "elb_eip" {
+ address = (known after apply)
+ charging_mode = (known after apply)
+ enterprise_project_id = (known after apply)
+ id = (known after apply)
+ ipv6_address = (known after apply)
+ port_id = (known after apply)
+ private_ip = (known after apply)
+ region = (known after apply)
+ status = (known after apply)
+ bandwidth {
+ charge_mode = (known after apply)
+ id = "f9999a46-ed3e-40a1-a9a7-52ea725c4274"
+ name = (known after apply)
+ share_type = "WHOLE"
+ size = (known after apply)
}
+ publicip {
+ ip_address = (known after apply)
+ ip_version = (known after apply)
+ port_id = (known after apply)
+ type = "5_bgp"
}
}
Plan: 2 to add, 0 to change, 0 to destroy.
Changes to Outputs:
+ elb_eip_public_ip = (known after apply)
huaweicloud_vpc_eip.elb_eip: Creating...
huaweicloud_vpc_eip.elb_eip: Creation complete after 7s [id=691f16eb-c3b1-40ff-87ce-0a934cdec5e4]
huaweicloud_networking_eip_associate.eip_elb_associate: Creating...
huaweicloud_networking_eip_associate.eip_elb_associate: Still creating... [10s elapsed]
huaweicloud_networking_eip_associate.eip_elb_associate: Creation complete after 13s [id=691f16eb-c3b1-40ff-87ce-0a934cdec5e4]
Apply complete! Resources: 2 added, 0 changed, 0 destroyed.
Outputs:
ecs_ids = [
"645ae0f7-704a-4097-9003-cce2020084f9",
"b052e963-9890-43f9-b33c-18622344b79d",
]
ecs_image = [
"CentOS 7.9 64bit",
"CentOS 7.9 64bit",
]
eip_public_ip = [
[
"49.4.48.245",
"49.4.48.144",
],
]
elb_eip_public_ip = "49.4.53.202"
ip_ipv4 = [
"172.16.0.149",
"172.16.0.92",
]访问 elb_eip 公网地址，检查 lb 效果
$ curl 49.4.53.202
node-1
$ curl 49.4.53.202
node-0
$ curl 49.4.53.202
node-1
$ curl 49.4.53.202
node-1OK，符合预期~
dns
定义模块
由于我唯一经过 ICP 备案的域名是阿里云平台的，直接调用已经写好并存在远程的 dns 模块，创建 DNS 解析记录即可
定义资源
这里会用到 aliyun 的 provider，去到 versions.tf 中定义下
terraform {
required_providers {
huaweicloud = {
source = "huaweicloud/huaweicloud"
version = ">=1.36.0"
}
alicloud = {
source = "hashicorp/alicloud"
version = "1.197.0"
}
}
}相关的 AK、SK 变量，使用环境变量的方式进行设置
export TF_VAR_ALICLOUD_ACCESS_KEY="xxxx"
export TF_VAR_ALICLOUD_SECRET_KEY="xxx"
export TF_VAR_ALICLOUD_REGION="cn-beijing"创建 env/dev/service/dns.tf 这里的资源指的就是 DNS 解析记录，如下所示
# env/dev/service/dns.tf
provider "alicloud" {
region = "cn-beijing"
}
locals {
domain_name = "yo-yo.fun"
host_record = "dev.demo"
record_type = "A"
ip_address = module.dev-eip.eip_public_ip
# 开启禁用解析记录 ENABLE、DISABLE
record_status = "ENABLE"
ttl = "600"
}
module "dev-dns" {
source = "git::ssh://git@e.coding.net/LotusChing/terraform/terraform-aliyun-modules.git//dns"
domain_name = local.domain_name
host_record = local.host_record
record_type = local.record_type
ip_address = local.ip_address
record_status = local.record_status
ttl = local.ttl
}初始化，拉取远程模块 dns
terraform.exe init
Initializing modules...
Downloading git::ssh://git@e.coding.net/LotusChing/terraform/terraform-aliyun-modules.git for dev-dns...
- dev-dns in .terraform\modules\dev-dns\dns
Initializing the backend...
Initializing provider plugins...
- terraform.io/builtin/terraform is built in to Terraform
- Reusing previous version of hashicorp/template from the dependency lock file
- Reusing previous version of huaweicloud/huaweicloud from the dependency lock file
- Finding hashicorp/alicloud versions matching "1.197.0"...
- Using previously-installed hashicorp/template v2.2.0
- Using previously-installed huaweicloud/huaweicloud v1.47.1
- Using hashicorp/alicloud v1.197.0 from the shared cache directory
Terraform has made some changes to the provider dependency selections recorded
in the .terraform.lock.hcl file. Review those changes and commit them to your
version control system if they represent changes you intended to make.
╷
│ Warning: Additional provider information from registry
│
│ The remote registry returned warnings for
│ registry.terraform.io/hashicorp/alicloud:
│ - For users on Terraform 0.13 or greater, this provider has moved to
│ aliyun/alicloud. Please update your source in required_providers.
╵
Terraform has been successfully initialized!这边先不急着创建资源，为了验证上面所有组件资源的定义正确，这次我们把 service 下的都释放掉，等下彻底串联下整个过程，先释放 service 再释放 network 下资源
这样，我们从头到尾的过一遍整个流程，看能否跑通
$ cd env/dev/network; tf destroy
$ cd env/dev/service; tf destroy
module.dev-vpc.huaweicloud_vpc.vpc: Still destroying... [id=506afa4d-7a05-4f29-88b7-80d14ffd62d9, 2m50s elapsed]
╷
│ Error: error deleting VPC 506afa4d-7a05-4f29-88b7-80d14ffd62d9: context deadline exceeded这里迟迟未能删除 vpc，这是因为前面我们通过 WebUI 以内网地址访问服务器，华为云会默认替我们创建一条对应 终端节点服务 记录（登录过几台，就会生成几条），所以在删除 VPC 时会超时失败
归根到底，还是尽量避免 WebUI 上操作，避免出现配置漂移的现象
手动清理终端节点服务记录，然后尝试删除 vpc 即可
global 部分我们不清理，因为资源变更的会反映到 tfstate 状态文件，而我们的状态文件是放在 obs 中，如果 obs 资源销毁的话，那么状态文件就没地方写了。
华为云这边也也做了额外的判断，如果你本身仍配置使用 s3 作为 remote backend，那么资源是不会删除的，这点儿和阿里云不一样，所以我们得先把 backend.tf 全部注释掉，然后从 s3 obs 中把对应 backend 对应的 tfstate 下载到本地，然后重新初始化，声明我现在不用 s3 了
terraform.exe init -migrate-state
Initializing the backend...
Terraform has detected you're unconfiguring your previously set "s3" backend.
Successfully unset the backend "s3". Terraform will now operate locally.
Initializing provider plugins...
- Reusing previous version of huaweicloud/huaweicloud from the dependency lock file
- Using previously-installed huaweicloud/huaweicloud v1.47.1
Terraform has been successfully initialized!通过 Web 控制台清空 obs 中的目录及文件，然后在尝试 tf destroy 销毁资源
terraform.exe destroy -auto-approve
huaweicloud_obs_bucket.bucket: Refreshing state... [id=tf-dayo-backend-bucket]
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
- destroy
Terraform will perform the following actions:
# huaweicloud_obs_bucket.bucket will be destroyed
- resource "huaweicloud_obs_bucket" "bucket" {
- acl = "private" -> null
- bucket = "tf-dayo-backend-bucket" -> null
- bucket_domain_name = "tf-dayo-backend-bucket.obs.cn-north-1.myhuaweicloud.com" -> null
- bucket_version = "3.0" -> null
- encryption = false -> null
- enterprise_project_id = "0" -> null
- force_destroy = false -> null
- id = "tf-dayo-backend-bucket" -> null
- multi_az = false -> null
- parallel_fs = false -> null
- policy_format = "obs" -> null
- quota = 0 -> null
- region = "cn-north-1" -> null
- storage_class = "STANDARD" -> null
- storage_info = [
- {
- object_number = 0
- size = 0
},
] -> null
- tags = {
- "type" = "bucket"
} -> null
- versioning = false -> null
}
Plan: 0 to add, 0 to change, 1 to destroy.
Changes to Outputs:
- bucket_domain_name = "tf-dayo-backend-bucket.obs.cn-north-1.myhuaweicloud.com" -> null
- bucket_name = "tf-dayo-backend-bucket" -> null
huaweicloud_obs_bucket.bucket: Destroying... [id=tf-dayo-backend-bucket]
huaweicloud_obs_bucket.bucket: Destruction complete after 1s
Destroy complete! Resources: 1 destroyed.三、全资源串联检查
0. 共享模块
当前目录结构如下
├── env
│ └── dev
│ ├── network
│ │ ├── backend.tf
│ │ ├── main.tf
│ │ ├── outputs.tf
│ │ ├── secgroup.tf
│ │ ├── variables.tf
│ │ ├── versions.tf
│ │ └── vpc.tf
│ └── service
│ ├── backend.tf
│ ├── dns.tf
│ ├── ecs.tf
│ ├── eip.tf
│ ├── elb.tf
│ ├── main.tf
│ ├── outputs.tf
│ ├── system-init-script.sh
│ ├── variables.tf
│ └── versions.tf
├── global
│ ├── backend
│ │ ├── backend.tf
│ │ ├── main.tf
│ │ ├── obs.tf
│ │ ├── outputs.tf
│ │ ├── terraform.tfstate
│ │ ├── variables.tf
│ │ └── versions.tf
│ └── terraform.tfstate
└── modules
├── ecs
│ ├── main.tf
│ ├── outputs.tf
│ ├── variables.tf
│ └── versions.tf
├── eip
│ ├── main.tf
│ ├── outputs.tf
│ ├── variables.tf
│ └── versions.tf
├── elb
│ ├── main.tf
│ ├── outputs.tf
│ ├── variables.tf
│ └── versions.tf
├── secgroup
│ ├── main.tf
│ ├── outputs.tf
│ ├── variables.tf
│ └── versions.tf
└── vpc
├── main.tf
├── outputs.tf
├── variables.tf
└── versions.tf
12 directories, 45 files为了让其他项目复用 vpc、dns、eip、elb 等资源，通常我们会比较通用的资源模块 modules 从项目中抽离出来，存放在内部的 Git 平台，这里也实践一下
具体操作过程很简单，创建 terraform-huaweiyun-modules Git 仓库，拉取到本地后，将 modules 目录中的内容复制（移动）过去，add/commit/push 即可
调整后的目录结构
├── env
│ └── dev
│ ├── network
│ │ ├── backend.tf
│ │ ├── main.tf
│ │ ├── outputs.tf
│ │ ├── secgroup.tf
│ │ ├── variables.tf
│ │ ├── versions.tf
│ │ └── vpc.tf
│ └── service
│ ├── backend.tf
│ ├── dns.tf
│ ├── ecs.tf
│ ├── eip.tf
│ ├── elb.tf
│ ├── main.tf
│ ├── outputs.tf
│ ├── system-init-script.sh
│ ├── variables.tf
│ └── versions.tf
└── global
├── backend
│ ├── backend.tf
│ ├── main.tf
│ ├── obs.tf
│ ├── outputs.tf
│ ├── terraform.tfstate
│ ├── variables.tf
│ └── versions.tf
└── terraform.tfstate
6 directories, 25 files修改创建资源时引用模块的方式，从原本的本地引用，改为通过 git 的远程调用
vpc.tf
# env/dev/network/vpc.tf
locals {
vpc_cidr = "172.16.0.0/12"
vpc_name = "dev-vpc"
subnet_name = "dev-subnet"
subnet_cidr = "172.16.0.0/24"
subnet_gateway_ip = "172.16.0.1"
availability_zone = "cn-north-1a"
}
module "dev-vpc" {
source = "git::ssh://git@e.coding.net/LotusChing/terraform/terraform-huaweiyun-modules.git//vpc"
vpc_name = local.vpc_name
vpc_cidr = local.vpc_cidr
subnet_name = local.subnet_name
subnet_cidr = local.subnet_cidr
subnet_gateway_ip = local.subnet_gateway_ip
availability_zone = local.availability_zone
}secgroup.tf
# env/dev/network/secgroup.tf
locals {
sec_group_name = "dev_secgroup"
sec_group_desc = "dev secgroup for terraform"
}
module "dev-secgroup" {
source = "git::ssh://git@e.coding.net/LotusChing/terraform/terraform-huaweiyun-modules.git//secgroup"
sec_group_name = local.sec_group_name
sec_group_desc = local.sec_group_desc
}ecs.tf
# env/dev/service/ecs.tf
locals {
availability_zone = "cn-north-1a"
counts = 2
# 从查询结果列表中选出第一个符合条件的实例规则
flavor_id = data.huaweicloud_compute_flavors.flavor_1C1G.ids[0]
image_id = data.huaweicloud_images_image.centos_image.id
vpc_id = data.terraform_remote_state.network.outputs.vpc_id
subnet_id = data.terraform_remote_state.network.outputs.subnet_id
secgroup_id = data.terraform_remote_state.network.outputs.secgroup_id
instance_name = "node"
admin_password = "Adm1n123"
os_init_script = "system-init-script.sh"
keypair_name = "tf-node-keypair"
public_key = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCTrVri2QWeZZPEdS+PvxG9bdHswybJjrljA6FvsfVB4ji5dwT6jrBLXt+auhEHz0F7YO5esQcJ3x+j1trgE1adZ4wU9tYvQekUO48imJOwTVnbJzdCpYGAoYqHMbrTaRiY3ryeqabbq4KdKJdy7ooHYtENTkAVU31wduq2xpEGBl9V6A2Z0QIgkCIJIUDh5rXakVOFTvKZrdA8y+aRnGt2lQ/JLoDJOU9n3nRNw757hRr048I6bQojXqx3F/y4IeQhQ2XTZ/4tbPyF6+vjEtbeWCkVYHbmq3StEcesOOc88ZhwKWrjtej67wU+V5C3DFoGF03a6IjQ8Zn6cII6vZwx liucong@lc"
}
# 基于公钥创建密钥对
resource "huaweicloud_compute_keypair" "keypair" {
name = local.keypair_name
public_key = local.public_key
}
module "dev-ecs" {
source = "git::ssh://git@e.coding.net/LotusChing/terraform/terraform-huaweiyun-modules.git//ecs"
count = local.counts
availability_zone = local.availability_zone
flavor_id = local.flavor_id
image_id = local.image_id
vpc_id = local.vpc_id
subnet_id = local.subnet_id
secgroup_id = local.secgroup_id
instance_name = "${local.instance_name}-${count.index}"
os_init_script = local.os_init_script
keypair_name = local.keypair_name
}
data "huaweicloud_images_image" "centos_image" {
# 企业使用建议用 name 固定镜像名称，避免受 most_recent 影响，
# name_regex = "^CentOS 7.9 64bit$"
name = "CentOS 7.9 64bit"
visibility = "public"
architecture = "x86"
# most_recent 配置为 false 会导致操作，估计是 bug
most_recent = true
}
data "huaweicloud_compute_flavors" "flavor_1C1G" {
region = var.region
availability_zone = local.availability_zone
/* 规格类型：
normal 通用计算型
computingv3 通用计算增强型
highmem 内存优化型
saphana 超大内存型
diskintensive 磁盘密集型
*/
performance_type = "normal"
cpu_core_count = 1
memory_size = 1
}
# 这里 华为云似乎无法获取镜像列表，只能
#output "huaweicloud_images_image" {
# value = data.huaweicloud_images_image.centos_image
#}
# 输出符合条件的实例规格列表，从中选择满足需求且性价比高的实例
#output "ecs_flavor" {
# value = data.huaweicloud_compute_flavors.flavor_1C1G.ids
#}eip.tf
# env/dev/service/eip.tf
locals {
bandwidth_name = "dev-bandwidth"
bandwidth_size = 5
# 获取所有 ecs实例 ID
instances = module.dev-ecs.*.instance_id
}
module "dev-eip" {
source = "git::ssh://git@e.coding.net/LotusChing/terraform/terraform-huaweiyun-modules.git//eip"
bandwidth_name = local.bandwidth_name
bandwidth_size = local.bandwidth_size
instances = local.instances
depends_on = [module.dev-ecs]
}elb.tf
# env/dev/service/elb.tf
locals {
# 获取 实例内网 IP
instance_ip_list = module.dev-ecs.*.instance_ip_ipv4
lb_listener_port = 80
lb_member_port = 80
}
module "dev-elb" {
source = "git::ssh://git@e.coding.net/LotusChing/terraform/terraform-huaweiyun-modules.git//elb"
/* 区分说明下
华为云这点弄得不是很友好容易混淆，网络 ID 与 子网 ID 都是 subnet* 标识
subnet_subnet_id VPC 子网 ID
*/
subnet_id = data.terraform_remote_state.network.outputs.subnet_subnet_id
instance_ip_list = local.instance_ip_list
lb_listener_port = local.lb_listener_port
lb_member_port = local.lb_member_port
depends_on = [module.dev-eip]
}
# 为 EIB 创建 EIP，关联共享带宽
resource "huaweicloud_vpc_eip" "elb_eip" {
publicip {
type = "5_bgp"
}
bandwidth {
share_type = "WHOLE"
# 这里用的还是共享带宽
id = module.dev-eip.bandwidth_id
}
}
# 绑定 eip <---> 负载均衡 vip 监听端口
resource "huaweicloud_networking_eip_associate" "eip_elb_associate" {
port_id = module.dev-elb.elb_vip_port_id
public_ip = huaweicloud_vpc_eip.elb_eip.address
depends_on = [module.dev-elb]
}dnf.tf
# env/dev/service/dns.tf
locals {
domain_name = "yo-yo.fun"
host_record = "dev.demo"
record_type = "A"
ip_address = huaweicloud_vpc_eip.elb_eip.address
# 开启禁用解析记录 ENABLE、DISABLE
record_status = "ENABLE"
ttl = "600"
}
module "dev-dns" {
source = "git::ssh://git@e.coding.net/LotusChing/terraform/terraform-aliyun-modules.git//dns"
domain_name = local.domain_name
host_record = local.host_record
record_type = local.record_type
ip_address = local.ip_address
record_status = local.record_status
ttl = local.ttl
depends_on = [huaweicloud_networking_eip_associate.eip_elb_associate]
}这里再补充下 network、service 的 outputs.tf 内容
service/outputs.tf
# env/dev/service/outputs.tf
output "ecs_ids" {
value = module.demo7-dev-ecs[*].ecs_id
}
output "ecs_images" {
value = module.demo7-dev-ecs[*].ecs_image_id
}
output "ecs_spec" {
value = module.demo7-dev-ecs[*].ecs_spec
}
output "ecs_public_ip" {
value = module.demo7-dev-ecs[*].ecs_public_ip
}
output "eip_address" {
value = module.demo7-dev-eip.eip_ipaddress
}
output "web_url" {
value = module.demo7-dev-dns.web_access_url
}network/outputs.tf
# env/dev/network/outputs.tf
output "vpc_id" {
value = module.demo7-dev-vpc.vpc_id
}
output "vsw_id" {
value = module.demo7-dev-vpc.vsw_id
}
output "sg_id" {
value = module.demo7-dev-sg.sg_id
}1. Backend：obs 创建
在最开始，初次执行初始化 backend 目录下的资源时，由于华为云还未创建 obs 资源，所以一定要注释 backend.tf 配置，生成本地 tfstate 文件
不然百分比会报下面的错误
Initializing the backend...
╷
│ Error: Failed to get existing workspaces: S3 bucket does not exist.
│
│ The referenced S3 bucket must have been previously created. If the S3 bucket
│ was created within the last minute, please wait for a minute or two and try
│ again.
│
│ Error: NoSuchBucket: The specified bucket does not exist
│ status code: 404, request id: 000001878852BC7984C91C2D1B666357, host id: 32AAAQAAEAABAAAQAAEAABAAAQAAEAABCSTGzqfGPNEag1Ee+MBf92+rK00f/zIc初始化 backend 目录，生成 tfstate
terraform.exe init
Initializing the backend...
Initializing provider plugins...
- Finding huaweicloud/huaweicloud versions matching ">= 1.36.0"...
- Using huaweicloud/huaweicloud v1.47.1 from the shared cache directory
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.
Terraform has been successfully initialized!
# ...状态文件 .terraform/terraform.tfstate 生成，完毕
接下来，创建 obs 资源
terraform.exe apply -auto-approve
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# huaweicloud_obs_bucket.bucket will be created
+ resource "huaweicloud_obs_bucket" "bucket" {
+ acl = "private"
+ bucket = "tf-dayo-backend-bucket"
+ bucket_domain_name = (known after apply)
+ bucket_version = (known after apply)
+ encryption = false
+ enterprise_project_id = (known after apply)
+ force_destroy = false
+ id = (known after apply)
+ kms_key_project_id = (known after apply)
+ multi_az = (known after apply)
+ policy = (known after apply)
+ policy_format = "obs"
+ quota = 0
+ region = (known after apply)
+ storage_class = "STANDARD"
+ storage_info = (known after apply)
+ tags = {
+ "type" = "bucket"
}
+ versioning = false
}
Plan: 1 to add, 0 to change, 0 to destroy.
Changes to Outputs:
+ bucket_domain_name = (known after apply)
+ bucket_name = "tf-dayo-backend-bucket"
huaweicloud_obs_bucket.bucket: Creating...
huaweicloud_obs_bucket.bucket: Creation complete after 1s [id=tf-dayo-backend-bucket]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Outputs:
bucket_domain_name = "tf-dayo-backend-bucket.obs.cn-north-1.myhuaweicloud.com"
bucket_name = "tf-dayo-backend-bucket"obs 创建完毕后，就可以配置使用远程 obs 作为 backend 存储状态文件，取消 backend.tf，执行 tf init
terraform.exe init
Initializing the backend...
Do you want to copy existing state to the new backend?
Pre-existing state was found while migrating the previous "local" backend to the
newly configured "s3" backend. No existing state was found in the newly
configured "s3" backend. Do you want to copy this state to the new "s3"
backend? Enter "yes" to copy and "no" to start with an empty state.
Enter a value: yes
Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing provider plugins...
- Reusing previous version of huaweicloud/huaweicloud from the dependency lock file
- Using previously-installed huaweicloud/huaweicloud v1.47.1
Terraform has been successfully initialized!
# ...可以看到，本地状态文件已经上传到了 obs 远程 backend 中，此时可以删除掉 global/terraform.tfstate 与 global/terraform.tfstate.backup 两个文件了
2. 底层网络资源：vpc、secgroup 创建
初始化 tf init，由于配置使用的是远程模块，这里会看到下载模块的信息
terraform.exe init
Initializing modules...
Downloading git::ssh://git@e.coding.net/LotusChing/terraform/terraform-huaweiyun-modules.git for dev-secgroup...
- dev-secgroup in .terraform\modules\dev-secgroup\secgroup
Downloading git::ssh://git@e.coding.net/LotusChing/terraform/terraform-huaweiyun-modules.git for dev-vpc...
- dev-vpc in .terraform\modules\dev-vpc\vpc
Initializing the backend...
Initializing provider plugins...
- Finding huaweicloud/huaweicloud versions matching ">= 1.36.0"...
- Using huaweicloud/huaweicloud v1.47.1 from the shared cache directory
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.
Terraform has been successfully initialized!
# ...检查确认 plan，创建资源 tf apply
# tf plan
# ...
terraform.exe apply -auto-approve
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# module.dev-secgroup.huaweicloud_networking_secgroup.secgroup will be created
+ resource "huaweicloud_networking_secgroup" "secgroup" {
+ created_at = (known after apply)
+ description = "dev secgroup for terraform"
+ enterprise_project_id = (known after apply)
+ id = (known after apply)
+ name = "dev_secgroup"
+ region = (known after apply)
+ rules = (known after apply)
+ updated_at = (known after apply)
}
# module.dev-secgroup.huaweicloud_networking_secgroup_rule.secgroup_rule_in_22 will be created
+ resource "huaweicloud_networking_secgroup_rule" "secgroup_rule_in_22" {
+ action = (known after apply)
+ direction = "ingress"
+ ethertype = "IPv4"
+ id = (known after apply)
+ port_range_max = 22
+ port_range_min = 22
+ ports = (known after apply)
+ priority = (known after apply)
+ protocol = "tcp"
+ region = (known after apply)
+ remote_address_group_id = (known after apply)
+ remote_group_id = (known after apply)
+ remote_ip_prefix = "0.0.0.0/0"
+ security_group_id = (known after apply)
}
# module.dev-secgroup.huaweicloud_networking_secgroup_rule.secgroup_rule_in_80 will be created
+ resource "huaweicloud_networking_secgroup_rule" "secgroup_rule_in_80" {
+ action = (known after apply)
+ direction = "ingress"
+ ethertype = "IPv4"
+ id = (known after apply)
+ port_range_max = 80
+ port_range_min = 80
+ ports = (known after apply)
+ priority = (known after apply)
+ protocol = "tcp"
+ region = (known after apply)
+ remote_address_group_id = (known after apply)
+ remote_group_id = (known after apply)
+ remote_ip_prefix = "0.0.0.0/0"
+ security_group_id = (known after apply)
}
# module.dev-secgroup.huaweicloud_networking_secgroup_rule.secgroup_rule_out_all will be created
+ resource "huaweicloud_networking_secgroup_rule" "secgroup_rule_out_all" {
+ action = (known after apply)
+ direction = "egress"
+ ethertype = "IPv4"
+ id = (known after apply)
+ port_range_max = 65535
+ port_range_min = 1
+ ports = (known after apply)
+ priority = (known after apply)
+ protocol = "tcp"
+ region = (known after apply)
+ remote_address_group_id = (known after apply)
+ remote_group_id = (known after apply)
+ remote_ip_prefix = "0.0.0.0/0"
+ security_group_id = (known after apply)
}
# module.dev-vpc.huaweicloud_vpc.vpc will be created
+ resource "huaweicloud_vpc" "vpc" {
+ cidr = "172.16.0.0/12"
+ enterprise_project_id = (known after apply)
+ id = (known after apply)
+ name = "dev-vpc"
+ region = (known after apply)
+ routes = (known after apply)
+ status = (known after apply)
}
# module.dev-vpc.huaweicloud_vpc_subnet.subnet will be created
+ resource "huaweicloud_vpc_subnet" "subnet" {
+ availability_zone = "cn-north-1a"
+ cidr = "172.16.0.0/24"
+ dhcp_enable = true
+ dns_list = (known after apply)
+ gateway_ip = "172.16.0.1"
+ id = (known after apply)
+ ipv4_subnet_id = (known after apply)
+ ipv6_cidr = (known after apply)
+ ipv6_gateway = (known after apply)
+ ipv6_subnet_id = (known after apply)
+ name = "dev-subnet"
+ primary_dns = (known after apply)
+ region = (known after apply)
+ secondary_dns = (known after apply)
+ subnet_id = (known after apply)
+ vpc_id = (known after apply)
}
Plan: 6 to add, 0 to change, 0 to destroy.
Changes to Outputs:
+ secgroup_id = (known after apply)
+ subnet_id = (known after apply)
+ subnet_subnet_id = (known after apply)
+ vpc_id = (known after apply)
module.dev-secgroup.huaweicloud_networking_secgroup.secgroup: Creating...
module.dev-vpc.huaweicloud_vpc.vpc: Creating...
module.dev-secgroup.huaweicloud_networking_secgroup.secgroup: Creation complete after 1s [id=4cd6b1ed-a98c-426c-8795-cc94addd6048]
module.dev-secgroup.huaweicloud_networking_secgroup_rule.secgroup_rule_out_all: Creating...
module.dev-secgroup.huaweicloud_networking_secgroup_rule.secgroup_rule_in_22: Creating...
module.dev-secgroup.huaweicloud_networking_secgroup_rule.secgroup_rule_in_80: Creating...
module.dev-secgroup.huaweicloud_networking_secgroup_rule.secgroup_rule_out_all: Creation complete after 1s [id=637e4fa6-8e88-46d1-a835-67f1463f43f9]
module.dev-secgroup.huaweicloud_networking_secgroup_rule.secgroup_rule_in_22: Creation complete after 1s [id=b3e94943-a910-48d5-bb6b-77c66d3b0727]
module.dev-secgroup.huaweicloud_networking_secgroup_rule.secgroup_rule_in_80: Creation complete after 1s [id=87c43785-5cf6-4d3a-a2cf-6b23402916bc]
module.dev-vpc.huaweicloud_vpc.vpc: Creation complete after 8s [id=6fd80aa4-e002-4ca4-8505-451c2837aceb]
module.dev-vpc.huaweicloud_vpc_subnet.subnet: Creating...
module.dev-vpc.huaweicloud_vpc_subnet.subnet: Creation complete after 8s [id=b9af8fb0-af41-41a6-aa2f-a8195cbc2325]
Apply complete! Resources: 6 added, 0 changed, 0 destroyed.
Outputs:
secgroup_id = "4cd6b1ed-a98c-426c-8795-cc94addd6048"
subnet_id = "b9af8fb0-af41-41a6-aa2f-a8195cbc2325"
subnet_subnet_id = "17018a76-7018-4d81-b547-d34494171d4d"
vpc_id = "6fd80aa4-e002-4ca4-8505-451c2837aceb"3. 上层服务资源：ecs、eip、elb、dns 创建
初始化拉取模块
terraform.exe init
Initializing modules...
Downloading git::ssh://git@e.coding.net/LotusChing/terraform/terraform-huaweiyun-modules.git for dev-ecs...
- dev-ecs in .terraform\modules\dev-ecs\ecs
Downloading git::ssh://git@e.coding.net/LotusChing/terraform/terraform-huaweiyun-modules.git for dev-eip...
- dev-eip in .terraform\modules\dev-eip\eip
Downloading git::ssh://git@e.coding.net/LotusChing/terraform/terraform-huaweiyun-modules.git for dev-elb...
- dev-elb in .terraform\modules\dev-elb\elb
Initializing the backend...
Initializing provider plugins...
- terraform.io/builtin/terraform is built in to Terraform
- Finding hashicorp/alicloud versions matching "1.197.0"...
- Finding latest version of hashicorp/template...
- Finding huaweicloud/huaweicloud versions matching ">= 1.36.0"...
- Using hashicorp/alicloud v1.197.0 from the shared cache directory
- Using hashicorp/template v2.2.0 from the shared cache directory
- Using huaweicloud/huaweicloud v1.47.1 from the shared cache directory
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.
╷
│ Warning: Additional provider information from registry
│
│ The remote registry returned warnings for
│ registry.terraform.io/hashicorp/alicloud:
│ - For users on Terraform 0.13 or greater, this provider has moved to
│ aliyun/alicloud. Please update your source in required_providers.
╵
Terraform has been successfully initialized!OK，就一个版本相关的 warning，没啥问题~
tc plan 检查确认， tc apply 创建资源
# tf plan
# ...
terraform.exe apply -auto-approve
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# huaweicloud_compute_keypair.keypair will be created
+ resource "huaweicloud_compute_keypair" "keypair" {
+ id = (known after apply)
+ key_file = (known after apply)
+ name = "tf-node-keypair"
+ public_key = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCTrVri2QWeZZPEdS+PvxG9bdHswybJjrljA6FvsfVB4ji5dwT6jrBLXt+auhEHz0F7YO5esQcJ3x+j1trgE1adZ4wU9tYvQekUO48imJOwTVnbJzdCpYGAoYqHMbrTaRiY3ryeqabbq4KdKJdy7ooHYtENTkAVU31wduq2xpEGBl9V6A2Z0QIgkCIJIUDh5rXakVOFTvKZrdA8y+aRnGt2lQ/JLoDJOU9n3nRNw757hRr048I6bQojXqx3F/y4IeQhQ2XTZ/4tbPyF6+vjEtbeWCkVYHbmq3StEcesOOc88ZhwKWrjtej67wU+V5C3DFoGF03a6IjQ8Zn6cII6vZwx liucong@lc"
+ region = (known after apply)
}
# huaweicloud_networking_eip_associate.eip_elb_associate will be created
+ resource "huaweicloud_networking_eip_associate" "eip_elb_associate" {
+ fixed_ip = (known after apply)
+ id = (known after apply)
+ mac_address = (known after apply)
+ network_id = (known after apply)
+ port_id = (known after apply)
+ public_ip = (known after apply)
+ region = (known after apply)
+ status = (known after apply)
}
# huaweicloud_vpc_eip.elb_eip will be created
+ resource "huaweicloud_vpc_eip" "elb_eip" {
+ address = (known after apply)
+ charging_mode = (known after apply)
+ enterprise_project_id = (known after apply)
+ id = (known after apply)
+ ipv6_address = (known after apply)
+ port_id = (known after apply)
+ private_ip = (known after apply)
+ region = (known after apply)
+ status = (known after apply)
+ bandwidth {
+ charge_mode = (known after apply)
+ id = (known after apply)
+ name = (known after apply)
+ share_type = "WHOLE"
+ size = (known after apply)
}
+ publicip {
+ ip_address = (known after apply)
+ ip_version = (known after apply)
+ port_id = (known after apply)
+ type = "5_bgp"
}
}
# module.dev-dns.alicloud_alidns_record.dns will be created
+ resource "alicloud_alidns_record" "dns" {
+ domain_name = "yo-yo.fun"
+ id = (known after apply)
+ line = "default"
+ remark = "terraform alidns demo"
+ rr = "dev.demo"
+ status = "ENABLE"
+ ttl = 600
+ type = "A"
+ value = (known after apply)
}
# module.dev-ecs[0].huaweicloud_compute_instance.basic will be created
+ resource "huaweicloud_compute_instance" "basic" {
+ access_ip_v4 = (known after apply)
+ access_ip_v6 = (known after apply)
+ availability_zone = "cn-north-1a"
+ charging_mode = (known after apply)
+ created_at = (known after apply)
+ delete_eip_on_termination = true
+ description = (known after apply)
+ enterprise_project_id = (known after apply)
+ flavor_id = "s2.small.1"
+ flavor_name = (known after apply)
+ id = (known after apply)
+ image_id = "d3c8947a-06cc-40a2-a43b-c0e53a6325c0"
+ image_name = (known after apply)
+ key_pair = "tf-node-keypair"
+ name = "node-0"
+ power_action = (known after apply)
+ public_ip = (known after apply)
+ region = (known after apply)
+ security_group_ids = [
+ "4cd6b1ed-a98c-426c-8795-cc94addd6048",
]
+ security_groups = (known after apply)
+ spot_duration_count = (known after apply)
+ status = (known after apply)
+ stop_before_destroy = false
+ system_disk_id = (known after apply)
+ system_disk_size = (known after apply)
+ system_disk_type = (known after apply)
+ updated_at = (known after apply)
+ user_data = "5995e0baa9fa4f58f63298c650405f917012ca63"
+ volume_attached = (known after apply)
+ network {
+ access_network = false
+ fixed_ip_v4 = (known after apply)
+ fixed_ip_v6 = (known after apply)
+ mac = (known after apply)
+ port = (known after apply)
+ source_dest_check = true
+ uuid = "b9af8fb0-af41-41a6-aa2f-a8195cbc2325"
}
+ scheduler_hints {
+ deh_id = (known after apply)
+ fault_domain = (known after apply)
+ group = (known after apply)
+ tenancy = (known after apply)
}
}
# module.dev-ecs[1].huaweicloud_compute_instance.basic will be created
+ resource "huaweicloud_compute_instance" "basic" {
+ access_ip_v4 = (known after apply)
+ access_ip_v6 = (known after apply)
+ availability_zone = "cn-north-1a"
+ charging_mode = (known after apply)
+ created_at = (known after apply)
+ delete_eip_on_termination = true
+ description = (known after apply)
+ enterprise_project_id = (known after apply)
+ flavor_id = "s2.small.1"
+ flavor_name = (known after apply)
+ id = (known after apply)
+ image_id = "d3c8947a-06cc-40a2-a43b-c0e53a6325c0"
+ image_name = (known after apply)
+ key_pair = "tf-node-keypair"
+ name = "node-1"
+ power_action = (known after apply)
+ public_ip = (known after apply)
+ region = (known after apply)
+ security_group_ids = [
+ "4cd6b1ed-a98c-426c-8795-cc94addd6048",
]
+ security_groups = (known after apply)
+ spot_duration_count = (known after apply)
+ status = (known after apply)
+ stop_before_destroy = false
+ system_disk_id = (known after apply)
+ system_disk_size = (known after apply)
+ system_disk_type = (known after apply)
+ updated_at = (known after apply)
+ user_data = "5995e0baa9fa4f58f63298c650405f917012ca63"
+ volume_attached = (known after apply)
+ network {
+ access_network = false
+ fixed_ip_v4 = (known after apply)
+ fixed_ip_v6 = (known after apply)
+ mac = (known after apply)
+ port = (known after apply)
+ source_dest_check = true
+ uuid = "b9af8fb0-af41-41a6-aa2f-a8195cbc2325"
}
+ scheduler_hints {
+ deh_id = (known after apply)
+ fault_domain = (known after apply)
+ group = (known after apply)
+ tenancy = (known after apply)
}
}
# module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[0] will be created
+ resource "huaweicloud_compute_eip_associate" "eip_associate" {
+ fixed_ip = (known after apply)
+ id = (known after apply)
+ instance_id = (known after apply)
+ port_id = (known after apply)
+ public_ip = (known after apply)
+ region = (known after apply)
}
# module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[1] will be created
+ resource "huaweicloud_compute_eip_associate" "eip_associate" {
+ fixed_ip = (known after apply)
+ id = (known after apply)
+ instance_id = (known after apply)
+ port_id = (known after apply)
+ public_ip = (known after apply)
+ region = (known after apply)
}
# module.dev-eip.huaweicloud_vpc_bandwidth.bandwidth_1 will be created
+ resource "huaweicloud_vpc_bandwidth" "bandwidth_1" {
+ bandwidth_type = (known after apply)
+ charge_mode = (known after apply)
+ enterprise_project_id = (known after apply)
+ id = (known after apply)
+ name = "dev-bandwidth"
+ publicips = (known after apply)
+ region = (known after apply)
+ share_type = (known after apply)
+ size = 5
+ status = (known after apply)
}
# module.dev-eip.huaweicloud_vpc_eip.eip[0] will be created
+ resource "huaweicloud_vpc_eip" "eip" {
+ address = (known after apply)
+ charging_mode = (known after apply)
+ enterprise_project_id = (known after apply)
+ id = (known after apply)
+ ipv6_address = (known after apply)
+ port_id = (known after apply)
+ private_ip = (known after apply)
+ region = (known after apply)
+ status = (known after apply)
+ bandwidth {
+ charge_mode = (known after apply)
+ id = (known after apply)
+ name = (known after apply)
+ share_type = "WHOLE"
+ size = (known after apply)
}
+ publicip {
+ ip_address = (known after apply)
+ ip_version = (known after apply)
+ port_id = (known after apply)
+ type = "5_bgp"
}
}
# module.dev-eip.huaweicloud_vpc_eip.eip[1] will be created
+ resource "huaweicloud_vpc_eip" "eip" {
+ address = (known after apply)
+ charging_mode = (known after apply)
+ enterprise_project_id = (known after apply)
+ id = (known after apply)
+ ipv6_address = (known after apply)
+ port_id = (known after apply)
+ private_ip = (known after apply)
+ region = (known after apply)
+ status = (known after apply)
+ bandwidth {
+ charge_mode = (known after apply)
+ id = (known after apply)
+ name = (known after apply)
+ share_type = "WHOLE"
+ size = (known after apply)
}
+ publicip {
+ ip_address = (known after apply)
+ ip_version = (known after apply)
+ port_id = (known after apply)
+ type = "5_bgp"
}
}
# module.dev-elb.huaweicloud_lb_listener.listener will be created
+ resource "huaweicloud_lb_listener" "listener" {
+ admin_state_up = true
+ connection_limit = (known after apply)
+ default_pool_id = (known after apply)
+ http2_enable = false
+ id = (known after apply)
+ loadbalancer_id = (known after apply)
+ name = (known after apply)
+ protocol = "HTTP"
+ protocol_port = 80
+ region = (known after apply)
+ tenant_id = (known after apply)
}
# module.dev-elb.huaweicloud_lb_loadbalancer.lb will be created
+ resource "huaweicloud_lb_loadbalancer" "lb" {
+ admin_state_up = true
+ enterprise_project_id = (known after apply)
+ id = (known after apply)
+ loadbalancer_provider = (known after apply)
+ public_ip = (known after apply)
+ region = (known after apply)
+ security_group_ids = (known after apply)
+ tenant_id = (known after apply)
+ vip_address = (known after apply)
+ vip_port_id = (known after apply)
+ vip_subnet_id = "17018a76-7018-4d81-b547-d34494171d4d"
}
# module.dev-elb.huaweicloud_lb_member.member[0] will be created
+ resource "huaweicloud_lb_member" "member" {
+ address = (known after apply)
+ admin_state_up = true
+ id = (known after apply)
+ pool_id = (known after apply)
+ protocol_port = 80
+ region = (known after apply)
+ subnet_id = "17018a76-7018-4d81-b547-d34494171d4d"
+ tenant_id = (known after apply)
+ weight = (known after apply)
}
# module.dev-elb.huaweicloud_lb_member.member[1] will be created
+ resource "huaweicloud_lb_member" "member" {
+ address = (known after apply)
+ admin_state_up = true
+ id = (known after apply)
+ pool_id = (known after apply)
+ protocol_port = 80
+ region = (known after apply)
+ subnet_id = "17018a76-7018-4d81-b547-d34494171d4d"
+ tenant_id = (known after apply)
+ weight = (known after apply)
}
# module.dev-elb.huaweicloud_lb_pool.pool will be created
+ resource "huaweicloud_lb_pool" "pool" {
+ admin_state_up = true
+ id = (known after apply)
+ lb_method = "ROUND_ROBIN"
+ listener_id = (known after apply)
+ loadbalancer_id = (known after apply)
+ protocol = "HTTP"
+ region = (known after apply)
+ tenant_id = (known after apply)
+ persistence {
+ cookie_name = "dev-cookie"
+ timeout = (known after apply)
+ type = "APP_COOKIE"
}
}
Plan: 16 to add, 0 to change, 0 to destroy.
Changes to Outputs:
+ ecs_ids = [
+ (known after apply),
+ (known after apply),
]
+ ecs_image = [
+ (known after apply),
+ (known after apply),
]
+ eip_public_ip = [
+ [
+ (known after apply),
+ (known after apply),
],
]
+ elb_eip_public_ip = (known after apply)
+ ip_ipv4 = [
+ (known after apply),
+ (known after apply),
]
huaweicloud_compute_keypair.keypair: Creating...
module.dev-ecs[0].huaweicloud_compute_instance.basic: Creating...
module.dev-ecs[1].huaweicloud_compute_instance.basic: Creating...
huaweicloud_compute_keypair.keypair: Creation complete after 0s [id=tf-node-keypair]
module.dev-ecs[0].huaweicloud_compute_instance.basic: Still creating... [10s elapsed]
module.dev-ecs[1].huaweicloud_compute_instance.basic: Still creating... [10s elapsed]
module.dev-ecs[0].huaweicloud_compute_instance.basic: Still creating... [20s elapsed]
module.dev-ecs[1].huaweicloud_compute_instance.basic: Still creating... [20s elapsed]
module.dev-ecs[0].huaweicloud_compute_instance.basic: Still creating... [30s elapsed]
module.dev-ecs[1].huaweicloud_compute_instance.basic: Still creating... [30s elapsed]
module.dev-ecs[0].huaweicloud_compute_instance.basic: Still creating... [40s elapsed]
module.dev-ecs[1].huaweicloud_compute_instance.basic: Still creating... [40s elapsed]
module.dev-ecs[0].huaweicloud_compute_instance.basic: Still creating... [50s elapsed]
module.dev-ecs[1].huaweicloud_compute_instance.basic: Still creating... [50s elapsed]
module.dev-ecs[0].huaweicloud_compute_instance.basic: Still creating... [1m0s elapsed]
module.dev-ecs[1].huaweicloud_compute_instance.basic: Still creating... [1m0s elapsed]
module.dev-ecs[1].huaweicloud_compute_instance.basic: Creation complete after 1m6s [id=b14ba0b1-e21b-4e81-afee-bc4a8b30044a]
module.dev-ecs[0].huaweicloud_compute_instance.basic: Still creating... [1m10s elapsed]
module.dev-ecs[0].huaweicloud_compute_instance.basic: Creation complete after 1m14s [id=c2160f7f-dd29-4c1a-820f-439d5f293359]
module.dev-eip.huaweicloud_vpc_bandwidth.bandwidth_1: Creating...
module.dev-eip.huaweicloud_vpc_bandwidth.bandwidth_1: Creation complete after 4s [id=618feb33-5ea8-419e-a09a-73cecf7610a1]
module.dev-eip.huaweicloud_vpc_eip.eip[0]: Creating...
module.dev-eip.huaweicloud_vpc_eip.eip[1]: Creating...
huaweicloud_vpc_eip.elb_eip: Creating...
module.dev-eip.huaweicloud_vpc_eip.eip[0]: Creation complete after 6s [id=a8b10002-9f41-4203-b853-d9845189cc7a]
huaweicloud_vpc_eip.elb_eip: Creation complete after 7s [id=d46d10b9-10ec-42c9-bee1-d90d4679cfb5]
module.dev-eip.huaweicloud_vpc_eip.eip[1]: Creation complete after 7s [id=0b6ac43d-ed68-401b-9087-02d88326cb5b]
module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[1]: Creating...
module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[0]: Creating...
module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[0]: Creation complete after 7s [id=49.4.53.202/c2160f7f-dd29-4c1a-820f-439d5f293359/172.16.0.189]
module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[1]: Creation complete after 8s [id=49.4.48.245/b14ba0b1-e21b-4e81-afee-bc4a8b30044a/172.16.0.54]
module.dev-elb.huaweicloud_lb_loadbalancer.lb: Creating...
module.dev-elb.huaweicloud_lb_loadbalancer.lb: Creation complete after 8s [id=bced8ef5-5371-4875-af6a-8afcf6c9e9f9]
module.dev-elb.huaweicloud_lb_listener.listener: Creating...
module.dev-elb.huaweicloud_lb_listener.listener: Still creating... [10s elapsed]
module.dev-elb.huaweicloud_lb_listener.listener: Creation complete after 14s [id=1af9d2e2-fd29-4271-b45f-4127f94a5a1e]
module.dev-elb.huaweicloud_lb_pool.pool: Creating...
module.dev-elb.huaweicloud_lb_pool.pool: Still creating... [10s elapsed]
module.dev-elb.huaweicloud_lb_pool.pool: Creation complete after 16s [id=9c41cd53-3a50-4488-9265-4a872f622d59]
module.dev-elb.huaweicloud_lb_member.member[1]: Creating...
module.dev-elb.huaweicloud_lb_member.member[0]: Creating...
module.dev-elb.huaweicloud_lb_member.member[0]: Still creating... [10s elapsed]
module.dev-elb.huaweicloud_lb_member.member[1]: Still creating... [10s elapsed]
module.dev-elb.huaweicloud_lb_member.member[0]: Creation complete after 14s [id=1ac358f4-b5cc-4758-9c63-e85ac156fde4]
module.dev-elb.huaweicloud_lb_member.member[1]: Creation complete after 14s [id=191d0920-c70b-4f2a-b89c-817a383651a3]
huaweicloud_networking_eip_associate.eip_elb_associate: Creating...
huaweicloud_networking_eip_associate.eip_elb_associate: Still creating... [10s elapsed]
huaweicloud_networking_eip_associate.eip_elb_associate: Creation complete after 13s [id=d46d10b9-10ec-42c9-bee1-d90d4679cfb5]
module.dev-dns.alicloud_alidns_record.dns: Creating...
module.dev-dns.alicloud_alidns_record.dns: Creation complete after 6s [id=823731051226818560]
Apply complete! Resources: 16 added, 0 changed, 0 destroyed.
Outputs:
ecs_ids = [
"c2160f7f-dd29-4c1a-820f-439d5f293359",
"b14ba0b1-e21b-4e81-afee-bc4a8b30044a",
]
ecs_image = [
"CentOS 7.9 64bit",
"CentOS 7.9 64bit",
]
eip_public_ip = [
[
"49.4.53.202",
"49.4.48.245",
],
]
elb_eip_public_ip = "49.4.48.144"
ip_ipv4 = [
"172.16.0.189",
"172.16.0.54",
]
web_url = "http://dev.demo.yo-yo.fun/"
Process finished with exit code 04. 检查确认
检查 lb 负载均衡效果
$ curl 49.4.48.144
node-0
$ curl 49.4.48.144
node-1
$ curl 49.4.48.144
node-1
$ curl 49.4.48.144
node-0检查解析记录
OK，通过域名尝试
$ curl dev.demo.yo-yo.fun
node-0
$ curl dev.demo.yo-yo.fun
node-0
$ curl dev.demo.yo-yo.fun
node-1
$ curl dev.demo.yo-yo.fun
node-0
$ curl dev.demo.yo-yo.fun
node-1
$ curl dev.demo.yo-yo.fun
node-0符合预期~
四、扩缩容后端集群
扩容 ecs 节点
现在 elb 后面是绑定了两台 nginx server，假如此时访问压力增大，需要扩容节点，这里增加一台
# env/dev/service/ecs.tf
locals {
availability_zone = "cn-north-1a"
counts = 3
# ..
}执行 tf plan，观察 change 输出
# ...
Terraform used the selected providers to generate the following execution
####### 变更类型是 create 资源
plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
############# 1. 创建 ecs 资源，module.dev-ecs[2] 则说明是第三台（以0开始）
# module.dev-ecs[2].huaweicloud_compute_instance.basic will be created
+ resource "huaweicloud_compute_instance" "basic" {
+ access_ip_v4 = (known after apply)
+ access_ip_v6 = (known after apply)
+ availability_zone = "cn-north-1a"
+ charging_mode = (known after apply)
+ created_at = (known after apply)
+ delete_eip_on_termination = true
+ description = (known after apply)
+ enterprise_project_id = (known after apply)
+ flavor_id = "s2.small.1"
+ flavor_name = (known after apply)
+ id = (known after apply)
+ image_id = "d3c8947a-06cc-40a2-a43b-c0e53a6325c0"
+ image_name = (known after apply)
+ key_pair = "tf-node-keypair"
###### 主机名 node-2，已有 node-0，node-1
+ name = "node-2"
+ power_action = (known after apply)
+ public_ip = (known after apply)
+ region = (known after apply)
+ security_group_ids = [
+ "4cd6b1ed-a98c-426c-8795-cc94addd6048",
]
+ security_groups = (known after apply)
+ spot_duration_count = (known after apply)
+ status = (known after apply)
+ stop_before_destroy = false
+ system_disk_id = (known after apply)
+ system_disk_size = (known after apply)
+ system_disk_type = (known after apply)
+ updated_at = (known after apply)
+ user_data = "5995e0baa9fa4f58f63298c650405f917012ca63"
+ volume_attached = (known after apply)
+ network {
+ access_network = false
+ fixed_ip_v4 = (known after apply)
+ fixed_ip_v6 = (known after apply)
+ mac = (known after apply)
+ port = (known after apply)
+ source_dest_check = true
+ uuid = "b9af8fb0-af41-41a6-aa2f-a8195cbc2325"
}
+ scheduler_hints {
+ deh_id = (known after apply)
+ fault_domain = (known after apply)
+ group = (known after apply)
+ tenancy = (known after apply)
}
}
###### 2. 创建 eip 与 instance 绑定关系
# module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[2] will be created
+ resource "huaweicloud_compute_eip_associate" "eip_associate" {
+ fixed_ip = (known after apply)
+ id = (known after apply)
+ instance_id = (known after apply)
+ port_id = (known after apply)
+ public_ip = (known after apply)
+ region = (known after apply)
}
###### 3. 创建 eip 资源
# module.dev-eip.huaweicloud_vpc_eip.eip[2] will be created
+ resource "huaweicloud_vpc_eip" "eip" {
+ address = (known after apply)
+ charging_mode = (known after apply)
+ enterprise_project_id = (known after apply)
+ id = (known after apply)
+ ipv6_address = (known after apply)
+ port_id = (known after apply)
+ private_ip = (known after apply)
+ region = (known after apply)
+ status = (known after apply)
+ bandwidth {
+ charge_mode = (known after apply)
+ id = "618feb33-5ea8-419e-a09a-73cecf7610a1"
+ name = (known after apply)
+ share_type = "WHOLE"
+ size = (known after apply)
}
+ publicip {
+ ip_address = (known after apply)
+ ip_version = (known after apply)
+ port_id = (known after apply)
+ type = "5_bgp"
}
}
###### 4. 创建添加 elb 后端服务器组成员
# module.dev-elb.huaweicloud_lb_member.member[2] will be created
+ resource "huaweicloud_lb_member" "member" {
+ address = (known after apply)
+ admin_state_up = true
+ id = (known after apply)
+ pool_id = "9c41cd53-3a50-4488-9265-4a872f622d59"
+ protocol_port = 80
+ region = (known after apply)
+ subnet_id = "17018a76-7018-4d81-b547-d34494171d4d"
+ tenant_id = (known after apply)
+ weight = (known after apply)
}
Plan: 4 to add, 0 to change, 0 to destroy.
Changes to Outputs:
~ ecs_ids = [
# (1 unchanged element hidden)
"b14ba0b1-e21b-4e81-afee-bc4a8b30044a",
+ (known after apply),
]
~ ecs_image = [
# (1 unchanged element hidden)
"CentOS 7.9 64bit",
+ (known after apply),
]
~ eip_public_ip = [
- [
- "49.4.53.202",
- "49.4.48.245",
],
+ [
+ "49.4.53.202",
+ "49.4.48.245",
+ (known after apply),
],
]
~ ip_ipv4 = [
# (1 unchanged element hidden)
"172.16.0.54",
+ (known after apply),
]
─────────────────────────────────────────────────────────────────────────────
# ...可以看到，共有四个添加动作，分别是
- 创建 ecs 服务器 node-2
- 创建 eip 与 instance 绑定关系
- 创建 eip 资源
- 创建添加 elb 后端服务器组成员
符合预期，执行 tf apply，这里不要再使用自动允许，比照之前的 plan 返回，仔细查看 change 信息，确认后再手动输入 yes 继续变更
terraform.exe apply
huaweicloud_compute_keypair.keypair: Refreshing state... [id=tf-node-keypair]
module.dev-ecs[1].huaweicloud_compute_instance.basic: Refreshing state... [id=b14ba0b1-e21b-4e81-afee-bc4a8b30044a]
module.dev-ecs[0].huaweicloud_compute_instance.basic: Refreshing state... [id=c2160f7f-dd29-4c1a-820f-439d5f293359]
module.dev-eip.huaweicloud_vpc_bandwidth.bandwidth_1: Refreshing state... [id=618feb33-5ea8-419e-a09a-73cecf7610a1]
module.dev-eip.huaweicloud_vpc_eip.eip[1]: Refreshing state... [id=0b6ac43d-ed68-401b-9087-02d88326cb5b]
huaweicloud_vpc_eip.elb_eip: Refreshing state... [id=d46d10b9-10ec-42c9-bee1-d90d4679cfb5]
module.dev-eip.huaweicloud_vpc_eip.eip[0]: Refreshing state... [id=a8b10002-9f41-4203-b853-d9845189cc7a]
module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[0]: Refreshing state... [id=49.4.53.202/c2160f7f-dd29-4c1a-820f-439d5f293359/172.16.0.189]
module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[1]: Refreshing state... [id=49.4.48.245/b14ba0b1-e21b-4e81-afee-bc4a8b30044a/172.16.0.54]
module.dev-elb.huaweicloud_lb_loadbalancer.lb: Refreshing state... [id=bced8ef5-5371-4875-af6a-8afcf6c9e9f9]
module.dev-elb.huaweicloud_lb_listener.listener: Refreshing state... [id=1af9d2e2-fd29-4271-b45f-4127f94a5a1e]
module.dev-elb.huaweicloud_lb_pool.pool: Refreshing state... [id=9c41cd53-3a50-4488-9265-4a872f622d59]
module.dev-elb.huaweicloud_lb_member.member[0]: Refreshing state... [id=1ac358f4-b5cc-4758-9c63-e85ac156fde4]
module.dev-elb.huaweicloud_lb_member.member[1]: Refreshing state... [id=191d0920-c70b-4f2a-b89c-817a383651a3]
huaweicloud_networking_eip_associate.eip_elb_associate: Refreshing state... [id=d46d10b9-10ec-42c9-bee1-d90d4679cfb5]
module.dev-dns.alicloud_alidns_record.dns: Refreshing state... [id=823731051226818560]
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# module.dev-ecs[2].huaweicloud_compute_instance.basic will be created
+ resource "huaweicloud_compute_instance" "basic" {
+ access_ip_v4 = (known after apply)
+ access_ip_v6 = (known after apply)
+ availability_zone = "cn-north-1a"
+ charging_mode = (known after apply)
+ created_at = (known after apply)
+ delete_eip_on_termination = true
+ description = (known after apply)
+ enterprise_project_id = (known after apply)
+ flavor_id = "s2.small.1"
+ flavor_name = (known after apply)
+ id = (known after apply)
+ image_id = "d3c8947a-06cc-40a2-a43b-c0e53a6325c0"
+ image_name = (known after apply)
+ key_pair = "tf-node-keypair"
+ name = "node-2"
+ power_action = (known after apply)
+ public_ip = (known after apply)
+ region = (known after apply)
+ security_group_ids = [
+ "4cd6b1ed-a98c-426c-8795-cc94addd6048",
]
+ security_groups = (known after apply)
+ spot_duration_count = (known after apply)
+ status = (known after apply)
+ stop_before_destroy = false
+ system_disk_id = (known after apply)
+ system_disk_size = (known after apply)
+ system_disk_type = (known after apply)
+ updated_at = (known after apply)
+ user_data = "5995e0baa9fa4f58f63298c650405f917012ca63"
+ volume_attached = (known after apply)
+ network {
+ access_network = false
+ fixed_ip_v4 = (known after apply)
+ fixed_ip_v6 = (known after apply)
+ mac = (known after apply)
+ port = (known after apply)
+ source_dest_check = true
+ uuid = "b9af8fb0-af41-41a6-aa2f-a8195cbc2325"
}
+ scheduler_hints {
+ deh_id = (known after apply)
+ fault_domain = (known after apply)
+ group = (known after apply)
+ tenancy = (known after apply)
}
}
# module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[2] will be created
+ resource "huaweicloud_compute_eip_associate" "eip_associate" {
+ fixed_ip = (known after apply)
+ id = (known after apply)
+ instance_id = (known after apply)
+ port_id = (known after apply)
+ public_ip = (known after apply)
+ region = (known after apply)
}
# module.dev-eip.huaweicloud_vpc_eip.eip[2] will be created
+ resource "huaweicloud_vpc_eip" "eip" {
+ address = (known after apply)
+ charging_mode = (known after apply)
+ enterprise_project_id = (known after apply)
+ id = (known after apply)
+ ipv6_address = (known after apply)
+ port_id = (known after apply)
+ private_ip = (known after apply)
+ region = (known after apply)
+ status = (known after apply)
+ bandwidth {
+ charge_mode = (known after apply)
+ id = "618feb33-5ea8-419e-a09a-73cecf7610a1"
+ name = (known after apply)
+ share_type = "WHOLE"
+ size = (known after apply)
}
+ publicip {
+ ip_address = (known after apply)
+ ip_version = (known after apply)
+ port_id = (known after apply)
+ type = "5_bgp"
}
}
# module.dev-elb.huaweicloud_lb_member.member[2] will be created
+ resource "huaweicloud_lb_member" "member" {
+ address = (known after apply)
+ admin_state_up = true
+ id = (known after apply)
+ pool_id = "9c41cd53-3a50-4488-9265-4a872f622d59"
+ protocol_port = 80
+ region = (known after apply)
+ subnet_id = "17018a76-7018-4d81-b547-d34494171d4d"
+ tenant_id = (known after apply)
+ weight = (known after apply)
}
Plan: 4 to add, 0 to change, 0 to destroy.
Changes to Outputs:
~ ecs_ids = [
# (1 unchanged element hidden)
"b14ba0b1-e21b-4e81-afee-bc4a8b30044a",
+ (known after apply),
]
~ ecs_image = [
# (1 unchanged element hidden)
"CentOS 7.9 64bit",
+ (known after apply),
]
~ eip_public_ip = [
- [
- "49.4.53.202",
- "49.4.48.245",
],
+ [
+ "49.4.53.202",
+ "49.4.48.245",
+ (known after apply),
],
]
~ ip_ipv4 = [
# (1 unchanged element hidden)
"172.16.0.54",
+ (known after apply),
]
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
###### 确认确认再确认
Enter a value: yes
module.dev-ecs[2].huaweicloud_compute_instance.basic: Creating...
module.dev-ecs[2].huaweicloud_compute_instance.basic: Still creating... [10s elapsed]
module.dev-ecs[2].huaweicloud_compute_instance.basic: Still creating... [20s elapsed]
module.dev-ecs[2].huaweicloud_compute_instance.basic: Still creating... [30s elapsed]
module.dev-ecs[2].huaweicloud_compute_instance.basic: Still creating... [40s elapsed]
module.dev-ecs[2].huaweicloud_compute_instance.basic: Still creating... [50s elapsed]
module.dev-ecs[2].huaweicloud_compute_instance.basic: Still creating... [1m0s elapsed]
module.dev-ecs[2].huaweicloud_compute_instance.basic: Creation complete after 1m7s [id=c8e35898-99ca-4c70-a67c-62189eb04a76]
module.dev-eip.huaweicloud_vpc_eip.eip[2]: Creating...
module.dev-eip.huaweicloud_vpc_eip.eip[2]: Creation complete after 7s [id=075c563c-6622-4613-a070-b6ca796f3190]
module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[2]: Creating...
module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[2]: Creation complete after 8s [id=49.4.49.97/c8e35898-99ca-4c70-a67c-62189eb04a76/172.16.0.85]
module.dev-elb.huaweicloud_lb_member.member[2]: Creating...
module.dev-elb.huaweicloud_lb_member.member[2]: Still creating... [10s elapsed]
module.dev-elb.huaweicloud_lb_member.member[2]: Creation complete after 15s [id=66a6385b-d6cf-4180-8cd8-d1550c55f53d]
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.
Outputs:
ecs_ids = [
"c2160f7f-dd29-4c1a-820f-439d5f293359",
"b14ba0b1-e21b-4e81-afee-bc4a8b30044a",
"c8e35898-99ca-4c70-a67c-62189eb04a76",
]
ecs_image = [
"CentOS 7.9 64bit",
"CentOS 7.9 64bit",
"CentOS 7.9 64bit",
]
eip_public_ip = [
[
"49.4.53.202",
"49.4.48.245",
"49.4.49.97",
],
]
elb_eip_public_ip = "49.4.48.144"
ip_ipv4 = [
"172.16.0.189",
"172.16.0.54",
"172.16.0.85",
]
web_url = "http://dev.demo.yo-yo.fun/"
Process finished with exit code 0测试负载均衡效果
$ curl http://dev.demo.yo-yo.fun/
node-2
$ curl http://dev.demo.yo-yo.fun/
node-0
$ curl http://dev.demo.yo-yo.fun/
node-1
$ curl http://dev.demo.yo-yo.fun/
node-2
$ curl http://dev.demo.yo-yo.fun/
node-0
$ curl http://dev.demo.yo-yo.fun/
node-1OK，node-2 顺利添加到后端服务器组，作为集群中的一员
缩容 ecs 节点
当压力降低需要扩容节点
# env/dev/service/ecs.tf
locals {
availability_zone = "cn-north-1a"
counts = 2
# ..
}执行 tf plan 观察 change
terraform.exe plan
huaweicloud_compute_keypair.keypair: Refreshing state... [id=tf-node-keypair]
module.dev-ecs[2].huaweicloud_compute_instance.basic: Refreshing state... [id=c8e35898-99ca-4c70-a67c-62189eb04a76]
module.dev-ecs[0].huaweicloud_compute_instance.basic: Refreshing state... [id=c2160f7f-dd29-4c1a-820f-439d5f293359]
module.dev-ecs[1].huaweicloud_compute_instance.basic: Refreshing state... [id=b14ba0b1-e21b-4e81-afee-bc4a8b30044a]
module.dev-eip.huaweicloud_vpc_bandwidth.bandwidth_1: Refreshing state... [id=618feb33-5ea8-419e-a09a-73cecf7610a1]
huaweicloud_vpc_eip.elb_eip: Refreshing state... [id=d46d10b9-10ec-42c9-bee1-d90d4679cfb5]
module.dev-eip.huaweicloud_vpc_eip.eip[2]: Refreshing state... [id=075c563c-6622-4613-a070-b6ca796f3190]
module.dev-eip.huaweicloud_vpc_eip.eip[1]: Refreshing state... [id=0b6ac43d-ed68-401b-9087-02d88326cb5b]
module.dev-eip.huaweicloud_vpc_eip.eip[0]: Refreshing state... [id=a8b10002-9f41-4203-b853-d9845189cc7a]
module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[0]: Refreshing state... [id=49.4.53.202/c2160f7f-dd29-4c1a-820f-439d5f293359/172.16.0.189]
module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[2]: Refreshing state... [id=49.4.49.97/c8e35898-99ca-4c70-a67c-62189eb04a76/172.16.0.85]
module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[1]: Refreshing state... [id=49.4.48.245/b14ba0b1-e21b-4e81-afee-bc4a8b30044a/172.16.0.54]
module.dev-elb.huaweicloud_lb_loadbalancer.lb: Refreshing state... [id=bced8ef5-5371-4875-af6a-8afcf6c9e9f9]
module.dev-elb.huaweicloud_lb_listener.listener: Refreshing state... [id=1af9d2e2-fd29-4271-b45f-4127f94a5a1e]
module.dev-elb.huaweicloud_lb_pool.pool: Refreshing state... [id=9c41cd53-3a50-4488-9265-4a872f622d59]
module.dev-elb.huaweicloud_lb_member.member[0]: Refreshing state... [id=1ac358f4-b5cc-4758-9c63-e85ac156fde4]
module.dev-elb.huaweicloud_lb_member.member[1]: Refreshing state... [id=191d0920-c70b-4f2a-b89c-817a383651a3]
module.dev-elb.huaweicloud_lb_member.member[2]: Refreshing state... [id=66a6385b-d6cf-4180-8cd8-d1550c55f53d]
huaweicloud_networking_eip_associate.eip_elb_associate: Refreshing state... [id=d46d10b9-10ec-42c9-bee1-d90d4679cfb5]
module.dev-dns.alicloud_alidns_record.dns: Refreshing state... [id=823731051226818560]
Note: Objects have changed outside of Terraform
Terraform detected the following changes made outside of Terraform since the
last "terraform apply":
# module.dev-ecs[2].huaweicloud_compute_instance.basic has changed
~ resource "huaweicloud_compute_instance" "basic" {
id = "c8e35898-99ca-4c70-a67c-62189eb04a76"
name = "node-2"
+ public_ip = "49.4.49.97"
+ tags = {}
~ updated_at = "2023-04-16T05:21:37Z" -> "2023-04-16T05:22:02Z"
# (21 unchanged attributes hidden)
# (1 unchanged block hidden)
}
# module.dev-eip.huaweicloud_vpc_bandwidth.bandwidth_1 has changed
~ resource "huaweicloud_vpc_bandwidth" "bandwidth_1" {
id = "618feb33-5ea8-419e-a09a-73cecf7610a1"
name = "dev-bandwidth"
~ publicips = [
# (1 unchanged element hidden)
{
id = "0b6ac43d-ed68-401b-9087-02d88326cb5b"
ip_address = "49.4.48.245"
ip_version = 4
type = "5_bgp"
},
+ {
+ id = "075c563c-6622-4613-a070-b6ca796f3190"
+ ip_address = "49.4.49.97"
+ ip_version = 4
+ type = "5_bgp"
},
{
id = "a8b10002-9f41-4203-b853-d9845189cc7a"
ip_address = "49.4.53.202"
ip_version = 4
type = "5_bgp"
},
]
# (6 unchanged attributes hidden)
}
# module.dev-eip.huaweicloud_vpc_eip.eip[2] has changed
~ resource "huaweicloud_vpc_eip" "eip" {
id = "075c563c-6622-4613-a070-b6ca796f3190"
+ port_id = "cd82ef6f-c8d7-4fdb-a757-a479e2955221"
+ private_ip = "172.16.0.85"
~ status = "UNBOUND" -> "BOUND"
+ tags = {}
# (3 unchanged attributes hidden)
~ publicip {
+ port_id = "cd82ef6f-c8d7-4fdb-a757-a479e2955221"
# (3 unchanged attributes hidden)
}
# (1 unchanged block hidden)
}
Unless you have made equivalent changes to your configuration, or ignored the
relevant attributes using ignore_changes, the following plan may include
actions to undo or respond to these changes.
─────────────────────────────────────────────────────────────────────────────
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
- destroy
Terraform will perform the following actions:
####### 1. 销毁 node-2 ecs 节点
# module.dev-ecs[2].huaweicloud_compute_instance.basic will be destroyed
# (because module.dev-ecs[2] is not in configuration)
- resource "huaweicloud_compute_instance" "basic" {
- access_ip_v4 = "172.16.0.85" -> null
- availability_zone = "cn-north-1a" -> null
- charging_mode = "postPaid" -> null
- created_at = "2023-04-16T05:20:48Z" -> null
- delete_eip_on_termination = true -> null
- enterprise_project_id = "0" -> null
- flavor_id = "s2.small.1" -> null
- flavor_name = "s2.small.1" -> null
- id = "c8e35898-99ca-4c70-a67c-62189eb04a76" -> null
- image_id = "d3c8947a-06cc-40a2-a43b-c0e53a6325c0" -> null
- image_name = "CentOS 7.9 64bit" -> null
- key_pair = "tf-node-keypair" -> null
- name = "node-2" -> null
- public_ip = "49.4.49.97" -> null
- region = "cn-north-1" -> null
- security_group_ids = [
- "4cd6b1ed-a98c-426c-8795-cc94addd6048",
] -> null
- security_groups = [
- "dev_secgroup",
] -> null
- status = "ACTIVE" -> null
- stop_before_destroy = false -> null
- system_disk_id = "422db651-ac55-45a1-9318-c95d7c1a4edb" -> null
- system_disk_size = 40 -> null
- system_disk_type = "GPSSD" -> null
- tags = {} -> null
- updated_at = "2023-04-16T05:22:02Z" -> null
- user_data = "5995e0baa9fa4f58f63298c650405f917012ca63" -> null
- volume_attached = [
- {
- boot_index = 0
- kms_key_id = ""
- pci_address = "0000:02:01.0"
- size = 40
- type = "GPSSD"
- volume_id = "422db651-ac55-45a1-9318-c95d7c1a4edb"
},
] -> null
- network {
- access_network = false -> null
- fixed_ip_v4 = "172.16.0.85" -> null
- ipv6_enable = false -> null
- mac = "fa:16:3e:fb:eb:40" -> null
- port = "cd82ef6f-c8d7-4fdb-a757-a479e2955221" -> null
- source_dest_check = true -> null
- uuid = "b9af8fb0-af41-41a6-aa2f-a8195cbc2325" -> null
}
}
####### 2. 销毁 ecs eip 绑定关系
# module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[2] will be destroyed
# (because index [2] is out of range for count)
- resource "huaweicloud_compute_eip_associate" "eip_associate" {
- fixed_ip = "172.16.0.85" -> null
- id = "49.4.49.97/c8e35898-99ca-4c70-a67c-62189eb04a76/172.16.0.85" -> null
- instance_id = "c8e35898-99ca-4c70-a67c-62189eb04a76" -> null
- port_id = "cd82ef6f-c8d7-4fdb-a757-a479e2955221" -> null
- public_ip = "49.4.49.97" -> null
- region = "cn-north-1" -> null
}
####### 3. 销毁 eip 资源
# module.dev-eip.huaweicloud_vpc_eip.eip[2] will be destroyed
# (because index [2] is out of range for count)
- resource "huaweicloud_vpc_eip" "eip" {
- address = "49.4.49.97" -> null
- enterprise_project_id = "0" -> null
- id = "075c563c-6622-4613-a070-b6ca796f3190" -> null
- port_id = "cd82ef6f-c8d7-4fdb-a757-a479e2955221" -> null
- private_ip = "172.16.0.85" -> null
- region = "cn-north-1" -> null
- status = "BOUND" -> null
- tags = {} -> null
- bandwidth {
- charge_mode = "bandwidth" -> null
- id = "618feb33-5ea8-419e-a09a-73cecf7610a1" -> null
- name = "dev-bandwidth" -> null
- share_type = "WHOLE" -> null
- size = 5 -> null
}
- publicip {
- ip_address = "49.4.49.97" -> null
- ip_version = 4 -> null
- port_id = "cd82ef6f-c8d7-4fdb-a757-a479e2955221" -> null
- type = "5_bgp" -> null
}
}
####### 4. 将 node-2 从后端 ecs 成员组中剔除
# module.dev-elb.huaweicloud_lb_member.member[2] will be destroyed
# (because index [2] is out of range for count)
- resource "huaweicloud_lb_member" "member" {
- address = "172.16.0.85" -> null
- admin_state_up = true -> null
- id = "66a6385b-d6cf-4180-8cd8-d1550c55f53d" -> null
- pool_id = "9c41cd53-3a50-4488-9265-4a872f622d59" -> null
- protocol_port = 80 -> null
- region = "cn-north-1" -> null
- subnet_id = "17018a76-7018-4d81-b547-d34494171d4d" -> null
- tenant_id = "0d58621e5300f3052ff0c00e6ff0a42b" -> null
- weight = 1 -> null
}
Plan: 0 to add, 0 to change, 4 to destroy.
Changes to Outputs:
~ ecs_ids = [
# (1 unchanged element hidden)
"b14ba0b1-e21b-4e81-afee-bc4a8b30044a",
- "c8e35898-99ca-4c70-a67c-62189eb04a76",
]
~ ecs_image = [
# (1 unchanged element hidden)
"CentOS 7.9 64bit",
- "CentOS 7.9 64bit",
]
~ eip_public_ip = [
- [
- "49.4.53.202",
- "49.4.48.245",
- "49.4.49.97",
],
+ [
+ "49.4.53.202",
+ "49.4.48.245",
],
]
~ ip_ipv4 = [
# (1 unchanged element hidden)
"172.16.0.54",
- "172.16.0.85",
]4 to destroy，分别是
- 销毁 node-2 ecs 节点
- 销毁 ecs eip 绑定关系
- 销毁 eip 资源
- 将 node-2 从后端 ecs 成员组中剔除
执行 tf apply，观察 change，确认后输入 yes 执行变更
terraform.exe apply
huaweicloud_compute_keypair.keypair: Refreshing state... [id=tf-node-keypair]
module.dev-ecs[1].huaweicloud_compute_instance.basic: Refreshing state... [id=b14ba0b1-e21b-4e81-afee-bc4a8b30044a]
module.dev-ecs[0].huaweicloud_compute_instance.basic: Refreshing state... [id=c2160f7f-dd29-4c1a-820f-439d5f293359]
module.dev-ecs[2].huaweicloud_compute_instance.basic: Refreshing state... [id=c8e35898-99ca-4c70-a67c-62189eb04a76]
module.dev-eip.huaweicloud_vpc_bandwidth.bandwidth_1: Refreshing state... [id=618feb33-5ea8-419e-a09a-73cecf7610a1]
module.dev-eip.huaweicloud_vpc_eip.eip[0]: Refreshing state... [id=a8b10002-9f41-4203-b853-d9845189cc7a]
module.dev-eip.huaweicloud_vpc_eip.eip[1]: Refreshing state... [id=0b6ac43d-ed68-401b-9087-02d88326cb5b]
huaweicloud_vpc_eip.elb_eip: Refreshing state... [id=d46d10b9-10ec-42c9-bee1-d90d4679cfb5]
module.dev-eip.huaweicloud_vpc_eip.eip[2]: Refreshing state... [id=075c563c-6622-4613-a070-b6ca796f3190]
module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[2]: Refreshing state... [id=49.4.49.97/c8e35898-99ca-4c70-a67c-62189eb04a76/172.16.0.85]
module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[0]: Refreshing state... [id=49.4.53.202/c2160f7f-dd29-4c1a-820f-439d5f293359/172.16.0.189]
module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[1]: Refreshing state... [id=49.4.48.245/b14ba0b1-e21b-4e81-afee-bc4a8b30044a/172.16.0.54]
module.dev-elb.huaweicloud_lb_loadbalancer.lb: Refreshing state... [id=bced8ef5-5371-4875-af6a-8afcf6c9e9f9]
module.dev-elb.huaweicloud_lb_listener.listener: Refreshing state... [id=1af9d2e2-fd29-4271-b45f-4127f94a5a1e]
module.dev-elb.huaweicloud_lb_pool.pool: Refreshing state... [id=9c41cd53-3a50-4488-9265-4a872f622d59]
module.dev-elb.huaweicloud_lb_member.member[2]: Refreshing state... [id=66a6385b-d6cf-4180-8cd8-d1550c55f53d]
module.dev-elb.huaweicloud_lb_member.member[0]: Refreshing state... [id=1ac358f4-b5cc-4758-9c63-e85ac156fde4]
module.dev-elb.huaweicloud_lb_member.member[1]: Refreshing state... [id=191d0920-c70b-4f2a-b89c-817a383651a3]
huaweicloud_networking_eip_associate.eip_elb_associate: Refreshing state... [id=d46d10b9-10ec-42c9-bee1-d90d4679cfb5]
module.dev-dns.alicloud_alidns_record.dns: Refreshing state... [id=823731051226818560]
Note: Objects have changed outside of Terraform
Terraform detected the following changes made outside of Terraform since the
last "terraform apply":
# module.dev-ecs[2].huaweicloud_compute_instance.basic has changed
~ resource "huaweicloud_compute_instance" "basic" {
id = "c8e35898-99ca-4c70-a67c-62189eb04a76"
name = "node-2"
+ public_ip = "49.4.49.97"
+ tags = {}
~ updated_at = "2023-04-16T05:21:37Z" -> "2023-04-16T05:22:02Z"
# (21 unchanged attributes hidden)
# (1 unchanged block hidden)
}
# module.dev-eip.huaweicloud_vpc_bandwidth.bandwidth_1 has changed
~ resource "huaweicloud_vpc_bandwidth" "bandwidth_1" {
id = "618feb33-5ea8-419e-a09a-73cecf7610a1"
name = "dev-bandwidth"
~ publicips = [
# (1 unchanged element hidden)
{
id = "0b6ac43d-ed68-401b-9087-02d88326cb5b"
ip_address = "49.4.48.245"
ip_version = 4
type = "5_bgp"
},
+ {
+ id = "075c563c-6622-4613-a070-b6ca796f3190"
+ ip_address = "49.4.49.97"
+ ip_version = 4
+ type = "5_bgp"
},
{
id = "a8b10002-9f41-4203-b853-d9845189cc7a"
ip_address = "49.4.53.202"
ip_version = 4
type = "5_bgp"
},
]
# (6 unchanged attributes hidden)
}
# module.dev-eip.huaweicloud_vpc_eip.eip[2] has changed
~ resource "huaweicloud_vpc_eip" "eip" {
id = "075c563c-6622-4613-a070-b6ca796f3190"
+ port_id = "cd82ef6f-c8d7-4fdb-a757-a479e2955221"
+ private_ip = "172.16.0.85"
~ status = "UNBOUND" -> "BOUND"
+ tags = {}
# (3 unchanged attributes hidden)
~ publicip {
+ port_id = "cd82ef6f-c8d7-4fdb-a757-a479e2955221"
# (3 unchanged attributes hidden)
}
# (1 unchanged block hidden)
}
Unless you have made equivalent changes to your configuration, or ignored the
relevant attributes using ignore_changes, the following plan may include
actions to undo or respond to these changes.
─────────────────────────────────────────────────────────────────────────────
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
- destroy
Terraform will perform the following actions:
# module.dev-ecs[2].huaweicloud_compute_instance.basic will be destroyed
# (because module.dev-ecs[2] is not in configuration)
- resource "huaweicloud_compute_instance" "basic" {
- access_ip_v4 = "172.16.0.85" -> null
- availability_zone = "cn-north-1a" -> null
- charging_mode = "postPaid" -> null
- created_at = "2023-04-16T05:20:48Z" -> null
- delete_eip_on_termination = true -> null
- enterprise_project_id = "0" -> null
- flavor_id = "s2.small.1" -> null
- flavor_name = "s2.small.1" -> null
- id = "c8e35898-99ca-4c70-a67c-62189eb04a76" -> null
- image_id = "d3c8947a-06cc-40a2-a43b-c0e53a6325c0" -> null
- image_name = "CentOS 7.9 64bit" -> null
- key_pair = "tf-node-keypair" -> null
- name = "node-2" -> null
- public_ip = "49.4.49.97" -> null
- region = "cn-north-1" -> null
- security_group_ids = [
- "4cd6b1ed-a98c-426c-8795-cc94addd6048",
] -> null
- security_groups = [
- "dev_secgroup",
] -> null
- status = "ACTIVE" -> null
- stop_before_destroy = false -> null
- system_disk_id = "422db651-ac55-45a1-9318-c95d7c1a4edb" -> null
- system_disk_size = 40 -> null
- system_disk_type = "GPSSD" -> null
- tags = {} -> null
- updated_at = "2023-04-16T05:22:02Z" -> null
- user_data = "5995e0baa9fa4f58f63298c650405f917012ca63" -> null
- volume_attached = [
- {
- boot_index = 0
- kms_key_id = ""
- pci_address = "0000:02:01.0"
- size = 40
- type = "GPSSD"
- volume_id = "422db651-ac55-45a1-9318-c95d7c1a4edb"
},
] -> null
- network {
- access_network = false -> null
- fixed_ip_v4 = "172.16.0.85" -> null
- ipv6_enable = false -> null
- mac = "fa:16:3e:fb:eb:40" -> null
- port = "cd82ef6f-c8d7-4fdb-a757-a479e2955221" -> null
- source_dest_check = true -> null
- uuid = "b9af8fb0-af41-41a6-aa2f-a8195cbc2325" -> null
}
}
# module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[2] will be destroyed
# (because index [2] is out of range for count)
- resource "huaweicloud_compute_eip_associate" "eip_associate" {
- fixed_ip = "172.16.0.85" -> null
- id = "49.4.49.97/c8e35898-99ca-4c70-a67c-62189eb04a76/172.16.0.85" -> null
- instance_id = "c8e35898-99ca-4c70-a67c-62189eb04a76" -> null
- port_id = "cd82ef6f-c8d7-4fdb-a757-a479e2955221" -> null
- public_ip = "49.4.49.97" -> null
- region = "cn-north-1" -> null
}
# module.dev-eip.huaweicloud_vpc_eip.eip[2] will be destroyed
# (because index [2] is out of range for count)
- resource "huaweicloud_vpc_eip" "eip" {
- address = "49.4.49.97" -> null
- enterprise_project_id = "0" -> null
- id = "075c563c-6622-4613-a070-b6ca796f3190" -> null
- port_id = "cd82ef6f-c8d7-4fdb-a757-a479e2955221" -> null
- private_ip = "172.16.0.85" -> null
- region = "cn-north-1" -> null
- status = "BOUND" -> null
- tags = {} -> null
- bandwidth {
- charge_mode = "bandwidth" -> null
- id = "618feb33-5ea8-419e-a09a-73cecf7610a1" -> null
- name = "dev-bandwidth" -> null
- share_type = "WHOLE" -> null
- size = 5 -> null
}
- publicip {
- ip_address = "49.4.49.97" -> null
- ip_version = 4 -> null
- port_id = "cd82ef6f-c8d7-4fdb-a757-a479e2955221" -> null
- type = "5_bgp" -> null
}
}
# module.dev-elb.huaweicloud_lb_member.member[2] will be destroyed
# (because index [2] is out of range for count)
- resource "huaweicloud_lb_member" "member" {
- address = "172.16.0.85" -> null
- admin_state_up = true -> null
- id = "66a6385b-d6cf-4180-8cd8-d1550c55f53d" -> null
- pool_id = "9c41cd53-3a50-4488-9265-4a872f622d59" -> null
- protocol_port = 80 -> null
- region = "cn-north-1" -> null
- subnet_id = "17018a76-7018-4d81-b547-d34494171d4d" -> null
- tenant_id = "0d58621e5300f3052ff0c00e6ff0a42b" -> null
- weight = 1 -> null
}
Plan: 0 to add, 0 to change, 4 to destroy.
Changes to Outputs:
~ ecs_ids = [
# (1 unchanged element hidden)
"b14ba0b1-e21b-4e81-afee-bc4a8b30044a",
- "c8e35898-99ca-4c70-a67c-62189eb04a76",
]
~ ecs_image = [
# (1 unchanged element hidden)
"CentOS 7.9 64bit",
- "CentOS 7.9 64bit",
]
~ eip_public_ip = [
- [
- "49.4.53.202",
- "49.4.48.245",
- "49.4.49.97",
],
+ [
+ "49.4.53.202",
+ "49.4.48.245",
],
]
~ ip_ipv4 = [
# (1 unchanged element hidden)
"172.16.0.54",
- "172.16.0.85",
]
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
module.dev-elb.huaweicloud_lb_member.member[2]: Destroying... [id=66a6385b-d6cf-4180-8cd8-d1550c55f53d]
module.dev-elb.huaweicloud_lb_member.member[2]: Still destroying... [id=66a6385b-d6cf-4180-8cd8-d1550c55f53d, 10s elapsed]
module.dev-elb.huaweicloud_lb_member.member[2]: Destruction complete after 14s
module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[2]: Destroying... [id=49.4.49.97/c8e35898-99ca-4c70-a67c-62189eb04a76/172.16.0.85]
module.dev-eip.huaweicloud_compute_eip_associate.eip_associate[2]: Destruction complete after 2s
module.dev-eip.huaweicloud_vpc_eip.eip[2]: Destroying... [id=075c563c-6622-4613-a070-b6ca796f3190]
module.dev-eip.huaweicloud_vpc_eip.eip[2]: Still destroying... [id=075c563c-6622-4613-a070-b6ca796f3190, 10s elapsed]
module.dev-eip.huaweicloud_vpc_eip.eip[2]: Destruction complete after 11s
module.dev-ecs[2].huaweicloud_compute_instance.basic: Destroying... [id=c8e35898-99ca-4c70-a67c-62189eb04a76]
module.dev-ecs[2].huaweicloud_compute_instance.basic: Still destroying... [id=c8e35898-99ca-4c70-a67c-62189eb04a76, 10s elapsed]
module.dev-ecs[2].huaweicloud_compute_instance.basic: Still destroying... [id=c8e35898-99ca-4c70-a67c-62189eb04a76, 20s elapsed]
module.dev-ecs[2].huaweicloud_compute_instance.basic: Still destroying... [id=c8e35898-99ca-4c70-a67c-62189eb04a76, 30s elapsed]
module.dev-ecs[2].huaweicloud_compute_instance.basic: Still destroying... [id=c8e35898-99ca-4c70-a67c-62189eb04a76, 40s elapsed]
module.dev-ecs[2].huaweicloud_compute_instance.basic: Still destroying... [id=c8e35898-99ca-4c70-a67c-62189eb04a76, 50s elapsed]
module.dev-ecs[2].huaweicloud_compute_instance.basic: Destruction complete after 56s
Apply complete! Resources: 0 added, 0 changed, 4 destroyed.
Outputs:
ecs_ids = [
"c2160f7f-dd29-4c1a-820f-439d5f293359",
"b14ba0b1-e21b-4e81-afee-bc4a8b30044a",
]
ecs_image = [
"CentOS 7.9 64bit",
"CentOS 7.9 64bit",
]
eip_public_ip = [
[
"49.4.53.202",
"49.4.48.245",
],
]
elb_eip_public_ip = "49.4.48.144"
ip_ipv4 = [
"172.16.0.189",
"172.16.0.54",
]
web_url = "http://dev.demo.yo-yo.fun/"
Process finished with exit code 0访问测试
$ curl http://dev.demo.yo-yo.fun/
node-0
$ curl http://dev.demo.yo-yo.fun/
node-1
$ curl http://dev.demo.yo-yo.fun/
node-0
$ curl http://dev.demo.yo-yo.fun/
node-1五、优化集群上线效率
通常来说，ecs 节点拿到手后，第一时间不是上线，而是进行标准化配置，根据要工作角色包括但不限于
- 主机名配置
- 基础软件包安装
- 内核参数调整
- 防火墙策略调整
- 其他服务配置参数调整
标准化配置后，那就要根据它即将从事的角色，按照特定应用或服务的需求进行再次配置，以最基本的 Kubernetes worker 节点为例
- 系统内核升级
- 内核参数调优
- 容器运行时安装
- k8s 相关程序安装
- 相关镜像预拉取
- 获取 join 信息加入集群
这部分繁琐的工作，可以由可以拆成两部分
- 变更频次低的部分：通过自定义镜像将耗时最多的部分省掉，如：系统环境初始化、软件依赖包安装部署、容器镜像预先拉取等
- 需要动态调整的部分：使用 ansible role，通过模版渲染、逻辑判断，进行快速部署
所以，补充一下 如何在华为云平台使用 ansible 动态 inventry，基本上每个云平台都会向用户提供这个功能
下载相关脚本及配置文件
# 下载 动态 inventry 脚本 $ wget https://raw.githubusercontent.com/huaweicloud/huaweicloud-ansible-modules/master/contrib/inventory/hwc_ecs.py # 下载 动态 inventry 配置文件 $ wget https://raw.githubusercontent.com/huaweicloud/huaweicloud-ansible-modules/master/contrib/inventory/hwc_ecs.ini安装依赖与基本配置
$ pip install huaweicloudsdkcore==3.0.50 huaweicloudsdkecs==3.0.50 PyYAML==5.4.1 urllib3==1.26.6 $ chmod +x hwc_ecs.py $ cat > hwc_ecs.ini << EOF access_key=xxx secret_key=xxx # 并非字面上的项目ID，类似于地区ID，查询路径：华为云控制台 → 我的凭证 → API 凭证 → 项目列表 → 项目ID project_id=0d58621e5300f3052ff0c00e6ff0a42b region=cn-north-1 EOF获取服务器列表
$ ./hwc_ecs.py --list准备私钥，前面我们是通过导入公钥到了 ecs 节点的
authorized_keys，所以这里用到对应的私钥登录$ chmod 600 huaweicluod_ecs.rsa $ mv huaweicluod_ecs.rsa ~/.ssh/使用 ansible 远程执行命令
$ ansible -i hwc_ecs.py "node*" -m ping --private-key=/root/.ssh/huaweicluod_ecs.rsa [WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details node_0 | SUCCESS => { "ansible_facts": { "discovered_interpreter_python": "/usr/bin/python" }, "changed": false, "ping": "pong" } node_1 | SUCCESS => { "ansible_facts": { "discovered_interpreter_python": "/usr/bin/python" }, "changed": false, "ping": "pong" }使用 ansible 远程执行剧本
--- - hosts: "node*" remote_user: root gather_facts: no vars: text: Hello, World! tasks: - name: debug demo debug: msg: "He say {{ text }}"执行剧本
$ ap -i hwc_ecs.py playbook-debug-demo1.yml --private-key=/root/.ssh/huaweicluod_ecs.rsa [WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details PLAY [node*] *************** TASK [debug demo] *************** ok: [node_0] => { "msg": "He say Hello, World!" } ok: [node_1] => { "msg": "He say Hello, World!" } PLAY RECAP *************** node_0 : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 node_1 : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0使用 ansible 远程执行 role，操作思路是一样的，不再赘述
$ ap -i hwc_ecs.py --tags=initial,cri,kubernetes setup.yml
