This page describes the fields supported in the GKE on Bare Metal cluster configuration file. For each field, the following table identifies whether the field is required. The table also shows which fields are mutable, meaning which fields can be changed after a cluster has been created. As noted in the table, some mutable fields can only be changed during a cluster upgrade.
Generating a template for your cluster configuration file
You can create a cluster configuration file with the bmctl create config
command. Although some fields have default values and others, such as
metadata.name
can be auto-filled, this YAML format configuration file is a
template for specifying information about your cluster.
To create a new cluster configuration file, use the following command in the
/baremetal
folder:
bmctl create config -c CLUSTER_NAME
Replace CLUSTER_NAME
with the name for the cluster you want
to create. For more information about bmctl
, see bmctl tool.
For an example of the generated cluster configuration file, see
Cluster configuration file sample.
Filling in your configuration file
In your configuration file, enter field values as described in the following field reference table before you create or upgrade your cluster.
Cluster configuration fields
Field name | Resource type | Required? | Mutable? |
---|---|---|---|
anthosBareMetalVersion
Required. String. The cluster version. This value is set for cluster creation and cluster upgrades. Mutability: This value can't be modified for existing clusters. The version can be updated only through the cluster upgrade process. |
Cluster resource | Required | Mutable |
authentication
This section contains settings needed to use OpenID Connect (OIDC). OIDC lets you use your existing identity provider to manage user and group authentication in GKE on Bare Metal clusters. |
Cluster resource | — | — |
authentication.oidc.certificateAuthorityData
Optional. A For example (sample wrapped to fit table): certificateAuthorityData: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tC ...k1JSUN2RENDQWFT== |
Cluster resource | Optional | Immutable |
authentication.oidc.clientID
Optional. String. The ID for the client application that makes authentication requests to the OpenID provider. |
Cluster resource | Optional | Immutable |
authentication.oidc.clientSecret
Optional. String. Shared secret between OIDC client application and OIDC provider. |
Cluster resource | Optional | Immutable |
authentication.oidc.deployCloudConsoleProxy
Optional. Boolean ( |
Cluster resource | Optional | Immutable |
authentication.oidc.extraParams
Optional. Comma-delimited list. Additional key-value parameters to send to the OpenID provider. |
Cluster resource | Optional | Immutable |
authentication.oidc.groupPrefix
Optional. String. Prefix prepended to group claims to prevent clashes
with existing names. For example, given a group |
Cluster resource | Optional | Immutable |
authentication.oidc.group
Optional. String. JWT claim that the provider uses to return your security groups. |
Cluster resource | Optional | Immutable |
authentication.oidc.issuerURL
Optional. URL string. URL where authorization requests are sent to
your OpenID, such as |
Cluster resource | Optional | Immutable |
authentication.oidc.kubectlRedirectURL
Optional. URL string. The redirect URL that |
Cluster resource | Optional | Immutable |
authentication.oidc.proxy
Optional. URL string. Proxy server to use for the cluster to connect
to your OIDC provider, if applicable. The value should include a
hostname/IP address and optionally a port, username, and password. For
example: |
Cluster resource | Optional | Immutable |
authentication.oidc.scopes
Optional. Comma-delimited list. Additional scopes to send to the
OpenID provider. Microsoft Azure and Okta require the |
Cluster resource | Optional | Immutable |
authentication.oidc.usernamePrefix
Optional. String. Prefix prepended to username claims. |
Cluster resource | Optional | Immutable |
authentication.oidc.username
Optional. String.
JWT
claim to use as the username. If not specified, defaults to |
Cluster resource | Optional | Immutable |
bypassPreflightCheck
Optional. Boolean ( Mutability: This value can be modified for existing clusters
with the |
Cluster resource | Optional | Mutable |
clusterNetwork
This section contains network settings for your cluster. |
Cluster resource | Required | Mutable |
clusterNetwork.advancedNetworking
Boolean. Set this field to For more information about Network Gateway for GDC and related advanced networking features, see Configure an egress NAT gateway and Configure bundled load balancers with BGP. |
Cluster resource | Optional | Immutable |
clusterNetwork.bundledIngress
Boolean. Set this field to For more information about the bundled Ingress capability, see Create a Service and an Ingress. |
Cluster resource | Optional | Mutable |
clusterNetwork.flatIPv4
Boolean. Set this field to |
Cluster resource | Optional | Immutable |
clusterNetwork.forwardMode
Optional. String. Specifies the networking mode for Dataplane V2 load
balancing. Source network address translation (SNAT) is the default
networking mode. Direct Server Return (DSR) mode overcomes issues with
SNAT load balancing. In DSR mode ( Allowed values: For more information, see Configure load balancing networking mode. |
Cluster resource | Optional | Immutable |
clusterNetwork.multipleNetworkInterfaces
Optional. Boolean. Set this field to For more information about the setting up and using multiple network interfaces, see the Configure multiple network interfaces for Pods documentation. |
Cluster resource | Optional | Immutable |
clusterNetwork.pods.cidrBlocks
Required. Range of IPv4 addresses in CIDR block format. Pods specify the IP ranges from which pod networks are allocated.
For example: pods: cidrBlocks: - 192.168.0.0/16 |
Cluster resource | Required | Immutable |
clusterNetwork.sriovOperator
Optional. Boolean. Set this field to For more information about configuring and using SR-IOV networking, see the Set up SR-IOV networking documentation. |
Cluster resource | Optional | Mutable |
clusterNetwork.services.cidrBlocks
Required. Range of IPv4 addresses in CIDR block format. Specify the range of IP addresses from which service virtual IP (VIP) addresses are allocated. The ranges must not overlap with any subnets reachable from your network. For more information about address allocation for private internets, see RFC 1918. Starting with GKE on Bare Metal release 1.15.0, this field is mutable. If needed, you can increase the number of IP addresses allocated for services after you have created a cluster. For more information, see Increase service network range. You can only increase the range of the IPv4 service CIDR. The network range can't be reduced, which means the mask (the value after "/") can't be increased.
For example: services: cidrBlocks: - 10.96.0.0/12 |
Cluster resource | Required | Mutable |
clusterOperations
This section holds information for Cloud Logging and Cloud Monitoring. |
Cluster resource | Required | Mutable |
clusterOperations.enableApplication
This field is no longer used and has no effect. Application logging and monitoring is enabled in the stackdriver custom resource. For more information about enabling application logging and monitoring, see Enable application logging and monitoring. |
Cluster resource | No-op | Mutable |
clusterOperations.disableCloudAuditLogging
Boolean. Cloud Audit Logs is useful for investigating suspicious API
requests and for collecting statistics. Cloud Audit Logs is enabled
( For more information, see Use Audit Logging. |
Cluster resource | Optional | Mutable |
clusterOperations.location
String. A Google Cloud region where you want to store Logging logs and Monitoring metrics. It's a good idea to choose a region that is near your on-premises data center. For more information, see Global Locations. For example: location: us-central1 |
Cluster resource | Required | Immutable |
clusterOperations.projectID
String. The project ID of the Google Cloud project where you want to view logs and metrics. |
Cluster resource | Required | Immutable |
controlPlane
This section holds information about the control plane and its components. |
Cluster resource | Required | Mutable |
controlPlane.apiServerCertExtraSANs
Optional. An array of strings (domain names and IP addresses). A subject alternative name (SAN) is a feature of SSL certificates that lets you define the domain names and subdomains on which you want a certificate to be valid. On a GKE on Bare Metal cluster, the SANs for the API server certificate include the IP and VIP addresses of the control plane nodes and the Kubernetes DNS names by default. Use this field to add extra SANs to the API server certificate for the cluster. Domain names must comply with RFC 1035. For more information, see Add domains to the API server certificate. For example: ... controlPlane: apiServerCertExtraSANs: - "demo-dns.example.com" - "sample-dns.com" nodePoolSpec: ... This field can be added or changed at any time. |
Cluster resource | Optional | Mutable |
controlPlane.nodePoolSpec
This section specifies the IP addresses for the node pool used by the control plane and its components. The control plane node pool specification (like the load balancer node pool specification) is special. This specification declares and controls critical cluster resources. The canonical source for this resource is this section in the cluster configuration file. Don't modify the top-level control plane node pool resources directly. Modify the associated sections in the cluster configuration file instead. |
Cluster resource | Required | Mutable |
controlPlane.nodePoolSpec.nodes
Required. An array of IP addresses. Typically, this array is either an IP address for a single machine, or IP addresses for three machines for a high-availability (HA) deployment. For example: controlPlane: nodePoolSpec: nodes: - address: 192.168.1.212 - address: 192.168.1.213 - address: 192.168.1.214 This field can be changed whenever you update or upgrade a cluster. |
Cluster resource | Required | Mutable |
controlPlane.nodePoolSpec.kubeletConfig
Optional. This section contains fields that configure kubelet on all nodes in the control plane node pool. For example: controlPlane: nodePoolSpec: kubeletConfig: registryBurst: 15 registryPullQPS: 10 serializeImagePulls: false |
Cluster resource | Optional | Mutable |
controlPlane.nodePoolSpec.kubeletConfig.registryBurst
Optional. Integer (non-negative). Specifies the maximum quantity of
image pull requests that can be added to the processing queue to handle
spikes in requests. As soon as a pull starts, a new request can be added
to the queue. The default value is 10. This field corresponds to the
The value for This field can be set whenever you create, update, or upgrade a cluster and the setting persists through cluster upgrades. For more information, see Configure kubelet image pull settings. |
Cluster resource | Optional | Mutable |
controlPlane.nodePoolSpec.kubeletConfig.registryPullQPS
Optional. Integer (non-negative). Specifies the processing rate for
queries for container registry image pulls in queries per second (QPS).
When
This field corresponds to the
This field can be set whenever you create, update, or upgrade a cluster and the setting persists through cluster upgrades. For more information, see Configure kubelet image pull settings. |
Cluster resource | Optional | Mutable |
controlPlane.nodePoolSpec.kubeletConfig.serializeImagePulls
Optional. Boolean ( This field can be set whenever you create, update, or upgrade a cluster and the setting persists through cluster upgrades. For more information, see Configure kubelet image pull settings. |
Cluster resource | Optional | Mutable |
gkeConnect
This section holds information about the Google Cloud project you want to use to connect your cluster to Google Cloud. |
Cluster resource | Required | Immutable |
gkeConnect.projectID
Required: String. The ID of the Google Cloud project that you want to use for connecting your cluster to Google Cloud. This is also referred to as the fleet host project. For example: spec: ... gkeConnect: projectID: "my-connect-project-123" This value can't be modified for existing clusters. |
Cluster resource | Required | Immutable |
gkeConnect.location
Optional. String. Default value: For a list of supported regions, see Supported regions for the GKE On-Prem API. If not specified, the global instances of the services are used. Note the following:
For example: spec: ... gkeConnect: projectID: "my-connect-project-123" location: "us-central1" This value can't be modified for existing clusters. |
Cluster resource | Optional | Immutable |
gkeOnPremAPI
In 1.16 and later, if the GKE On-Prem API is enabled in your
Google Cloud project, all clusters in the project are
enrolled in the
GKE On-Prem API automatically in the region configured in
Enrolling your admin or user cluster in the GKE On-Prem API lets you use standard tools—the Google Cloud console, Google Cloud CLI, or Terraform—to view cluster details and to manage the cluster lifecycle. For example, you run can gcloud CLI commands to get information about your cluster. The GKE On-Prem API stores cluster state metadata in Google Cloud. This metadata lets the API manage the cluster lifecycle. The standard tools use the GKE On-Prem API, and collectively they are referred to as the GKE On-Prem API clients.
If you set After you add this section and create or update the cluster, if subsequently you remove the section and update the cluster, the update will fail.
If you prefer to create the cluster using a standard tool
instead of
When you create a cluster using a standard tool, the cluster is automatically enrolled in the GKE On-Prem API. |
Cluster resource | Optional | |
gkeOnPremAPI.enabled
By default, the cluster is enrolled in the GKE On-Prem API if the
GKE On-Prem API is enabled in your project. Set to After the cluster is enrolled in the GKE On-Prem API, if you need to unenroll the cluster, make the following change and then update the cluster: gkeOnPremAPI: enabled: false |
Cluster resource | Required | Mutable |
gkeOnPremAPI.location
The Google Cloud region where the GKE On-Prem API runs and
stores cluster metadata. Choose one of the
supported regions. Must be a non-empty string if
If this section isn't included in your configuration file, this
field is set to |
Cluster resource | Optional | Immutable |
kubevirt.useEmulation (deprecated)
Deprecated. As of release 1.11.2, you can enable or disable
VM Runtime on GDC by updating the VMRuntime custom resource
only.
Boolean. Determines whether or not software emulation is used to run
virtual machines. If the node supports hardware virtualization, set
|
Cluster resource | Optional | Mutable |
loadBalancer
This section contains settings for cluster load balancing. |
Cluster resource | Required | Mutable |
loadBalancer.addressPools
Object. The name and an array of IP addresses for your cluster load
balancer pool. Address pool configuration is only valid for
|
Cluster resource | Optional | Immutable |
loadBalancer.addressPools.addresses
Array of IP address ranges. Specify a list of non-overlapping IP ranges for the data plane load balancer. All addresses must be in the same subnet as the load balancer nodes. For example: addressPools: - name: pool1 addresses: - 192.168.1.0-192.168.1.4 - 192.168.1.240/28 |
Cluster resource | Optional | Immutable |
loadBalancer.addressPools.name
String. The name you choose for your cluster load balancer pool. |
Cluster resource | Required | Immutable |
loadBalancer.addressPools.avoidBuggyIPs
Optional. Boolean ( |
Cluster resource | Optional | Mutable |
loadBalancer.addressPools.manualAssign
Optional. Boolean ( |
Cluster resource | Optional | Mutable |
loadBalancer.mode
Required. String. Specifies the load-balancing mode. In
Allowed values: |
Cluster resource | Required | Immutable |
loadBalancer.type
Optional. String. Specifies the type of bundled load-balancing used,
Layer 2 or Border Gateway Protocol (BGP). If you are using the
standard, bundled load
balancing, set
Allowed values: |
Cluster resource | Optional | Immutable |
loadBalancer.nodePoolSpec
Optional. Use this section to configure a load balancer node pool. The
nodes you specify are part of the Kubernetes cluster and run regular
workloads and load balancers. If you don't specify a node pool, then
the control plane nodes are used for load balancing. This section
applies only when the load-balancing mode is set to If you want to prevent workloads from running on a node in the load balancer node pool, add the following taint to the node: node-role.kubernetes.io/load-balancer:NoSchedule GKE on Bare Metal adds tolerations for this taint to the pods that are required for load balancing. |
Cluster resource | Optional | Mutable |
loadBalancer.nodePoolSpec.nodes
This section contains an array of IP addresses for the nodes in your load-balancer node pool. |
Cluster resource | Optional | Mutable |
loadBalancer.nodePoolSpec.nodes.address
Optional. String (IPv4 address). IP address of a node. Although nodes in the load balancer node pool can run workloads, they're separate from the nodes in the worker node pools. You can't include a given cluster node in more than one node pool. Overlapping node IP addresses block cluster creation and other cluster operations. |
Cluster resource | Optional | Mutable |
loadBalancer.nodePoolSpec.kubeletConfig
Optional. This section contains fields that configure kubelet on all nodes in the control plane node pool. For example: loadBalancer: nodePoolSpec: kubeletConfig: registryBurst: 15 registryPullQPS: 10 serializeImagePulls: false |
Cluster resource | Optional | Mutable |
loadBalancer.nodePoolSpec.kubeletConfig.registryBurst
Optional. Integer (non-negative). Specifies the maximum number of
image pull requests that can be added to the processing queue to handle
spikes in requests. As soon as a pull starts, a new request can be added
to the queue. The default value is 10. This field corresponds to the
The value for This field can be set whenever you create, update, or upgrade a cluster and the setting persists through cluster upgrades. For more information, see Configure kubelet image pull settings. |
Cluster resource | Optional | Mutable |
loadBalancer.nodePoolSpec.kubeletConfig.registryPullQPS
Optional. Integer (non-negative). Specifies the processing rate for
queries for container registry image pulls in queries per second (QPS).
When
This field corresponds to the
This field can be set whenever you create, update, or upgrade a cluster and the setting persists through cluster upgrades. For more information, see Configure kubelet image pull settings. |
Cluster resource | Optional | Mutable |
loadBalancer.nodePoolSpec.kubeletConfig.serializeImagePulls
Optional. Boolean ( This field can be set whenever you create, update, or upgrade a cluster and the setting persists through cluster upgrades. For more information, see Configure kubelet image pull settings. |
Cluster resource | Optional | Mutable |
loadBalancer.ports.controlPlaneLBPort
Number. The destination port used for traffic sent to the Kubernetes control plane (the Kubernetes API servers). |
Cluster resource | Required | Immutable |
loadBalancer.vips.controlPlaneVIP
Required. Specifies the virtual IP address (VIP) to connect to the
Kubernetes API server. This address must not fall within the range of
any IP addresses used for load balancer address pools,
|
Cluster resource | Required | Immutable |
loadBalancer.vips.ingressVIP
Optional. String (IPv4 address). The IP address that you have chosen to configure on the load balancer for ingress traffic. |
Cluster resource | Optional | Immutable |
loadBalancer.localASN
Optional. String. Specifies the autonomous system number (ASN) for the cluster being created. This field is used when setting up the bundled load-balancing solution that uses border gateway protocol (BGP). For more information, see Configure bundled load balancers with BGP. |
Cluster resource | Optional | Mutable |
loadBalancer.bgpPeers
Optional. Object (list of mappings). This section specifies one or more border gateway protocol (BGP) peers from your (external to the cluster) local network. You specify BGP peers when you set up control plane load balancing part of the bundled load-balancing solution that uses BGP. Each peer is specified with a mapping, consisting of an IP address, an autonomous system number (ASN), and, optionally, a list of one or more IP addresses for control plane nodes. The BGP-peering configuration for control plane load balancing can't be updated after the cluster has been created. For example: loadBalancer: mode: bundled type: bgp localASN: 65001 bgpPeers: - ip: 10.0.1.254 asn: 65002 controlPlaneNodes: - 10.0.1.10 - 10.0.1.11 - ip: 10.0.2.254 asn: 65002 controlPlaneNodes: - 10.0.2.10 For more information, see Configure bundled load balancers with BGP. |
Cluster resource | Optional | Mutable |
loadBalancer.bgpPeers.ip
Optional. String (IPv4 address). The IP address of an external peering device from your local network. For more information, see Configure bundled load balancers with BGP. |
Cluster resource | Optional | Mutable |
loadBalancer.bgpPeers.asn
Optional. String. The autonomous system number (ASN) for the network that contains the external peer device. Specify an ASN for every BGP peer you set up for control plane load balancing, when you set up the bundled load-balancing solution that uses BGP. For more information, see Configure bundled load balancers with BGP. |
Cluster resource | Optional | Mutable |
loadBalancer.bgpPeers.controlPlaneNodes
Optional. Array of IP (IPv4) addresses. One or more IP addresses for control plane nodes that connect to the external BGP peer, when you set up the bundled load-balancing solution that uses BGP. If you don't specify any control plane nodes, all control plane nodes will connect to the external peer. If you specify one or more IP addresses, only the nodes specified participate in peering sessions. For more information, see Configure bundled load balancers with BGP. |
Cluster resource | Optional | Mutable |
maintenanceBlocks.cidrBlocks
Optional. Single IPv4 address or a range of IPv4 addresses. Specify the IP addresses for the node machines you want to put into maintenance mode. For more information, see Put nodes into maintenance mode. For example: maintenanceBlocks: cidrBlocks: - 192.168.1.200 # Single machine - 192.168.1.100-192.168.1.109 # Ten machines |
Cluster resource | Optional | Mutable |
nodeAccess.loginUser
Optional. String. Specify the non-root username you want to use for
passwordless SUDO capability access to the node machines in your
cluster. Your SSH key,
|
Cluster resource | Optional | Mutable |
osEnvironmentConfig.addPackageRepo
Optional. Boolean ( |
Cluster resource | Optional | Immutable |
nodeConfig
This section contains settings for cluster node configuration. |
Cluster resource | Optional | Mutable (upgrade only) |
nodeConfig.containerRuntime (deprecated)
Deprecated. As of release 1.13.0, GKE on Bare Metal supports
|
Cluster resource | Optional | Mutable (upgrade only) |
nodeConfig.podDensity
This section specifies the pod density configuration. |
Cluster resource | Optional | Immutable |
nodeConfig.podDensity.maxPodsPerNode
Optional. Integer. Specifies the maximum number of pods that can be
run on a single node. For self-managed clusters, allowable values for
Kubernetes assigns a Classless Inter-Domain Routing (CIDR) block to each node so that each pod can have a unique IP address. The size of the CIDR block corresponds to the maximum number of pods per node. For more information about setting the maximum number of pods per node, see Pod networking. |
Cluster resource | Optional | Immutable |
nodePoolUpgradeStrategy
Optional. This section contains settings for configuring the upgrade strategy for the worker node pools in your cluster. For more information, see Parallel upgrades. |
Cluster resource | Optional | Mutable |
nodePoolUpgradeStrategy.concurrentNodePools
Optional. Boolean ( apiVersion: baremetal.cluster.gke.io/v1 kind: Cluster metadata: name: cluster1 namespace: cluster-cluster1 spec: ... nodePoolUpgradeStrategy: concurrentNodePools: 0 ... For more information, see Node pool upgrade strategy. The nodes in each worker node pool upgrade according to the upgrade strategy in their corresponding NodePool spec. |
Cluster resource | Optional | Mutable |
nodePoolUpgradeStrategy.pause
Optional. Boolean ( GKE on Bare Metal 1.29The upgrade pause and resume feature is GA for clusters with all control plane nodes at minor version 1.29 or higher. For version 1.29 clusters, this feature is enabled by default. GKE on Bare Metal 1.28
The upgrade pause and resume feature is available in
Preview for
clusters with all control plane nodes at minor version 1.28 or
higher. For version 1.28 clusters, use the
Update the apiVersion: baremetal.cluster.gke.io/v1 kind: Cluster metadata: name: cluster1 namespace: cluster-cluster1 annotations: preview.baremetal.cluster.gke.io/upgrade-pause-and-resume spec: ... nodePoolUpgradeStrategy: pause: true ... For more information, see Pause and resume upgrades. |
Cluster resource | Optional | Mutable |
periodicHealthCheck
This section holds configuration information for periodic health
checks. In the Cluster resource, the only setting available for
periodic health checks is the |
Cluster resource | Optional | Mutable |
periodicHealthCheck.enable
Optional. Boolean ( |
Cluster resource | Optional | Mutable |
profile
Optional. String. When |
Cluster resource | Optional | Immutable |
proxy
If your network is behind a proxy server, fill in this section. Otherwise, remove this section. |
Cluster resource | Optional | Mutable |
proxy.noProxy
String. A comma-separated list of IP addresses, IP address ranges, host names, and domain names that shouldn't go through the proxy server. When GKE on Bare Metal sends a request to one of these addresses, hosts, or domains, the request is sent directly. |
Cluster resource | Optional | Immutable |
proxy.url
String. The HTTP address of your proxy server. Include the port number even if it's the same as the scheme's default port. For example: proxy: url: "http://my-proxy.example.local:80" noProxy: "10.151.222.0/24, my-host.example.local,10.151.2.1" |
Cluster resource | Optional | Immutable |
clusterSecurity
This section specifies the cluster security-related settings. |
Cluster resource | Optional | Mutable |
clusterSecurity.enableSeccomp (Preview)
Optional. Boolean ( |
Cluster resource | Optional | Mutable (upgrade only) |
clusterSecurity.enableRootlessContainers
Optional. Boolean ( |
Cluster resource | Optional | Mutable (upgrade only) |
clusterSecurity.authorization
Optional. Authorization configures user access to the cluster. |
Cluster resource | Optional | Mutable |
clusterSecurity.authorization.clusterAdmin
Optional. Specifies cluster administrator for this cluster. |
Cluster resource | Optional | Mutable |
clusterSecurity.authorization.clusterAdmin.gcpAccounts
Optional. The These RBAC policies also let users sign in to the Google Cloud console using their Google identity, if they have the required Identity and Access Management roles to access the console.
This field takes an array of account names. User accounts and
service accounts are supported. For users, you specify their
Google Cloud account email addresses. For service accounts, specify
the email addresses in the following format:
... clusterSecurity: authorization: clusterAdmin: gcpAccounts: - alex@example.com - hao@example.com - my-sa@example-project-123.iam.gserviceaccount.com ... When updating a cluster to add an account, be sure to include all accounts in the list (both existing and new accounts) because the update command overwrites the list with what you specify in the update.
This field only applies to clusters that can run workloads. For
example, you can't specify |
Cluster resource | Optional | Mutable |
clusterSecurity.startUIDRangeRootlessContainers
Optional. Integer. Default value: Allowed values: For example: apiVersion: baremetal.cluster.gke.io/v1 kind: Cluster metadata: name: name-of-cluster spec: clusterSecurity: startUIDRangeRootlessContainers: 5000 ... For more information, see Don't run containers as root user. |
Cluster resource | Optional | Mutable (upgrade only) |
storage.lvpNodeMounts.path
Required. String. Use the |
Cluster resource | Required | Immutable |
storage
This section contains settings for cluster storage. |
Cluster resource | Required | Immutable |
storage.lvpNodeMounts
This section specifies the configuration (path) for local persistent volumes backed by mounted disks. You must format and mount these disks yourself. You can do this task before or after cluster creation. For more information, see LVP node mounts. |
Cluster resource | Required | Immutable |
storage.lvpShare
This section specifies the configuration for local persistent volumes backed by subdirectories in a shared file system. These subdirectories are automatically created during cluster creation. For more information, see LVP share. |
Cluster resource | Required | Immutable |
storage.lvpShare.path
Required. String. Use the |
Cluster resource | Required | Immutable |
storage.lvpShare.numPVUnderSharedPath
Required. String. Specify the number of subdirectories to create under
|
Cluster resource | Required | Immutable |
storage.lvpShare.storageClassName
Required. String. Specify the StorageClass to use to create persistent
volumes. The StorageClass is created during cluster creation. The
default value is |
Cluster resource | Optional | Immutable |
type
Required. String. Specifies the type of cluster. The standard deployment model consists of a single admin cluster and one or more user clusters, which are managed by the admin cluster. GKE on Bare Metal supports the following types of clusters:
Cluster type is specified at cluster creation and can't be changed for updates or upgrades. For more information about how to create a cluster, see Creating clusters: overview.
Allowed values: This value can't be modified for existing clusters. |
Cluster resource | Required | Immutable |
name
Required. String. Typically, the namespace name uses a pattern of
This value can't be modified for existing clusters. |
Namespace resource | Required | Immutable |
clusterName
String. Required. The name of the cluster to which you are adding the node pool. Create the node pool resource in the same namespace as the associated cluster and reference the cluster name in this field. For more information, see Add and remove node pools in a cluster. For example: apiVersion: baremetal.cluster.gke.io/v1 kind: NodePool metadata: name: node-pool-new namespace: cluster-my-cluster spec: clusterName: my-cluster nodes: - address: 10.200.0.10 - address: 10.200.0.11 - address: 10.200.0.12 |
NodePool resource | Required | Immutable |
nodes
Optional. Array of IP (IPv4) addresses. This defines the node pool for your worker nodes. |
NodePool resource | Optional | Mutable |
nodes.address
Optional. String (IPv4 address). One or more IP addresses for the nodes that make your pool for worker nodes. |
NodePool resource | Optional | Mutable |
kubeletConfig
Optional. This section contains fields that configure kubelet on all nodes in the control plane node pool. For example: apiVersion: baremetal.cluster.gke.io/v1 kind: NodePool metadata: name: node-pool-new namespace: cluster-my-cluster spec: clusterName: my-cluster ... kubeletConfig: serializeImagePulls: true registryBurst: 20 registryPullQPS: 10 |
NodePool resource | Optional | Mutable |
kubeletConfig.registryBurst
Optional. Integer (non-negative). Specifies the maximum quantity of
image pull requests that can be added to the processing queue to handle
spikes in requests. As soon as a pull starts, a new request can be added
to the queue. The default value is 10. This field corresponds to the
The value for This field can be set whenever you create, update, or upgrade a cluster and the setting persists through cluster upgrades. For more information, see Configure kubelet image pull settings. |
NodePool resource | Optional | Mutable |
kubeletConfig.registryPullQPS
Optional. Integer (non-negative). Specifies the processing rate for
queries for container registry image pulls in queries per second (QPS).
When
This field corresponds to the
This field can be set whenever you create, update, or upgrade a cluster and the setting persists through cluster upgrades. For more information, see Configure kubelet image pull settings. |
NodePool resource | Optional | Mutable |
kubeletConfig.serializeImagePulls
Optional. Boolean ( This field can be set whenever you create, update, or upgrade a cluster and the setting persists through cluster upgrades. For more information, see Configure kubelet image pull settings. |
NodePool resource | Optional | Mutable |
taints
Optional. Object. A node taint lets you mark a node so that the
scheduler avoids or prevents using it for certain pods. A taint
consists of a key-value pair and an associated effect. The
The
For GKE on Bare Metal, taints are reconciled to the nodes of the
node pool unless the For example: taints: - key: status value: testpool effect: NoSchedule |
NodePool resource | Optional | Mutable |
labels
Optional. Mapping (key-value pairs).
Labels are reconciled to the nodes of the node pool unless the
|
NodePool resource | Optional | Mutable |
upgradeStrategy
Optional. This section contains settings for configuring upgrade strategy for the nodes in a worker node pool. For more information, see Parallel upgrades. Note: Don't add this section for control plane or load balancer node pools. |
NodePool resource | Optional | Mutable |
upgradeStrategy.parallelUpgrade
Optional. This section contains settings for configuring parallel node upgrades for a worker node pool. In a typical, default cluster upgrade, each cluster node is upgraded sequentially, one after the other. You can configure worker node pools so that multiple nodes upgrade in parallel when you upgrade your cluster. Upgrading nodes in parallel speeds up cluster upgrades significantly, especially for clusters that have hundreds of nodes. For a worker node pool, you can specify the number of nodes to upgrade concurrently and you can set a minimum threshold for the number of nodes able to run workloads throughout the upgrade process. For more information, see Node upgrade strategy. apiVersion: baremetal.cluster.gke.io/v1 kind: NodePool metadata: name: np1 namespace: cluster-cluster1 spec: clusterName: cluster1 nodes: - address: 10.200.0.1 ... upgradeStrategy: parallelUpgrade: concurrentNodes: 2 minimumAvailableNodes: 5 |
NodePool resource | Optional | Mutable |
upgradeStrategy.parallelUpgrade.concurrentNodes
Optional. Integer (positive). Default:
Parallel upgrades don't honor the
Pod Disruption Budget (PDB).
If your workloads are sensitive to disruptions, we recommend that you
specify |
NodePool resource | Optional | Mutable |
upgradeStrategy.parallelUpgrade.minimumAvailableNodes
Optional. Integer (non-negative). Default: Depends on
When you use this field together with the
A high value for |
NodePool resource | Optional | Mutable |
registryMirrors
Optional. Use this section to specify a registry mirror to use for
installing clusters, instead of Container Registry
( For example: registryMirrors: - endpoint: https://172.18.0.20:5000 caCertPath: /root/ca.crt pullCredentialConfigPath: /root/.docker/config.json hosts: - somehost.io - otherhost.io |
Registry mirror | Optional | Mutable |
registryMirrors.endpoint
String. The endpoint of the mirror, consisting of the registry server
IP address and port number. Optionally, you can use your own namespace
in your registry server instead of the root namespace. Without a
namespace, the endpoint format is
The For example: - endpoint: https://172.18.0.20:5000/v2/test-namespace |
Registry mirror | Optional | Mutable |
registryMirrors.caCertPath
Optional. String. Path of the CA cert file (server root CA) if your registry server uses a private TLS certificate. If your local registry doesn't require a private TLS certificate, then you can omit this field. |
Registry mirror | Optional | Mutable |
registryMirrors.pullCredentialConfigPath
Optional. String. Path to the
Docker CLI configuration file, For example: registryMirrors: - endpoint: https://172.18.0.20:5000 caCertPath: /root/ca.crt pullCredentialConfigPath: /root/.docker/config.json |
Registry mirror | Optional | Mutable |
registryMirrors.hosts
Optional. An array of domain names for hosts that are mirrored locally
for the given registry mirror ( For example: registryMirrors: - endpoint: https://172.18.0.20:5000 caCertPath: /root/ca.crt pullCredentialConfigPath: /root/.docker/config.json hosts: - somehost.io - otherhost.io |
Registry mirror | Optional | Mutable |
The cluster configuration file generated by For example: gcrKeyPath: bmctl-workspace/.sa-keys/my-gcp-project-anthos-baremetal-gcr.json sshPrivateKeyPath: /home/root-user/.ssh/id_rsa gkeConnectAgentServiceAccountKeyPath: bmctl-workspace/.sa-keys/my-gcp-project-anthos-baremetal-connect.json gkeConnectRegisterServiceAccountKeyPath: bmctl-workspace/.sa-keys/my-gcp-project-anthos-baremetal-register.json cloudOperationsServiceAccountKeyPath: bmctl-workspace/.sa-keys/my-gcp-project-anthos-baremetal-cloud-ops.json |
Credentials | Optional | Mutable |
gcrKeyPath
String. The path to the Container Registry service account key. The Container Registry service account is a Google-managed service account that acts on behalf of Container Registry when interacting with Google Cloud services. |
Credentials | Optional | Mutable |
sshPrivateKeyPath
String. The path to the SSH private key. SSH is required for Node access. |
Credentials | Optional | Mutable |
gkeConnectAgentServiceAccountKeyPath
String. The path to the agent service account key. GKE on Bare Metal uses this service account to maintain a connection between GKE on Bare Metal and Google Cloud. For instructions on configuring this service account, see Configuring service accounts for use with Connect. |
Credentials | Optional | Mutable |
gkeConnectRegisterServiceAccountKeyPath
String. The path to the registration service account key. GKE on Bare Metal uses this service account to register your user clusters with Google Cloud. For instructions on configuring this service account, see Configuring service accounts for use with Connect. |
Credentials | Optional | Mutable |
cloudOperationsServiceAccountKeyPath
String. The path to the operations service account key. GKE on Bare Metal uses the operations service account to authenticate with Google Cloud Observability for access to the Logging API and the Monitoring API. For instructions on configuring this service account, see Configuring a service account for use with Logging and Monitoring. |
Credentials | Optional | Mutable |
ipv4
Defines the configuration for the IPv4 CIDR range. At least one of the
|
ClusterCIDRConfig resource | Optional | Immutable |
ipv4.cidr
String. Sets the IPv4 node CIDR block. Nodes can only have one range
from each family. This CIDR block must match the pod CIDR described in
the For example: ipv4: cidr: "10.1.0.0/16" |
ClusterCIDRConfig resource | Required | Immutable |
ipv4.perNodeMaskSize
Integer. Defines the mask size for the node IPv4 CIDR block. For
example, the value |
ClusterCIDRConfig resource | Required | Immutable |
ipv6
Defines the configuration for the IPv6 CIDR range. At least one of the
|
ClusterCIDRConfig resource | Optional | Immutable |
ipv6.cidr
String. Sets the IPv6 node CIDR block. Nodes can only have one range from each family. For example: ipv6: cidr: "2620:0:1000:2631:3:10:3:0/112" |
ClusterCIDRConfig resource | Required | Immutable |
ipv6.perNodeMaskSize
Integer. Defines the mask size for the node IPv6 CIDR block. For
example, the value |
ClusterCIDRConfig resource | Required | Immutable |
nodeSelector.matchLabels
Defines which nodes the CIDR configuration is applicable to. An empty node selector functions as a default that applies to all nodes. For example: nodeSelector: matchLabels: baremetal.cluster.gke.io/node-pool: "workers" |
ClusterCIDRConfig resource | Optional | Mutable |