GKE Networking Best Practices for Security and Operation

This is part two of our four-part GKE security blog series. Don’t forget to check out our previous blog post that covers security best practices for designing your GKE clusters.

Securing your GKE cluster’s network traffic and access is crucial for the entire cluster’s security and operation. Follow the below recommendations and best practices to protect your Kubernetes network on GKE.

Enable Network Policy

Why: By default, network traffic in a Kubernetes cluster can flow freely between pods and also leave the cluster network altogether. Creating restrictions to allow only necessary service-to-service and cluster egress connections decreases the number of potential targets for malicious or misconfigured pods and limits their ability to exploit the cluster resources.

GKE can install and manage the Calico CNI (Container Network Interface) for you. Calico offers the ability to control network traffic to and from the cluster’s pods by implementing the standard Kubernetes Network Policy API. Calico also offers some custom extensions to the standard policy type. Network Policies can control both ingress and egress traffic. Different pod attributes, like labels or namespaces, can be used, in addition to standard IP CIDR blocks, to define the rules.

What to do: Enable Network Policies in your GKE cluster, either at creation time or later. Create network policies to limit pod traffic only to what is required. Test the policies to make sure they block unwanted traffic while allowing required traffic.

Note that the NetworkPolicy resource type can exist in a cluster even though the cluster’s network setup does not actually support Network Policies. Therefore it would be possible to successfully create the policies, but they would have no effect.

Note that GKE Network Policy does not support Windows nodes at this time.

Create a List of Master Authorized Networks

Why: By default, the cluster API endpoint has a public IP address open to the world. To protect against future vulnerabilities in kube-apiserver and to limit use of stolen cluster or cloud credentials, limit network access to the API endpoint to trusted IP addresses.

What to do: If you cannot disable your cluster’s public API endpoint, create a list of master authorized networks to limit source client IP traffic to your endpoint.

Block Access to the Kubelet

Why: The kubelet runs on every node to manage the lifecycle of the pod containers assigned to that node. The kubelet needs strong protection because of its tight integration with the node’s container runtime. Even though GKE runs the kubelet with anonymous authentication disabled, blocking access to its port from the pod network provides additional safeguards for this critical service.

What to do: After enabling Network Policy in your GKE cluster, create a Calico GlobalNetworkPolicy to prevent all pods from connection to the kubelet’s port 10250/TCP.

apiVersion: crd.projectcalico.org/v1
kind: GlobalNetworkPolicy
metadata:
  name: deny-kubelet-port
spec:
  types:
  - Egress
  egress:
  - action: Deny
    protocol: TCP
    destination:
      nets:
      - 0.0.0.0/0
      ports:
      - 10250
    source: {}

Secure the Service Load Balancers

Why: By default, when creating a Kubernetes Service of type “LoadBalancer” in a GKE cluster, the cluster’s cloud controller creates an Internet-facing load balancing with no firewall restrictions.

What to do: If only sources inside the cluster’s VPC need to access the service’s endpoint, add the following annotation to the Kubernetes Service manifest to create an internal load balancer: cloud.google.com/load-balancer-type: \"Internal\"

If the load balancer needs to be Internet-facing but should not be open to all IP addresses, you can add the field loadBalancerSourceRanges to the Service specification to limit the IP address blocks allowed to connect.

Protect Cloud Metadata

Why: GKE nodes, like all Google Compute Engine virtual machines (VMs), use cloud metadata to pull the VMs’ cloud configuration and other data. This endpoint can also allow any network connection originating from the VM, including that from container processes, to access the metadata, which can include sensitive configurations or credentials.

What to do: Protect your nodes’ cloud metadata.