Unable to connect to application on EKS via Istio with deny-all GlobalNetworkPolicy

Question

My setup: Running EKS 1.14 with Calico and Istio 1.5.

Trying to get the sample bookinfo to run with specific NetworkPolicies.

I've applied a GlobalNetworkPolicy that denies all traffic:

apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
  name: default-deny
spec:
  selector: all()
  types:
  - Ingress
  - Egress

I also added a GlobalNetworkPolicy for Istio to namespace and intra-namespace traffic:

apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
  name: allow-istio-system-to-ns
spec:
  selector: all()
  namespaceSelector: istio-injection == 'enabled'
  types:
  - Ingress
  - Egress
  ingress:
  - action: Allow
    source:
      selector: all()
      namespaceSelector: projectcalico.org/namespace == 'istio-system'
  - action: Allow
    source:
      selector: all()
      namespaceSelector: istio-injection == 'enabled'
  egress:
  - action: Allow

and a NetworkPolicy allowing all ingress and egress on istio-system

apiVersion: projectcalico.org/v3
kind: NetworkPolicy
metadata:
  name: allow-istio-system-all
  namespace: istio-system
spec:
  selector: all()
  types:
  - Ingress
  - Egress
  ingress:
  - action: Allow 
  egress:
  - action: Allow

And a NetworkPolicy to allow traffic to the ingress-gateway on ports 80 and 443. I know this one is redundant, but I was hoping to cut down the previous one to only necessary ingress.

apiVersion: projectcalico.org/v3
kind: NetworkPolicy
metadata:
  name: allow-istio-ingress-on-80-443
  namespace: istio-system
spec:
  selector: app == 'istio-ingressgateway'
  ingress:
    - action: Allow
      protocol: TCP
      destination:
        ports:
          - 80
          - 443

Also some other, unrelated NetworkPolicies (access to kube-dns, metrics-server). Deploying bookinfo works with and without the policies and same with deploying the Gateway.

The connection between the components works (I can exec into one pod and connect to the others). But when I try to access the productpage via the gateway with the policies I get nothing back, instead of the HTML I get without the policies.

Am I missing some traffic that should be allowed? Should I have policies for the master nodes or for the loadbalancer somewhere else?

Edit: If I allow all ingress into istio-system and into the namespace, it works. So I think I'm just missing some other ingress from the namespace, is there a way to limit it to just the loadbalancer?

Piotr Malec Piotr Malec · Accepted Answer · 2020-03-13T15:53:44

First of all there is a typo in Your allow-istio-system-to-ns yaml:

      namespaceSelector: projectcalico.org/namespace == 'istio-system

There should be another ' at the end of the line.

Secondly this could be caused by the changes to policy and mixer in istio version 1.5.

According to Istio documentation:

The mixer policy is deprecated in Istio 1.5 and not recommended for production usage.

Rate limiting: Consider using Envoy native rate limiting instead of mixer rate limiting. Istio will add support for native rate limiting API through the Istio extensions API.

Control headers and routing: Consider using Envoy ext_authz filter, lua filter, or write a filter using the Envoy-wasm sandbox.

Denials and White/Black Listing: Please use the Authorization Policy for enforcing access control to a workload.

There is a guide in istio documentation which allows to turn back on depreciated features:

For an existing Istio mesh

Check the status of policy enforcement for your mesh.
$ kubectl -n istio-system get cm istio -o jsonpath="{@.data.mesh}" | grep disablePolicyChecks
disablePolicyChecks: true
If policy enforcement is enabled (disablePolicyChecks is false), no further action is needed.

Update the istio configuration to enable policy checks.

Execute the following command from the root Istio directory:
$ istioctl manifest apply --set values.global.disablePolicyChecks=false --set values.pilot.policy.enabled=true
configuration "istio" replaced
Validate that policy enforcement is now enabled.
$ kubectl -n istio-system get cm istio -o jsonpath="{@.data.mesh}" | grep disablePolicyChecks
disablePolicyChecks: false

Note that on calico documentation it says it requires the following istio versions:

Istio v1.0, v1.1, v1.2, or v1.3

Hope it helps.

Unable to connect to application on EKS via Istio with deny-all GlobalNetworkPolicy

2 Answers