Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove patching of istio; update values for seccompvalue via chart #2938

Open
wants to merge 4 commits into
base: istio-cni-by-default-with-seccomp-value
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 0 additions & 10 deletions .github/workflows/pss_test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,16 +33,6 @@ jobs:
- name: Install Istio CNI
run: ./tests/gh-actions/install_istio-cni.sh

- name: Configure istio init container with seccompProfile attribute
run: |
kubectl get cm istio-sidecar-injector -n istio-system -o yaml > temporary_patch.yaml
sed -i '0,/runAsNonRoot: true/{s//&\n seccompProfile:\n type: RuntimeDefault/}' temporary_patch.yaml
sed -i '/runAsNonRoot: true/{N; /runAsUser: {{ .ProxyUID | default "1337" }}/a\
seccompProfile:\n type: RuntimeDefault
}' temporary_patch.yaml
kubectl apply -f temporary_patch.yaml
rm temporary_patch.yaml

- name: Install oauth2-proxy
run: ./tests/gh-actions/install_oauth2-proxy.sh

Expand Down
4 changes: 3 additions & 1 deletion common/istio-1-23/istio-install/base/install.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2336,7 +2336,9 @@ data:
values: |-
{
"gateways": {
"seccompProfile": {},
"seccompProfile": {
"type": RuntimeDefault
},
"securityContext": {}
},
"global": {
Expand Down
4 changes: 3 additions & 1 deletion common/istio-1-23/profile.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,14 @@ spec:
enabled: true
hub: docker.io/istio
profile: default
tag: 1.23.2
tag: 1.24.2
values:
defaultRevision: ""
gateways:
istio-egressgateway: {}
istio-ingressgateway: {}
seccompProfile:
type: RuntimeDefault
global:
configValidation: true
istioNamespace: istio-system
Expand Down
94 changes: 94 additions & 0 deletions common/istio-1-24/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# Istio

## Upgrade Istio Manifests

Istio ships with an installer called `istioctl`, which is a deployment /
debugging / configuration management tool for Istio all in one package.
In this section, we explain how to upgrade our istio kustomize packages
by leveraging `istioctl`. Assuming the new version is `X.Y.Z` and the
old version is `X1.Y1.Z1`:

1. Make a copy of the old istio manifests tree, which will become the
kustomization for the new Istio version:

$ export MANIFESTS_SRC=<path/to/manifests/repo>
$ export ISTIO_OLD=$MANIFESTS_SRC/common/istio-X1-Y1
$ export ISTIO_NEW=$MANIFESTS_SRC/common/istio-X-Y
$ cp -a $ISTIO_OLD $ISTIO_NEW

2. Download `istioctl` for version `X.Y.Z`:

$ ISTIO_VERSION="X.Y.Z"
$ wget "https://github.com/istio/istio/releases/download/${ISTIO_VERSION}/istio-${ISTIO_VERSION}-linux-amd64.tar.gz"
$ tar xvfz istio-${ISTIO_VERSION}-linux-amd64.tar.gz
# sudo mv istio-${ISTIO_VERSION}/bin/istioctl /usr/local/bin/istioctl

3. Use `istioctl` to generate an `IstioOperator` resource, the
CustomResource used to describe the Istio Control Plane:

$ cd $ISTIO_NEW
$ istioctl profile dump default > profile.yaml

---
**NOTE**

`istioctl` comes with a bunch of [predefined profiles](https://istio.io/latest/docs/setup/additional-setup/config-profiles/)
(`default`, `demo`, `minimal`, etc.). The `default` profile is installed by default.

---

4. Generate manifests and add them to their respective packages. We
will generate manifests using `istioctl`, the
`profile.yaml` file from upstream and the
`profile-overlay.yaml` file that contains our desired
changes:

$ export PATH="$MANIFESTS_SRC/scripts:$PATH"
$ cd $ISTIO_NEW
$ istioctl manifest generate --cluster-specific -f profile.yaml -f profile-overlay.yaml > dump.yaml
$ ./split-istio-packages -f dump.yaml
$ mv $ISTIO_NEW/crd.yaml $ISTIO_NEW/istio-crds/base
$ mv $ISTIO_NEW/install.yaml $ISTIO_NEW/istio-install/base
$ mv $ISTIO_NEW/cluster-local-gateway.yaml $ISTIO_NEW/cluster-local-gateway/base
$ rm dump.yaml

---
**NOTE**

`split-istio-packages` is a python script in the same folder as this file.
The `ruamel.yaml` version used is 0.16.12.

`--cluster-specific` is a flag that determines if a current K8s cluster context will be used to dynamically
detect default settings. Ensure you have a target cluster ready before running the above commands.
We set this flag because `istioctl manifest generate` generates manifest files with resources that are no
longer supported in Kubernetes 1.25 (`policy/v1beta1`). See: https://github.com/istio/istio/issues/41220

---

## Changes to Istio's upstream manifests

### Changes to the upstream IstioOperator profile

Changes to Istio's upstream profile `default` are the following:

- Add a `cluster-local-gateway` component for Kserve. Knative-local-gateway is now obsolete https://github.com/kubeflow/manifests/pull/2355/commits/adc00b804404ea08685a044ae595be0bed9adb59.
- Disable the EgressGateway component. We do not use it and it adds unnecessary complexity.

Those changes are captured in the [profile-overlay.yaml](profile-overlay.yaml)
file.

### Changes to the upstream manifests using kustomize

The Istio kustomizations make the following changes:

- Remove PodDisruptionBudget from `istio-install` and `cluster-local-gateway` kustomizations. See:
- https://github.com/istio/istio/issues/12602
- https://github.com/istio/istio/issues/24000
- Add Istio AuthorizationPolicy to allow all requests to the Istio Ingressgateway and the Istio cluster-local gateway.
- Add Istio AuthorizationPolicy in Istio's root namespace, so that sidecars deny traffic by default (explicit deny-by-default authorization model).
- Add Gateway CRs for the Istio Ingressgateway and the Istio cluster-local gateway, as `istioctl` stopped generating them in later versions.
- Add the istio-system namespace object to `istio-namespace`, as `istioctl` stopped generating it in later versions.
- Configure TCP KeepAlives.
- Disable tracing as it causes DNS breakdown. See:
https://github.com/istio/istio/issues/29898
- Set ENABLE_DEBUG_ON_HTTP=false according to https://istio.io/latest/docs/ops/best-practices/security/#control-plane
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,12 @@ kind: ServiceAccount
metadata:
labels:
app: cluster-local-gateway
app.kubernetes.io/instance: istio
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: istio-ingressgateway
app.kubernetes.io/part-of: istio
app.kubernetes.io/version: 1.24.2
helm.sh/chart: istio-ingress-1.24.2
install.operator.istio.io/owning-resource: unknown
istio: cluster-local-gateway
istio.io/rev: default
Expand All @@ -16,6 +22,12 @@ kind: Deployment
metadata:
labels:
app: cluster-local-gateway
app.kubernetes.io/instance: istio
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: istio-ingressgateway
app.kubernetes.io/part-of: istio
app.kubernetes.io/version: 1.24.2
helm.sh/chart: istio-ingress-1.24.2
install.operator.istio.io/owning-resource: unknown
istio: cluster-local-gateway
istio.io/rev: default
Expand All @@ -42,7 +54,13 @@ spec:
sidecar.istio.io/inject: 'false'
labels:
app: cluster-local-gateway
app.kubernetes.io/instance: istio
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: istio-ingressgateway
app.kubernetes.io/part-of: istio
app.kubernetes.io/version: 1.24.2
chart: gateways
helm.sh/chart: istio-ingress-1.24.2
heritage: Tiller
install.operator.istio.io/owning-resource: unknown
istio: cluster-local-gateway
Expand Down Expand Up @@ -109,7 +127,8 @@ spec:
- name: ISTIO_META_WORKLOAD_NAME
value: cluster-local-gateway
- name: ISTIO_META_OWNER
value: kubernetes://apis/apps/v1/namespaces/istio-system/deployments/cluster-local-gateway
value:
kubernetes://apis/apps/v1/namespaces/istio-system/deployments/cluster-local-gateway
- name: ISTIO_META_MESH_ID
value: cluster.local
- name: TRUST_DOMAIN
Expand All @@ -122,7 +141,7 @@ spec:
valueFrom:
fieldRef:
fieldPath: spec.nodeName
image: docker.io/istio/proxyv2:1.23.2
image: gcr.io/istio-testing/proxyv2:1.24.2
name: istio-proxy
ports:
- containerPort: 15020
Expand Down Expand Up @@ -235,6 +254,12 @@ kind: PodDisruptionBudget
metadata:
labels:
app: cluster-local-gateway
app.kubernetes.io/instance: istio
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: istio-ingressgateway
app.kubernetes.io/part-of: istio
app.kubernetes.io/version: 1.24.2
helm.sh/chart: istio-ingress-1.24.2
install.operator.istio.io/owning-resource: unknown
istio: cluster-local-gateway
istio.io/rev: default
Expand All @@ -253,6 +278,12 @@ apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app.kubernetes.io/instance: istio
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: istio-ingressgateway
app.kubernetes.io/part-of: istio
app.kubernetes.io/version: 1.24.2
helm.sh/chart: istio-ingress-1.24.2
install.operator.istio.io/owning-resource: unknown
istio.io/rev: default
operator.istio.io/component: IngressGateways
Expand All @@ -273,6 +304,12 @@ apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app.kubernetes.io/instance: istio
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: istio-ingressgateway
app.kubernetes.io/part-of: istio
app.kubernetes.io/version: 1.24.2
helm.sh/chart: istio-ingress-1.24.2
install.operator.istio.io/owning-resource: unknown
istio.io/rev: default
operator.istio.io/component: IngressGateways
Expand All @@ -292,6 +329,12 @@ kind: HorizontalPodAutoscaler
metadata:
labels:
app: cluster-local-gateway
app.kubernetes.io/instance: istio
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: istio-ingressgateway
app.kubernetes.io/part-of: istio
app.kubernetes.io/version: 1.24.2
helm.sh/chart: istio-ingress-1.24.2
install.operator.istio.io/owning-resource: unknown
istio: cluster-local-gateway
istio.io/rev: default
Expand Down Expand Up @@ -320,6 +363,12 @@ metadata:
annotations:
labels:
app: cluster-local-gateway
app.kubernetes.io/instance: istio
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: istio-ingressgateway
app.kubernetes.io/part-of: istio
app.kubernetes.io/version: 1.24.2
helm.sh/chart: istio-ingress-1.24.2
install.operator.istio.io/owning-resource: unknown
istio: cluster-local-gateway
istio.io/rev: default
Expand All @@ -331,11 +380,9 @@ spec:
ports:
- name: status-port
port: 15020
protocol: TCP
targetPort: 15020
- name: http2
port: 80
protocol: TCP
targetPort: 8080
selector:
app: cluster-local-gateway
Expand Down
Loading