Flux image operations

Set the namespace (flux) where flux was installed for running fluxctl:

export FLUX_FORWARD_NAMESPACE=flux

Check how the git repository looks like in GitHub:

if [ -x /usr/bin/chromium-browser ]; then chromium-browser https://github.com/ruzickap/k8s-flux-repository/ & fi

Examine git Flux repository:

git -C tmp/k8s-flux-repository ls

Output:

README.md
namespaces/cert-manager-ns.yaml
namespaces/harbor-ns.yaml
namespaces/istio-ns.yaml
namespaces/kubed-ns.yaml
releases/cert-manager-release.yaml
releases/harbor-release.yaml
releases/istio-init-release.yaml
releases/istio-release.yaml
releases/kubed-release.yaml
workloads/cert-manager-00-crds.yaml
workloads/harbor-services.yaml
workloads/istio-gateway.yaml
workloads/istio-services.yaml

Install podinfo application using Flux:

envsubst << EOF > tmp/k8s-flux-repository/workloads/podinfo.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: podinfo
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: podinfo
  template:
    metadata:
      labels:
        app: podinfo
    spec:
      containers:
      - name: podinfo
        image: "stefanprodan/podinfo:2.1.2"
        ports:
        - containerPort: 9898
---
apiVersion: v1
kind: Service
metadata:
  name: podinfo-service
  namespace: default
  labels:
    app: podinfo
spec:
  type: ClusterIP
  selector:
    app: podinfo
  ports:
  - name: podinfo-http
    port: 9898
    protocol: TCP
    targetPort: 9898
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: podinfo-http-virtual-service
  namespace: default
spec:
  hosts:
  - podinfo.${MY_DOMAIN}
  gateways:
  - istio-system/istio-autogenerated-k8s-ingress
  http:
  - route:
    - destination:
        host: podinfo-service.default.svc.cluster.local
        port:
          number: 9898
EOF

Add it to the git repository and let Flux to deploy the application:

git -C tmp/k8s-flux-repository add --verbose .
git -C tmp/k8s-flux-repository commit -m "Add podinfo"
git -C tmp/k8s-flux-repository push -q
fluxctl sync
sleep 40

Output:

add 'workloads/podinfo.yaml'
[master 9c58afd] Add podinfo
 1 file changed, 54 insertions(+)
 create mode 100644 workloads/podinfo.yaml
Synchronizing with git@github.com:ruzickap/k8s-flux-repository
Revision of master to apply is 9c58afd
Waiting for 9c58afd to be applied ...
Done.

Start the web browser with https://podinfo.mylabs.dev:

curl http://podinfo.mylabs.dev
echo
if [ -x /usr/bin/chromium-browser ]; then chromium-browser --incognito https://podinfo.mylabs.dev & fi

Output:

{
  "hostname": "podinfo-56c6447655-7pwld",
  "version": "2.1.2",
  "revision": "ab74d6ef0bd3c5f39090134f59b12837757e80b8",
  "color": "blue",
  "message": "greetings from podinfo v2.1.2",
  "goos": "linux",
  "goarch": "amd64",
  "runtime": "go1.12.7",
  "num_goroutine": "6",
  "num_cpu": "2"
}

TIP

Workloads refers to any cluster resource responsible for the creation of containers from versioned images - in Kubernetes these are objects such as Deployments, DaemonSets, StatefulSets and CronJobs.

Check whether Flux can see any running workloads:

fluxctl list-workloads

Output:

WORKLOAD                    CONTAINER  IMAGE                       RELEASE  POLICY
default:deployment/podinfo  podinfo    stefanprodan/podinfo:2.1.2  ready

Inspect which versions of the image are running in the workload:

fluxctl list-images --workload default:deployment/podinfo 2>/dev/null

Output:

WORKLOAD                    CONTAINER  IMAGE                 CREATED
default:deployment/podinfo  podinfo    stefanprodan/podinfo
                                       |   2.1.3             13 Aug 19 09:33 UTC
                                       |   latest            13 Aug 19 09:33 UTC
                                       '-> 2.1.2             13 Aug 19 07:53 UTC
                                           2.1.1             13 Aug 19 07:51 UTC
                                           2.1.0             07 Aug 19 13:18 UTC
                                           2.0.5             07 Aug 19 12:50 UTC
                                           2.0.4             07 Aug 19 12:48 UTC
                                           2.0.3             07 Aug 19 12:45 UTC
                                           2.0.2             07 Aug 19 12:41 UTC
                                           2.0.1             07 Aug 19 12:39 UTC

Update all images belongs to "podinfo deployment":

fluxctl release --workload=default:deployment/podinfo --user=pruzicka --message="Update all podinfo images" --update-all-images

Output:

Submitting release ...
WORKLOAD                    STATUS   UPDATES
default:deployment/podinfo  success  podinfo: stefanprodan/podinfo:2.1.2 -> 2.1.3
Commit pushed:  f5587fe
Commit applied: f5587fe

Check the git repository:

git -C tmp/k8s-flux-repository pull -q
git -C tmp/k8s-flux-repository show

Output:

commit f5587fe7887e294b88da88490a17eed61e69b012 (HEAD -> master, tag: flux-sync, origin/master)
Author: Flux <petr.ruzicka@gmail.com>
Date:   Thu Aug 29 08:07:13 2019 +0000

    Update all podinfo images

─────────────────────────────────────────────────────────────────────────────────────────────
modified: workloads/podinfo.yaml
─────────────────────────────────────────────────────────────────────────────────────────────
@ workloads/podinfo.yaml:1 @
---
apiVersion: apps/v1
kind: Deployment
metadata:
@ workloads/podinfo.yaml:19 @ spec:
    spec:
      containers:
      - name: podinfo
        image: "stefanprodan/podinfo:2.1.2"
        image: "stefanprodan/podinfo:2.1.3"
        ports:
        - containerPort: 9898
---

Verify the updated version:

fluxctl list-images --workload default:deployment/podinfo 2>/dev/null

Output:

WORKLOAD                    CONTAINER  IMAGE                 CREATED
default:deployment/podinfo  podinfo    stefanprodan/podinfo
                                       '-> 2.1.3             13 Aug 19 09:33 UTC
                                           latest            13 Aug 19 09:33 UTC
                                           2.1.2             13 Aug 19 07:53 UTC
                                           2.1.1             13 Aug 19 07:51 UTC
...

Correct image version should be also visible directly from the pod:

kubectl describe pods | grep Image:

Output:

    Image:          stefanprodan/podinfo:2.1.3

Rolling back a workload

Roll back the podinfo image to previous version (2.1.2)

fluxctl release --workload=default:deployment/podinfo --update-image=stefanprodan/podinfo:2.1.2

Output:

Submitting release ...
WORKLOAD                    STATUS   UPDATES
default:deployment/podinfo  success  podinfo: stefanprodan/podinfo:2.1.3 -> 2.1.2
Commit pushed:  55c8ed1
Commit applied: 55c8ed1

Check the git repository:

git -C tmp/k8s-flux-repository pull -q
git -C tmp/k8s-flux-repository show

Output:

commit 55c8ed19b9558e87cf1811f1c9bc0fef576b2e7e (HEAD -> master, origin/master)
Author: Flux <petr.ruzicka@gmail.com>
Date:   Thu Aug 29 08:08:59 2019 +0000

    Release stefanprodan/podinfo:2.1.2 to default:deployment/podinfo

─────────────────────────────────────────────────────────────────────────────────────────────
modified: workloads/podinfo.yaml
─────────────────────────────────────────────────────────────────────────────────────────────
@ workloads/podinfo.yaml:19 @ spec:
    spec:
      containers:
      - name: podinfo
        image: "stefanprodan/podinfo:2.1.3"
        image: "stefanprodan/podinfo:2.1.2"
        ports:
        - containerPort: 9898
---

Verify the image version:

fluxctl list-images --workload=default:deployment/podinfo 2>/dev/null

Output:

WORKLOAD                    CONTAINER  IMAGE                 CREATED
default:deployment/podinfo  podinfo    stefanprodan/podinfo
                                       |   2.1.3             13 Aug 19 09:33 UTC
                                       |   latest            13 Aug 19 09:33 UTC
                                       '-> 2.1.2             13 Aug 19 07:53 UTC
                                           2.1.1             13 Aug 19 07:51 UTC
...

Image Tag Filtering

Set tag for the image:

fluxctl policy --workload=default:deployment/podinfo --tag-all='2.0.*'

Output:

WORKLOAD                    STATUS   UPDATES
default:deployment/podinfo  success
Commit pushed:  3744220

See what was pushed to git repository:

git -C tmp/k8s-flux-repository pull -q
git -C tmp/k8s-flux-repository show

Output:

commit 3744220888d3937b358fb6fc1a88ddf99dbea59a (HEAD -> master, origin/master)
Author: Flux <petr.ruzicka@gmail.com>
Date:   Thu Aug 29 08:10:24 2019 +0000

    Updated policies: default:deployment/podinfo

─────────────────────────────────────────────────────────────────────────────────────────────
modified: workloads/podinfo.yaml
─────────────────────────────────────────────────────────────────────────────────────────────
@ workloads/podinfo.yaml:7 @ kind: Deployment
metadata:
  name: podinfo
  namespace: default
  annotations:
    flux.weave.works/tag.podinfo: glob:2.0.*
spec:
  replicas: 1
  selector:

Instruct Flux to update all images for podinfo:

fluxctl release --workload=default:deployment/podinfo --update-all-images

Output:

Submitting release ...
WORKLOAD                    STATUS   UPDATES
default:deployment/podinfo  success  podinfo: stefanprodan/podinfo:2.1.2 -> 2.0.5
Commit pushed:  dd0af19
Commit applied: dd0af19

Check the git repository:

git -C tmp/k8s-flux-repository pull -q
git -C tmp/k8s-flux-repository show

Output:

commit dd0af19b22b0db8c9e448cae5248a23800d52803 (HEAD -> master, origin/master)
Author: Flux <petr.ruzicka@gmail.com>
Date:   Thu Aug 29 08:10:58 2019 +0000

    Release all latest to default:deployment/podinfo

─────────────────────────────────────────────────────────────────────────────────────────────
modified: workloads/podinfo.yaml
─────────────────────────────────────────────────────────────────────────────────────────────
@ workloads/podinfo.yaml:21 @ spec:
    spec:
      containers:
      - name: podinfo
        image: "stefanprodan/podinfo:2.1.2"
        image: "stefanprodan/podinfo:2.0.5"
        ports:
        - containerPort: 9898
---

Check the versions running in the workload:

fluxctl list-images --workload=default:deployment/podinfo 2>/dev/null

Output:

WORKLOAD                    CONTAINER  IMAGE                 CREATED
default:deployment/podinfo  podinfo    stefanprodan/podinfo
                                       |   2.1.3             13 Aug 19 09:33 UTC
                                       |   latest            13 Aug 19 09:33 UTC
                                       |   2.1.2             13 Aug 19 07:53 UTC
                                       |   2.1.1             13 Aug 19 07:51 UTC
                                       |   2.1.0             07 Aug 19 13:18 UTC
                                       '-> 2.0.5             07 Aug 19 12:50 UTC
                                           2.0.4             07 Aug 19 12:48 UTC
...

Automated container image installation

Open the Harbor container repository: https://harbor.mylabs.dev

if [ -x /usr/bin/chromium-browser ]; then chromium-browser https://harbor.mylabs.dev & fi

Create https://github.com/kubernetes-up-and-running/kuard application "manifest":

envsubst << EOF > tmp/k8s-flux-repository/workloads/kuard.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kuard
  namespace: default
  annotations:
    flux.weave.works/automated: "true"
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kuard
  template:
    metadata:
      labels:
        app: kuard
    spec:
      containers:
      - name: kuard
        image: "harbor.${MY_DOMAIN}/library/kuard:v1"
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: kuard-service
  namespace: default
  labels:
    app: kuard
spec:
  type: ClusterIP
  selector:
    app: kuard
  ports:
  - name: kuard-http
    port: 8080
    protocol: TCP
    targetPort: 8080
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: kuard-http-virtual-service
  namespace: default
spec:
  hosts:
  - kuard.${MY_DOMAIN}
  gateways:
  - istio-system/istio-autogenerated-k8s-ingress
  http:
  - route:
    - destination:
        host: kuard-service.default.svc.cluster.local
        port:
          number: 8080
EOF

Add it to the git repository and let Flux to deploy the application:

git -C tmp/k8s-flux-repository pull -q
git -C tmp/k8s-flux-repository add --verbose .
git -C tmp/k8s-flux-repository commit -m "Add kuard"
git -C tmp/k8s-flux-repository push -q
fluxctl sync
COUNTER=0; while [ $COUNTER -lt 12 ] ; do COUNTER=$((COUNTER+1)); fluxctl list-images --workload default:deployment/kuard 2>/dev/null; sleep 5; done

Output:

add 'workloads/kuard.yaml'
[master d792a6f] Add kuard
 1 file changed, 56 insertions(+)
 create mode 100644 workloads/kuard.yaml
Synchronizing with git@github.com:ruzickap/k8s-flux-repository
Revision of master to apply is d792a6f
Waiting for d792a6f to be applied ...
Done.
...
WORKLOAD                  CONTAINER  IMAGE                            CREATED
default:deployment/kuard  kuard      harbor.mylabs.dev/library/kuard  image data not available
                                     '-> v1                           ?
...
WORKLOAD                  CONTAINER  IMAGE                            CREATED
default:deployment/kuard  kuard      harbor.mylabs.dev/library/kuard
                                     '-> v1                           29 Aug 19 07:47 UTC
...

Open the page: https://kuard.mylabs.dev

if [ -x /usr/bin/chromium-browser ]; then chromium-browser --incognito https://kuard.mylabs.dev & fi

Change the VERSION environment variable:

sed -i "s/ENV VERSION=test/ENV VERSION=new_version/" tmp/kuard/Dockerfile
git -C tmp/kuard/ diff Dockerfile

Output:

─────────────────────────────────────────────────────────────────────────────────────────────
modified: Dockerfile
─────────────────────────────────────────────────────────────────────────────────────────────
@ Dockerfile:19 @ COPY . .
ENV VERBOSE=0
ENV PKG=github.com/kubernetes-up-and-running/kuard
ENV ARCH=amd64
ENV VERSION=test
ENV VERSION=new_version

# Do the build. Script is part of incoming sources.
RUN build/build.sh

Build kuard container image and push it to harbor.mylabs.dev/library/kuard:v2:

docker build --tag harbor.${MY_DOMAIN}/library/kuard:v2 tmp/kuard
echo admin | docker login --username admin --password-stdin harbor.${MY_DOMAIN}
docker push harbor.${MY_DOMAIN}/library/kuard:v2
COUNTER=0; while [ $COUNTER -lt 12 ] ; do COUNTER=$((COUNTER+1)); fluxctl list-images --workload default:deployment/kuard 2>/dev/null; sleep 5; done

Output:

Sending build context to Docker daemon  3.378MB
Step 1/14 : FROM golang:1.12-alpine AS build
 ---> e0d646523991
...
Step 14/14 : CMD [ "/kuard" ]
 ---> Using cache
 ---> 652d18a08b2d
Successfully built 652d18a08b2d
Successfully tagged harbor.mylabs.dev/library/kuard:v2
WARNING! Your password will be stored unencrypted in /home/pruzicka/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
The push refers to repository [harbor.mylabs.dev/library/kuard]
4af36e68af9b: Pushed
03901b4a2ea8: Layer already exists
v2: digest: sha256:14e584451dfd99dcd34e67226701786b952a8c5be11ba6091e64d690364bc646 size: 739
...
WORKLOAD                  CONTAINER  IMAGE                            CREATED
default:deployment/kuard  kuard      harbor.mylabs.dev/library/kuard
                                     '-> v1                           29 Aug 19 07:47 UTC
WORKLOAD                  CONTAINER  IMAGE                            CREATED
default:deployment/kuard  kuard      harbor.mylabs.dev/library/kuard
                                     |   v2                           29 Aug 19 07:49 UTC
                                     '-> v1                           29 Aug 19 07:47 UTC
WORKLOAD                  CONTAINER  IMAGE                            CREATED
default:deployment/kuard  kuard      harbor.mylabs.dev/library/kuard
                                     |   v2                           29 Aug 19 07:49 UTC
                                     '-> v1                           29 Aug 19 07:47 UTC
WORKLOAD                  CONTAINER  IMAGE                            CREATED
default:deployment/kuard  kuard      harbor.mylabs.dev/library/kuard
                                     |   v2                           29 Aug 19 07:49 UTC
                                     '-> v1                           29 Aug 19 07:47 UTC
WORKLOAD                  CONTAINER  IMAGE                            CREATED
default:deployment/kuard  kuard      harbor.mylabs.dev/library/kuard
                                     '-> v2                           29 Aug 19 07:49 UTC
                                         v1                           29 Aug 19 07:47 UTC

Check the git repository:

git -C tmp/k8s-flux-repository pull -q
git -C tmp/k8s-flux-repository show

Output:

commit c8e39248374934a8f9b4f184deb689eedb8f8cb4 (HEAD -> master, origin/master)
Author: Flux <petr.ruzicka@gmail.com>
Date:   Thu Aug 29 08:21:43 2019 +0000

    Auto-release harbor.mylabs.dev/library/kuard:v2

─────────────────────────────────────────────────────────────────────────────────────────────
modified: workloads/kuard.yaml
─────────────────────────────────────────────────────────────────────────────────────────────
@ workloads/kuard.yaml:1 @
---
apiVersion: apps/v1
kind: Deployment
metadata:
@ workloads/kuard.yaml:21 @ spec:
    spec:
      containers:
      - name: kuard
        image: "harbor.mylabs.dev/library/kuard:v1"
        image: "harbor.mylabs.dev/library/kuard:v2"
        ports:
        - containerPort: 8080
---

Remove the applications using git commit

See the running pods:

kubectl get virtualservice,service,deployment,pods

Output:

NAME                                                              GATEWAYS                                         HOSTS                  AGE
virtualservice.networking.istio.io/kuard-http-virtual-service     [istio-system/istio-autogenerated-k8s-ingress]   [kuard.mylabs.dev]     14m
virtualservice.networking.istio.io/podinfo-http-virtual-service   [istio-system/istio-autogenerated-k8s-ingress]   [podinfo.mylabs.dev]   29m

NAME                      TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/kuard-service     ClusterIP   100.70.120.183   <none>        8080/TCP   14m
service/kubernetes        ClusterIP   100.64.0.1       <none>        443/TCP    55m
service/podinfo-service   ClusterIP   100.70.190.65    <none>        9898/TCP   29m

NAME                            READY   UP-TO-DATE   AVAILABLE   AGE
deployment.extensions/kuard     1/1     1            1           14m
deployment.extensions/podinfo   1/1     1            1           29m

NAME                           READY   STATUS    RESTARTS   AGE
pod/kuard-5b8478d4-nksgf       1/1     Running   0          10m
pod/podinfo-5f4bf4fd57-clsmm   1/1     Running   0          21m

Let's remove the podinfo and kuard application:

rm tmp/k8s-flux-repository/workloads/podinfo.yaml tmp/k8s-flux-repository/workloads/kuard.yaml
git -C tmp/k8s-flux-repository add --verbose .
git -C tmp/k8s-flux-repository commit -m "Remove podinfo and kuard"
git -C tmp/k8s-flux-repository push -q
fluxctl sync

Output:

remove 'workloads/kuard.yaml'
remove 'workloads/podinfo.yaml'
[master 594fe1e] Remove podinfo and kuard
 2 files changed, 114 deletions(-)
 delete mode 100644 workloads/kuard.yaml
 delete mode 100644 workloads/podinfo.yaml
Synchronizing with git@github.com:ruzickap/k8s-flux-repository
Revision of master to apply is 594fe1e
Waiting for 594fe1e to be applied ...
Done.

Check the pods - Flux should remove the podinfo pod:

kubectl get virtualservice,service,deployment,pods

Output:

NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
service/kubernetes   ClusterIP   100.64.0.1   <none>        443/TCP   57m