A working CI/CD-pipeline
This is a tutorial to make a working CI/CD-pipeline, that uses gitlab for version control. The pipeline works as followed: 1. You commit a change to your product and the pipeline is triggered 2. Gitlab runner will build an image from your repository and push the image into your project's container register in Gitlab 3. Gitlab runner will also push the image to your project's container register in Google Cloud's 4. Runner will build a container and deploy the product to Google Cloud into a Kubernetes cluster, so you can access your product
NOTE: In this example our product is just a simple html file that prints out "Hello World".
First you need a google account and credits on your google cloud
Go to google cloud. Log in from top right corner and make sure you have enough credits by clicking navigation menu on the top left corner and select "billing". Once you have confirmed you have the credits, we'll start setting up the project.
Creating a new project
We'll be using the google cloud shell tool, so open it from top right corner. We'll use the command
gcloud init
to start initializing our project. When asked to pick a configuration to use, you can select 1 to use the already existing (default) configuration. Next you have to choose your account, select the one you are logged in as by choosing 1. Next you have to create a new project and you must enter a unique ID for your project. NOTE: ID's have to be unique across all projects ever made. If the ID is already used, you can restart the project creating process with the command gcloud init. Once the project is set up, you should see on the command prompt something similar to this:
username@cloudshell:~ (project_name)$
Creating a new Kubernetes cluster
Now we are ready to create our kubernetes cluster for our project. Use the command
gcloud container clusters create cluster_name --zone=europe-west1-b
If you get an error saying you need to enable Kubernetes Engine API, please do so (you also need to set up billing for this and wait for few minutes for the API to get activated).
Now you should have your cluster up and running. Next we need to get credentials for the cluster, this way all the commands take effect on the chosen cluster. Use the command
gcloud container clusters get-credentials cluster_name --zone=europe-west1-b
Now we have a project up and Kubernetes cluster of virtual machines running.
Next we will reserve an IP address for our cluster, so we can access our services with that IP. We use the following command:
gcloud compute addresses create demo-dev-ip --region europe-west1
where demo-dev-ip is just a name for the IP(s) you create. Now the cluster should have an IP address and we can check it with the following command:
gcloud compute addresses describe demo-dev-ip --region europe-west1
Get credentials for Google Cloud
We still need to connect the project in Google Cloud to our Gitlab project by using service account key. You can create the service account key for your project in Google Cloud in APIs & Services --> Credentials. Choose Create credentials and select Service accout key, then select Computer Engine default service account as your service account and JSON as key type. Open the JSON file created with any text editor and copy the contents to your Gitlab project. Do this by selecting Settings --> CI / CD --> Variables. Input SERVICE_ACCOUNT_KEY as Key and the copied contents to value and save variables.
Now you should be ready to start configuring your CI/CD-pipeline.
Required files for the pipeline
In order to work, we need to have at least three files where we configure the CI/CD-pipeline. First one is already familiar, the .gitlab-ci.yml. Here is an example for it
stages:
- build_product
- deploy_product
variables:
CLOUD_IMAGE_NAME: gcr.io/wimmalab/test-project # replace with your google cloud image name, so that gcr.io is your register, wimmalab your project an test-project your image
CLOUD_REGISTRY: gcr.io # google cloud register, this is the default
CLOUD_PROJECT: wimmalab # replace with google cloud project name
CLUSTER_NAME: eka # replace with your cluster name
build_product:
stage: build_product
image: docker:git
variables:
DOCKER_DRIVER: overlay2
services:
- docker:dind
before_script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY # logs in to gitlab container registry
script:
- docker build --cache-from $CI_REGISTRY_IMAGE --tag $CI_REGISTRY_IMAGE . # builds an image from current folder (.) and tags it with the correct name
- docker push $CI_REGISTRY_IMAGE # pushes the image to gitlab's registry
- docker tag $CI_REGISTRY_IMAGE:latest $CLOUD_IMAGE_NAME # renames the image for google cloud
- echo "$SERVICE_ACCOUNT_KEY" > key.json # gets credentials
- cat key.json | docker login -u _json_key --password-stdin $CLOUD_REGISTRY # logs in to google cloud registry, gcr.io is the default
- docker push $CLOUD_IMAGE_NAME # pushes the image to google cloud registry
only:
- dev
deploy_product:
stage: deploy_product
image: gitlab.labranet.jamk.fi:4567/fullstack-examples/google-cloud-sdk
script:
- echo "$SERVICE_ACCOUNT_KEY" > key.json
- gcloud auth activate-service-account --key-file=key.json
- gcloud config set project $CLOUD_PROJECT # sets project to your project
- gcloud config set container/cluster $CLUSTER_NAME # sets cluster to your cluster
- gcloud config set compute/zone europe-west1-b
- gcloud container clusters get-credentials $CLUSTER_NAME --zone europe-west1-b # gets credentials to use kubernetes cluster
# New deployment
- kubectl delete deployments demo
- kubectl delete services demo
- kubectl apply -f Deployment.yaml
only:
- dev
The second file you need is Dockerfile which we use to build an image of our repository. In this case, when the product is just a simple HTML-page, we use nginx web server running on alpine linux, so our Dockerfile will look like this:
The third file we need is used to deploy the product to Google Cloud, here it is called deployment.yaml. Here is an example of the file:
apiVersion: apps/v1
kind: Deployment
metadata:
name: demo
labels:
app: demo
spec:
selector:
matchLabels:
app: demo
template:
metadata:
labels:
app: demo
spec:
containers:
- name: demo
image: REGISTRY/PROJECT/IMAGE:latest # replace these with the correct ones
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
labels:
app: demo
name: demo
namespace: default
spec:
type: LoadBalancer
loadBalancerIP: 35.205.246.172 # replace with the IP your cluster has
ports:
- port: 8080 # target port where your product is accessed from
targetPort: 80
selector:
app: demo
Now, if all we as it should, your website should be accessed in the IP and port mentined in the deployment.yaml. It might take a few minutes for the service to get the IP, you can check this in the Google Cloud shell by typing
kubectl get services
If you can't access your page, you can try to debug by typing
kubectl get pods
and
kubectl get deployments
You can remove your pods/deployments/services by typing
kubectl delete pods/deployments/services name
where pods/deployments/services is the type and name is the name.