Service: Cloud Build¶

Overview¶

The CloudBuild is a serverless CICD platform within Google Cloud Platform

The table of attributes:

Goal is has related builds to provision services (Cloud Run, Cloud Storage, ...) within git trigger on repository

SAD - System Architechture Design¶

The logic flow for CloudBuild service:

flowchart LR

  %% Component
  subgraph gh[GitHub]
    repository[Repository]
  end

  subgraph gcp[Google Cloud Platform]

    subgraph project[Project]
      direction TB

      subgraph cloudbuild[CloudBuild]
        direction TB
        trigger[Trigger] --> execution[Execution] --> pool[Private Pool] --> runtime[Runtime Build]
      end

      subgraph log[Log Storage]
        gcs[Cloud Storage]
        clogging[Cloud Logging]
      end

      runtime -- [2] yield, write log records --> log
      other[Google Platform services]
    end
  end

  %% Flow
  repository -- [1] trigger webhook event --> cloudbuild
  cloudbuild <-- [2] interactive --> other
  cloudbuild -- [3] send metadata, log --> repository

The flow follow following component:

[1] Developer interactive with code in GitHub repository, example: create PR, push commit, ... will send payload to handle in CloudBuild

See: GitHub events and payloads

[2] CloudBuild based on the payload then filtered with trigger rules.

[2] The trigger then execute a build and send it into Private pool

[2] Private pool queue build and actual execution it.

[3] In that time, the logs has been writed to configed storage and send back metadata into GitHub.

The related component:

The following diagram shows how builds are routed to the private pool.

Provision configuration¶

[1] Established the core element

Project: $DATATEAM_PROJECT_ID

Region: $DATATEAM_PROJECT_REGION

[2] Enable required API of service

gcloud services enable cloudbuild.googleapis.com

[3] Config the private pool with following attribute

The config will be provision at inno-infra. For more detail, go with google-private-pool-overview

[4] Config storage for CloudBuild to store log

Following: [CloudBuild Log]¹

For the life cycle of object, follow the table:

Retired condition:

For the permissions:

• Requested admin of bucket

• Read priviledges for DE team and service accounts interactive with Cloud Build logs

To check the retention of object in a targeted bucket with name of STORAGE_BUCKET_NAME

export STORAGE_BUCKET_NAME=
gcloud storage buckets describe gs://$STORAGE_BUCKET_NAME --format="default(per_object_retention)"
# null

Reference:

• Terraform module for Storage Bucket

• Google - Storage Life Cycle

[5] Config connection into the Github repository, which contains various private repositositories

For the list of repositories, targeted into repositories in organization at: Innotech-Vietnam

Table: Related repository linked with CloudBuild

The table has been updated at 2024-04-22. For the updated data, go to Console CloudBuild > Repositories

To handle the mapping, following process will take place:

1. Go with admin repository, add runner account: innodatarunner@innotech.vn

For password, go to Cloud Secret and read from RUNNER_EMAIL_INNODATA_RUNNER::latest

1. Share Admin priviledges to runner account

1. Go to CloudBuild > Repositories to establised connection between Github and CloudBuild

This will sync repositories between GitHub and Google Cloud

Pricing¶

The property has been defined:

Source: Official CloudBuild product

The table of pricing detail

Estimate on build

Service account¶

There are 2 component:

• Default CloudBuild Service Account: Cloudbuild Service account $PROJECT_NUMBER@cloudbuild.gserviceaccount.com.

If your Cloud Storage bucket and Cloud Build are in the same Google Cloud project and you're using the Cloud Build service account, your Cloud Build service account has the necessary IAM permissions by default. You do not have to grant any additional permissions.

For more detail, go with [CloudBuild Log]¹ and [CloudBuild Service Account]²

• CloudBuild Service Agent: service-$PROJECT_NUMBER@gcp-sa-cloudbuild.iam.gserviceaccount.com

Note: Get number of project at the [#troubleshooting]

Reference:

• Configure access for cloud build service account

Troubleshooting¶

[1] Get project number by gcloud command ¶

Use following command

gcloud projects list --filter="$(gcloud config get-value project)" --format="value(PROJECT_NUMBER)";
# 992228553443

[2] Send logs to both Cloud Storage (with streaming) and Cloud Logging¶

For the configuration file:

(a) Set logStreamingOption into STREAM_ON

(b) Dont set the logging key

---
options:
  # logging: null # Use this option to specify if you want to store logs in Cloud Logging or Cloud Storage.
  logStreamingOption: STREAM_ON

Reference:

• GCP - Build config file schema

• GCP - Build store and manage logs

• GCP - Build service account

Source Reference¶