Skip to content

Changelog

2024-07-08

Bump version of project into v2.8.14 with backend of (2.5.1) and engine of (2.7.31)

Dataset:

  • [VSD] Updated the VSD element

  • [Vietstock] Fixed issue divide 1000 for timeset

  • Added handle for VSD component

  • Removed basement_csh_ohlcv_eod_index

  • Removed tables in basement layer: basement_ssi_profiles, basement_cafef_eod

  • Transfer the element of quote (security and idnex)

Internal:

  • Updated README with badges for project

  • Bump project dependencies:

  • ruff into min v0.4.8

  • pip into 24.1.1

  • Updated the maintainer in Dockerfile tags

  • [CICD] Handle the process to install ruff only

  • Added internal docs

  • Added target of: venv, install-dev, install-docs, docs

Engine:

  • Added configuration for dbt-checkpoint

Backend:

  • Updated flow

  • Updated target for: venv, install, install-profile

2024-07-02

Bump version of backend into v2.6.12

  • Updated element with quote from HSC into INNO

  • Handle the flow suite of quote for future

2024-06-03:

  • Updated documentation for working with propject

  • Release process

  • Prerequisite

  • Developer process

  • Installment

2024-06-02:

  • Updated namespace for the Run service with workflow

  • Triggers the Run build with the changes on folder application

  • Bump supported of dbt-parser into version 0.6.0

Documentation: PyPi dbt-artifacts-parser

Version of 0.6.0 supported pydantic v1 and dbt from v1.5 -> v.1.7

Version mapping is V11 based on Manifest JSON

  • Fixed required version in dbt configuration

  • Updated README

  • Align with dbt version 1.7 with changelog of dbt

  • Updated timeout into 900 (~15 minutes) instead of 600 (~10 minutes)

  • Fix docker-compose.yaml: version is obsolete.

  • [CICD] Handle the Dockefile with:

  • Updated labels of Dockerfile with vn.innotech.data.transform.internal reversed

  • Align version with dbt-project

  • Removed config for the script has been handle by CloudBuild

  • Validate the folder on .dockerignore and .gcloudignore in the top folder

2024-05-26:

2024-MM-DD:

  • Updated Cloudbuild layout component with advanced configurations

  • Disable end_of_line = lf

  • Rename SERVICE_ACCOUNT_KEYFILE with current service account

  • Add raw instruction included in command

  • Config path on the BUILD argument for application https://stackoverflow.com/questions/7126580/expand-a-possible-relative-path-in-bash

  • Check all trigger element

  • Different place for execute REL_ARG

  • Add scheduler for testing service

  • Add fail-fast on the trigger by push to branch (for master and production)

  • Config the secret mount for REL_ARG

  • RUN dbt deps --project-dir ./application/ --profiles-dir ./application/.dbt/;

  • C-parser for LibYAML

  • Check workflow element

  • Table transfer

  • Config Zone in Workflow

  • Review all the layout of Cloudbuild

  • Review all config element

  • Fix layout of Dockerfile

  • Seperated the requirement.txt with related build COPY in the Dockerfile

  • Fix space after the bash RUN apt-get update &&\ and put -y for confirm the update process

  • Config some Python environment

    ENV PYTHONIOENCODING=utf-8
ENV LANG=C.UTF-8

  • Add the LABEL for Dockerfile with maintainer

  • Mount the different context of backend to server/ and application seperated with different mount

Make sure this is handled

make console && dbt --version && dbt debug && pytest;

2024-MM-DD:

  • Schedule:

  • Deploy schedule with hash change on the frequency basic

  • CICD to build the schedule for:

a) 0 15 * * * to build the timelapse model (Asia/Ho_Chi_Minh)

b) Each 15 minutes for the provider model (Asia/Ho_Chi_Minh) start at 0:01:00 AM

  • CICD coding-standard change name to code-compliance and change dir into backend

  • Config the name for all related to new repo with the GH Secret for Slack Token

  • Add component to validate Python version in the backend

  • Check the secret name and value with the target syntax

  • Change this BQ_PREFIX_DATASET to BIGQUERY_DATASET_PREFIX_NAME

  • Check deployment environment align with class::Environment

  • Change bigquery-morphling into inno-lake-prep

  • Config resource for production level

  • @bao Refactor target clean in Makefile

2024-MM-DD:

  • Build trigger for build the the lake-prep repository

  • CICD:

  • After deploy to production. Required to CICD to refresh run.

  • CICD for artifacts https://hub.getdbt.com/brooklyn-data/dbt_artifacts/latest/

  • Notification on build for developers and maintainer

  • Updated concurrency to 1 for 2 instances, reduced memory

  • Implement server using FastAPI instead of Go

  • Returned the schema manifest into system

  • Enforce precommit with ruff

2024-01-01

  • Updated documentation for headstart

2023-09-12

  • CICD:

  • After deploy to production. Required to CICD to refresh run.

  • Notification on build for developers and maintainer

  • Updated concurrency to 1 instead of 2 instances, reduced memory

  • Implement server using FastAPI instead of Go

  • ⚓ Bump version of dbt-core to v1.6.0

2023-08-11

  • Updated documenation related to lake-prep

  • Add VSIC model for summary, search

2023-08-07

  • Added ruff and pre-commit enforce

2023-02-15

  • [Maintaince] Repository layout structure