Task: Update Security Metadata¶

Overview¶

This handle the metadata in the warehouse

Priority:

• Focus on issues that client meet first

• Focus on the range of ticker supported: (a) Index: VN30, HNX30 (b) Market: HOSE > HNX > UPCOM

Detail¶

Update dividend¶

Currently, the VSD take precense over the metadata comes from 3^rd party (like Vietstock)

Updated on Vietstock overview¶

Procedure to update the vietstock metadata element

For each element of metadata to update, the following will still trigger.

~> Start procedure

Step 1: [Data Operator] Go to Finance Vietstock https://finance.vietstock.vn/

Step 2: [Data Operator] Select ticker and get the targeted metadata

Step 3: [Data Operator] Updated in the table: vist_profile_snapshot

Example below is to update the the snapshot.

UPDATE <SCHEMA>.vist_profile_snapshot
SET
  dividend = NULL,
  updated_at = CURRENT_TIMESTAMP() /* Required for engine detect the change on databse */
  /* Currently not supported `updated_by` fielld but still required set the user has been action
  updated_by="@bao.truong"
  */
WHERE
  1=1
  AND `status` = "A"
  AND UPPER(ticker) = "CTG"

Loop procedure based on basket ticker.

Step 4: [Data Officer] Trigger the lake to refresh the new resources.

Note: This required internal run on the data team size

declare LOCATION=
declare WORKFLOW_NAME=prod-lake-prep-bound-inflow
gcloud workflows execute $WORKFLOW_NAME \
  --location $LOCATION \
  --call-log-level=log-all-calls \
  --data '{"command":["dbt run --selector galaxy"]}'

Reference:

• Execute a workflow

• gcloud workflows execute

Step 5: [Data Officer] Trigger the transfer on targeted service (with Madillo service)

Note: This required internal run on the data team size

declare LOCATION=
declare WORKFLOW_NAME=prod-madillo-executor
gcloud workflows execute $WORKFLOW_NAME \
  --location $LOCATION \
  --call-log-level=log-all-calls \
  --data '{"ranking":0,"replica_method":"REFRESH","source":{"discover":"TABLE:PROJECT.DATASET.pluto_symbol_profile"},"target":{"discover":"TABLE:DATASET.pluto_symbol_profile"}}'

Note: Please change the PROJECT.DATASET for source identifier and DATASET for target identifier

Step 6: [Data Operator] Validate the result in the endpoint

For Pluto service: go to dapi.innotech.vn

~> End procedure

For the list of element that mapping on target resources:

Example mapping: