Skip to content

Flow for operation

Overview

Operation for dataset:quote

Diagram

The following diagrams is the process to handle the

Diagram for operation of quote

The timeline for each section

RP (*1) Timeline Task Description Related pipelines
T 15-16 Snap last trading date Snapshot market at the trading date Calculated by IFC
15-18h30 Get quote from source Daily snapshot for security flow > inno > daily-security
Daily snapshot for index flow > inno > daily-index
Daily snapshot for future flow > hnx > daily-future
flow > inno > sync-from-hnx
T 17,20,23 Flow adjustment Adjust quote for basket of ticker flow > inno > adjustment
Report flow > inno > adjustment
T+1 4, 8h-30 Sync of B2B client Sync dataset from client Control by different team
T+1 8h55 [Report] Pre-market checklist Report for pre-market checklist flow > inno > report-pre-market
T+1 9h20 [Report] After-open-market checklist Snapshot market at the trading date flow > inno > report-after-open-market

Note:

  • Timeline based on 0-24H calculation

  • RP - meaning related period of T (which mean last trading date)

For the requirements of the operation:

  • Related quote of security has been adjusted.

  • The related team for this process included: (a) Team Data (b) Operation team that handle clients process

For detail section

Step 1: Snap last trading date

Last Trading date will be calculated to mark the last trading of the market

This based on: dataset::inno:settlement which contain the non-trading date by the notice of the market.

Control by IFC Internal package: inno-finance-core

Step 2: Get quote from provider

Get sync data from multiple provider and sync into dataset:inno:quote

Step 3: Confirmation for execution

If the quote has been updated to last-trading-date in place. Then the flow of adjustment will be executed.

  • Step 3a: Execute the execution for the dataset.

  • Step 3b: Execute lake transform to handle related pipelines for quote (calculated min-max, calculated market cap, ...).

The pipeline can have tags of tags:flow-adjustment, tags:quote (Currently force run galaxy)

  • Step 3c: Execute the pipeline to transfer data from lake into consume area (through Madillo service)

  • Step 3d: Report for the execution detail.

It's will included: (a) Metadata of the adjustment actionable items

  • Step 3e: Handle corner case after the process

Currently the process has been still have the corner case

NOT_EXIST_ANY_TRADE_DATA → Still success

MULTIPLE_ACTIONS_RELATED_TO_RATE → Easy to failure

For adjustment process, go through Internal documentation for flow adjustment process

Step 4: Report for pre-market checklist

This report contain the metadata for pre-market checklist and support the operator can confirmation

Example of report for pre-market checklist

To get the report automaticly related to data operation, please send email to bao.truong@innotech.vn and then you will be added to group of innodata@innotech.vn.

The mail will be send by the runner of data team innodatarunner@innotech.vn

Step 5: [Optional] The maker-checker process

(a) Register access/work-on-modify/work-on-approval of datasets through internal data portal

(b) Get metadata of the execution and handle operation process for datasets

Step 6: [Optional] Report for after-open-market within the next trading date.

Validate with source of exchange (in case of exists) and notify for the users related to manual update throguh portal.