Skip to content

Inno Docs

Strategy to design fast fetch from source

Initializing search

Innotech-Vietnam/inno-docs

Home
Collection
Project
Technical
Specification
Handbook
Member

Inno Docs

Innotech-Vietnam/inno-docs

Home
Home
- Introduction
Collection
Collection
- Introduction
- Dataset
  Dataset
  - Profile
    Profile
    
    Introduction
    
    Financial Instrument
    Financial Instrument
    
    Introduction
    
    Life Cycle of Financial Instrument
    
    Structure
    
    Types
    
    Types of Markets
    
    Operation
    Operation
    
    Index
    
    Profile: Change Trade Status of Security
    
    Equity
    Equity
    
    Profile: Collector
    
    Outstanding Share
    
    Index
    Index
    
    Index Metadata
    
    Real-time Snapshot World Index
    
    Coin
    Coin
    
    Coin Metadata
    
    Currency
    Currency
    
    Introduction
    
    ETF
    ETF
    
    Introduction
    
    Fund Manager
    
    Issuer
    
    Covered Warrant
    Covered Warrant
    
    Introduction
    
    Derivative
    Derivative
    
    Introduction
    
    Future Contract
    
    EPS
    
    The ERD model of VSD
    
    Vendors: VSD
  - Exchange - aka Market
    Exchange - aka Market
    
    Exchange: Reporter
    
    Trade Result
    
    Market State
  - Fundamental
    Fundamental
    
    Introduction
    
    SAD
    
    Period Fundamental Fiscal Variant
    
    Fundamental Statement Status
    
    Fundamental Business Category
    
    TODO
    
    Alternative dataset
    
    Troubleshooting
    
    Term
    
    Operation
    Operation
    
    Map dimension account chart
  - Quote
    Quote
    
    Introduction
    
    Operation
    
    Element
    Element
    
    Enumuration for quote to mapping from Exchange source
    
    Foreign Trade
    
    Historical index
    
    Flow adjustment
    Flow adjustment
    
    Introduction
    
    Adjustment Progress
    
    Coporate action
    
    Ops: Manual input
    
    Null Reference Price
    
    Troubleshooting
    
    Quality
    Quality
    
    Index
    
    Trade Flow
    Trade Flow
    
    Introduction
    
    Metric
    Metric
    
    Metric
    
    Technical Indicator
  - Metric
    Metric
    
    Security Performance
    
    Metric: Beta
  - News
    News
    
    Introduction
  - Reference
    Reference
    
    Core Dimension
    
    VAS
    
    VSIC
  - Economic
    Economic
    
    Economic Data
    
    World Trade
  - Analysis
    Analysis
    
    Introduction
    
    Create the target component
  - Signal
    Signal
    
    Trading Signals
    
    Signal Trade Process
- Model
  Model
  - Forecast
  - None
  - Exchange Listener
  - AI & Machine Learning
    AI & Machine Learning
    
    NER
- Operation
  Operation
  - Introduction
  - Tasks
    Tasks
    
    Task: Update Security Metadata
Project
Project
- Introduction
- TODO
- Changelog
- Master
  Master
  - [Draft] Proposal for Data Next-Generation
  - Tech Stack Landscape
- Project: Infra
  Project: Infra
  - Introduction
  - SAD
  - Changelog
  - TODO
  - Module
    Module
    
    Load Balancing for Data service
    
    IAP implement
    
    IAP context
  - Workload
    Workload
    
    Introduction
    
    Uptime Check
    
    Alignment datetime on specific sever
    
    Bump version of dependencies for services
    
    Config logging agent
    
    Infrastructure Handler
    
    Healthcheck service
    
    Reboot
    
    Resource Validate
    
    System healthcheck
    
    Threshold bigquery usage
    
    Update convention on editor
    
    Config the trigger for CloudBuild for all projects
    
    Change status on server
    
    Enchantress runner
- Project: Basement
  Project: Basement
  - Introduction
  - SAD
  - Changelog
  - TODO
  - Roadmap
  - Job Schedule
  - Orchestration
- Project: Submarine
  Project: Submarine
  - Introduction
  - SAD
  - Changelog
  - TODO
  - Register Namespace
  - Developer Guidance
    Developer Guidance
    
    Create endpoint in Submarine
- Project: Authentication
  Project: Authentication
  - Introduction
  - Changelog
  - TODO
  - Internal
    Internal
    
    Introduction
    
    SAD
    
    Changelog
    
    TODO
    
    Realm
    Realm
    
    Introduction
    
    Realm: Operation Portal
  - External
    External
    
    Introduction
    
    SAD
    
    Changelog
    
    TODO
  - User
    User
    
    Introduction
- Project: Lake Prep
  Project: Lake Prep
- Project: API Gateway
  Project: API Gateway
  - Introduction
  - SAD
  - TODO
- Project: MLOps
  Project: MLOps
  - Introduction
  - SAD
- Project: Docs
  Project: Docs
- Project: Pluto
  Project: Pluto
  - Introduction
  - SAD
  - Changelog
  - TODO
  - Spec: Endpoint
- Project: Processor
  Project: Processor
  - Introduction
  - SAD
  - Changelog
  - TODO
  - Spinosaurus
    Spinosaurus
    
    Introduction
    
    Pipeline: baryonyx
    
    Raptor
- Project: Notification
  Project: Notification
  - Introduction
  - SAD
  - Changelog
  - TODO
  - Troubleshooting
  - Idea
  - Agent
  - Convention
  - Workload
    Workload
    
    Introduction
    
    Notification: Build Status Change
    
    Notification: Pipeline Invoke
    
    Notification: Basement Error Report
    
    Notification: Interactive from Slack with slash commands
    
    Notification: Basement Server Reboot
  - Developer Guidance
    Developer Guidance
    
    Slack
    
    Create a Slack workflow webhook
    
    Declare slack application
    
    Flow for notification
- Project: Spectrum
  Project: Spectrum
- Project: Streaming
  Project: Streaming
  - Introduction
  - Changelog
  - TODO
  - Troubleshooting
  - Pipeline
    Pipeline
    
    Streaming from Data Layer
    
    Madillo - Syncronic
    
    Streaming from Data Layer
  - Workload
    Workload
    
    Change schema on model
- Project: Data Operation
  Project: Data Operation
  - Introduction
  - SAD
  - Changelog
  - TODO
  - Common
    Common
    
    Project Common
  - View
    View
    
    Introduction
    
    Datasets
    Datasets
    
    Introduction
    
    Dataset: Settlement
    
    Dataset: Quote
    
    Dataset: Corporate Action
    
    Dataset: Historical Outstanding Share
    
    Profile index
    
    Pipelines
    Pipelines
    
    Pipeline
    
    Pipe: Backdate for adjustment
    
    Pipe: Backdate for adjustment
    
    Pipe: Client register
    
    Pipe: Adjustment
    
    Pipe: Report oschestration job
    
    Pipe: Sync quote from provider
  - Tasks
    Tasks
    
    Tasks
  - Documentation
    Documentation
    
    Introduction
    
    Authentication Method
    
    OAuth Flow
    
    Audit Trail
    
    Handle error from backend
    
    Request - Appoval
    
    Diagram to handle the record component
- Project: Transflow
  Project: Transflow
  - Introduction
  - SAD
  - Changelog
  - TODO
  - Pipeline
    Pipeline
    
    SAD
- Project: Package
  Project: Package
  - Introduction
  - inno-finance-core
    inno-finance-core
    
    Introduction
    
    Changelog
    
    TODO
  - inno-python-client-submarine
    inno-python-client-submarine
    
    Introduction
    
    SAD
    
    Changelog
    
    TODO
  - inno-hybridate
    inno-hybridate
    
    Introduction
    
    Changelog
    
    TODO
  - inno-str-tune
    inno-str-tune
    
    Introduction
    
    Changelog
    
    TODO
  - inno-util
    inno-util
    
    Introduction
    
    Changelog
    
    TODO
Technical
Technical
- Introduction
- Services
  Services
  - Introduction
  - Register: Services
  - CloudBuild
    CloudBuild
    
    Introduction
  - GitHub Actions
    GitHub Actions
    
    Service: GitHub Actions
    
    Operation
    Operation
    
    Action: actions/setup-python
    
    Action: actions/slack-github-action
    
    Troubleshooting
  - IAM
    IAM
    
    Introduction
    
    Disable default service account
    
    Authentication into service from service account
  - Dataproc
    Dataproc
    
    Dataproc
- Archtechture Decision Record
  Archtechture Decision Record
- Concept
  Concept
  - Infrastructure
    Infrastructure
    
    Google Cloud SQL
    
    Deployment
    
    Infrastructure Permission
  - API
    API
    
    Introduction
    
    Conception about API
    
    Concept to handle exception
    
    The detail errors
    
    API: Rate Limit
    
    Head
    
    API Idea
  - Data Ingestion
    Data Ingestion
    
    SCD - Slowly Dimension
  - Data Management
    Data Management
    
    Introduction
    
    Data Catalog
    
    Introduction
    
    Introduction about metadata of dataset
    
    Profile
    
    Provider
    
    Data Validation
    
    Incident
    
    Track Change Data Method
    
    Application Testing
  - Research
    Research
    
    Platform Research
  - Versioning
- CICD
  CICD
  - Introduction
  - Component
    Component
    
    CICD
- Operation
  Operation
  - Index
  - Operation: Action when terminate members
  - Database
    Database
    
    DBA Tasks for Dawnbreaker
    
    DBA Tasks for Silencer
    
    Snippset: Get the
    
    Migration table within the database (MySQL)
  - 2fa
  - Disclaimer for User
  - Tracking
  - Strategy to design fast fetch from source Strategy to design fast fetch from source
    Table of contents
    
    Overview
  - SCD
Specification
Specification
- Introduction
- Internal
  Internal
  - Introduction
  - Errors
    Errors
    
    Introduction
    
    APPLICATION
    
    BAD_REQUEST
    
    CANNOT_PARSE_QUERY
    
    CAN_NOT_REFERENCE_FOREIGN_KEY
    
    COLUMN_NOT_EXIST
    
    DATABASE_DESIGN
    
    DATABASE_DISCONNECT
    
    DATA_TOO_LONG
    
    DUPLICATED_RECORD
    
    INVALID_PASSING_ARGUMENT
    
    INVALID_PERMISSION
    
    METHOD_NOT_ALLOWED
    
    NOT_AUTHENTICATE
    
    REQUEST_VALIDATION_ON_COOKIE
    
    REQUEST_VALIDATION_ON_HEADER
    
    DATABASE_DISCONNECT
    
    REQUEST_VALIDATION_ON_QUERY
    
    REQUEST_VALIDATION_ON_RESPONSE_MODEL
    
    RESOURCE_NOT_FOUND
    
    SEE_OTHER
    
    TABLE_NOT_EXIST
    
    UNKNOWN_DATABASE
- Authentication
  Authentication
  - Introduction
  - Errors
    Errors
    
    Introduction
    
    AUTH_AUTHENTICATION_ERROR
    
    AUTH_OPERATION_ERROR
    
    AUTH_RESOURCE_NOT_FOUND
    
    AUTH_SCOPE_NOT_FOUND
    
    AUTH_SERVICE_NOT_FOUND
    
    APPLICATION
    
    BAD_REQUEST
    
    INVALID_PERMISSION
    
    METHOD_NOT_ALLOWED
    
    REQUEST_VALIDATION_ON_COOKIE
    
    REQUEST_VALIDATION_ON_HEADER
    
    DATABASE_DISCONNECT
    
    REQUEST_VALIDATION_ON_QUERY
    
    REQUEST_VALIDATION_ON_RESPONSE_MODEL
    
    RESOURCE_NOT_FOUND
    
    SEE_OTHER
- Spectrum
  Spectrum
  - Introduction
  - Coverage of Dataset
  - Specification of API Endpoint
  - Parameters
  - Response
    Response
    
    Specification: Response Output
    
    Response: Error
    
    Filling
  - Guidance
    Guidance
    
    Developer Guidance
    
    Authentication
    
    Errors
- Data Operation
  Data Operation
  - Introduction
  - Quick Tour
  - Operation
  - Term
  - Release
Handbook
Handbook
- Introduction
- Headstart
  Headstart
  - Next Generation for the Data-*
  - Shell Introduction
  - Documentaion
  - Git Strategy
  - Google
    Google
    
    Index
    
    Google Authentication
    
    Headstart with Google Cloud Platform
    
    Cloud Run
  - Python
    Python
    
    Python Interpreter Setting
    
    Troubleshooting for common error
  - Development Concept
    Development Concept
    
    Prerequisite
    
    Deployment Process
    
    Submitting a Pull Request
  - Model to deploy machine learning model
  - Service Testing
Member
Member

Table of contents

Overview

Strategy to design fast fetch from source¶

Overview¶

Idea of selector list and ignore rules

Copyright © Innotech 2023 - 2024

Made with Material for MkDocs

Cookie consent

We use cookies to recognize your repeated visits and preferences, as well as to measure the effectiveness of our documentation and whether users find what they're searching for. With your consent, you're helping us to make our documentation better.

GitHub

Manage settings