Warning

🚧 Work in Progress: This page is currently under construction. Content may be incomplete or subject to change. To contribute, see the contribution guide.

Data Flows

Diagrams and descriptions of the main data flows between systems.

---

General data architecture

flowchart LR
    subgraph Sources
        S1[Operational systems]
        S2[External APIs]
        S3[Files / SharePoint]
    end

    subgraph Ingestion["Ingestion (GCP)"]
        CR[Cloud Run]
        N8N[N8N]
        AF[Airflow DAGs]
    end

    subgraph DataLake["Data Lake (BigQuery)"]
        RAW[Raw layer]
        STAGE[Stage layer]
        GOLD[Gold layer]
    end

    subgraph Consumption
        BI[BI / Dashboards]
        API2[Internal APIs]
        ML[AI Models]
    end

    Sources --> Ingestion
    Ingestion --> RAW
    RAW --> STAGE
    STAGE --> GOLD
    GOLD --> Consumption

---

Critical flows

Flow	Source	Destination	Frequency	Owner
(fill in)	(fill in)	BigQuery	(fill in)	Data Squad

---

Info

Individual flow details: Individual flows are documented in the Data Catalog and Airflow Pipelines.