Lab examples

Generated by the documentation agent from the lab's Git repositories and regenerated by hand with aks-terraform's Lab examples workflow - never edit this page by hand. An example of the lab, not the design: the design does not link here and neither PDF holds it.

Internal - Finland

Generated by the documentation agent of onek8s/aks-ai-agents (aks-agent docs lab) from Git alone: what Git deploys into the isolation zone int-fi of the spoke dev-int. Do not edit by hand - aks-terraform’s Lab examples workflow regenerates it.

Back to the lab examples.

   
Isolation zone Internal - Finland (int-fi)
Cells aks-dev-int-stateless-az3 (germanywestcentral, availability zone 3)
aks-dev-int-stateful (germanywestcentral) - defined, not built
Domain int-fi.dev.onek8s.lol
Gateway int-fi-gateway/int-fi (traefik), listener HTTPS 443 *.int-fi.dev.onek8s.lol
Published names hello-aks-dev-int-stateless-az3.int-fi.dev.onek8s.lol
hello.int-fi.dev.onek8s.lol
Zone allow-list login.microsoftonline.com (fleet)
Applications aks-hello in int-fi-example

Placement

The zone’s nodes are classic node pools of aks-terraform (zone_node_pools), each scaled by the cluster autoscaler from zero, labelled and tainted platform/zone; an application states its architecture and capacity with pod labels only.

Cell Pool Architecture Capacity VM size Nodes OS disk Pods per node
aks-dev-int-stateless-az3 intfi amd64 on-demand Standard_D2alds_v6 0-1 Ephemeral 250
aks-dev-int-stateless-az3 intfispot amd64 spot Standard_D2alds_v6 0-1 Ephemeral 250
aks-dev-int-stateless-az3 intfiarm arm64 on-demand Standard_D2pds_v6 0-1 Ephemeral 250
aks-dev-int-stateless-az3 intfiarmspot arm64 spot Standard_D2pds_v6 0-1 Ephemeral 250

Traffic of the zone

What the hub firewall opens for the zone: every destination below is open to every application of the zone.

flowchart LR
    subgraph Z["Isolation zone int-fi"]
        direction TB
        a_int_fi_example["int-fi-example<br/>team example"]
    end
    fw["Hub firewall<br/>source: IP group of zone int-fi"]
    subgraph OUT["What the firewall opens for the zone"]
        direction TB
        d_api_ipify_org["api.ipify.org:443<br/>asked by int-fi-example"]
        d_api_open_meteo_com["api.open-meteo.com:443<br/>asked by int-fi-example"]
        z_login_microsoftonline_com["login.microsoftonline.com:443<br/>zone allow-list"]
        p___grafana_net["*.grafana.net:443<br/>platform rule, every cell"]
        p_stats_grafana_org["stats.grafana.org:443<br/>platform rule, every cell"]
    end
    a_int_fi_example --> fw
    fw -->|"allowed"| d_api_ipify_org
    fw -->|"allowed"| d_api_open_meteo_com
    fw -->|"zone allow-list"| z_login_microsoftonline_com
    fw -->|"platform rule"| p___grafana_net
    fw -->|"platform rule"| p_stats_grafana_org
    none["Named by applications, no rule yet:<br/>hello.int-se.dev.onek8s.lol"]
    fw -.-x|"denied"| none
    classDef ns fill:#e3f2fd,stroke:#1565c0,color:#000
    classDef platform fill:#eceff1,stroke:#455a64,color:#000
    classDef fw fill:#ede7f6,stroke:#5e35b1,color:#000
    classDef allowed fill:#e8f5e9,stroke:#2e7d32,color:#000
    classDef unexplained fill:#fff8e1,stroke:#ef6c00,color:#000
    classDef missing fill:#ffebee,stroke:#c62828,color:#000,stroke-dasharray: 5 5
    classDef proposed fill:#f3e5f5,stroke:#7b1fa2,color:#000,stroke-dasharray: 5 5
    classDef zone fill:#e0f2f1,stroke:#00695c,color:#000
    class fw fw
    class a_int_fi_example ns
    class d_api_ipify_org,d_api_open_meteo_com allowed
    class z_login_microsoftonline_com zone
    class p___grafana_net,p_stats_grafana_org platform
    class none missing

aks-hello in int-fi-example

From onek8s/aks-hello’s README: A .NET hello world website - the example application of OneK8s, one Kubernetes platform for every cloud. It is deployed by Flux to the example team namespace of the clusters built by aks-terraform and published through the platform’s Gateway (Traefik Proxy, Kubernetes Gateway API). The page shows which cluster answered: https://hello.onek8s.lol is served by every stateless cluster at once, and DNS hands out one or the other.

   
Configuration onek8s/aks-fluxcd-example / firewall-egress-int-fi
Team, contact example, https://github.com/onek8s/aks-fluxcd-example/issues
Deployment aks-hello, replicas 2
Image acronek8sdev.azurecr.io/aks-hello, version 0.8.0, digest sha256:201cac9a95b8286e7861c7442ce11f09c495cee2802c8ae24abce07f6787cc1e
Source onek8s/aks-hello, version 0.8.0
Placement labels platform/arch: amd64, platform/capacity: spot - the zone’s pools intfispot, intfi
Firewall rule file int-fi-example.yaml
Traffic flows int-fi-example

Services and routes

Service Port Target port Protocol
aks-hello 80 h2c kubernetes.io/h2c
Route Hostnames Paths Backends Gateway platform/home-cell
aks-hello hello-aks-dev-int-stateless-az3.int-fi.dev.onek8s.lol / aks-hello:80 int-fi-gateway/int-fi aks-dev-int-stateless-az3
aks-hello-home hello.int-fi.dev.onek8s.lol / aks-hello:80 int-fi-gateway/int-fi aks-dev-int-stateless-az3

Environment

Variable Value
CLUSTER_NAME aks-dev-int-stateless-az3
DOTNET_EnableDiagnostics 0
NODE_NAME (from spec.nodeName)
PEERS int-se (Service)=http://aks-hello.int-se-example.svc.cluster.local/status
int-se (published)=https://hello.int-se.dev.onek8s.lol/status
WEATHER_PLACE Helsinki
WEATHER_URL https://api.open-meteo.com/v1/forecast?latitude=60.17&longitude=24.94&daily=temperature_2m_max,temperature_2m_min,precipitation_sum&timezone=auto&forecast_days=3

Endpoints of aks-hello

Path What
/ The page: cluster, pod, version, the node’s public IP, the weather forecast, the peers
/status The same as JSON, plus node (NODE_NAME), architecture, the HTTP protocol the caller spoke (protocol, 0.5.1), start time, uptime and readiness
/weather The forecast as JSON, or why there is none
/public-ip The node’s public IP found when the pod started, or why none was
/peers The peers (PEERS, 0.8.0): each one’s URL, the last poll’s answer or error, and the counts since the pod started
/health/live Liveness: 200 while the process answers HTTP; checks nothing else
/health/ready Readiness: 200 from the end of startup until SIGTERM, then 503
/healthz All checks (the probe path before 0.3.0)

Egress

Destination Port Status Named in the configuration Reason of the rule
api.ipify.org 443/https allowed - the application’s rule, named in its configuration mentioned in firewall-egress/kustomization.yaml aks-hello needs to query api.ipify.org for its public IP, as a new egress destination not yet in the configuration firewall-egress-int-fi/kustomization.yaml
api.open-meteo.com 443/https allowed - the application’s rule, named in its configuration Deployment aks-hello, WEATHER_URL
mentioned in firewall-egress-int-fi/kustomization.yaml
mentioned in firewall-egress/kustomization.yaml
aks-hello fetches a weather forecast from the internet (api.open-meteo.com) as configured in firewall-egress-int-fi/kustomization.yaml (Helsinki)
hello.int-se.dev.onek8s.lol 443/https proposed - in an open pull request, not allowed yet Deployment aks-hello, PEERS -

Notes:

  • Other cells sync another path of the same repository: aks-dev-int-stateful: apps.
flowchart LR
    client["Corporate network and VPN clients"]
    subgraph CELL["Cell aks-dev-int-stateless-az3 - pod subnet + hub firewall: the firewall sees the pod's own address"]
        direction LR
        gw["Gateway int-fi-gateway/int-fi<br/>hello-aks-dev-int-stateless-az3.int-fi.dev.onek8s.lol<br/>hello.int-fi.dev.onek8s.lol"]
        subgraph NS["namespace int-fi-example - team example"]
            direction TB
            svc_aks_hello["Service aks-hello"]
            wl_aks_hello["Deployment aks-hello x2<br/>acronek8sdev.azurecr.io/aks-hello:0.8.0@sha256:201cac9a95b8286e7861c7442ce11f09c495cee2802c8ae24abce07f6787cc1e"]
        end
        dns["CoreDNS"]
        eso["ESO int-fi - Key Vault share int-fi-example--*"]
    end
    fw["Hub firewall<br/>rule collection group dev-int-applications"]
    subgraph OUT["Outside the landing zone"]
        direction TB
        d_api_ipify_org["api.ipify.org:443"]
        d_api_open_meteo_com["api.open-meteo.com:443"]
        d_hello_int_se_dev_onek8s_lol["hello.int-se.dev.onek8s.lol:443"]
    end
    client --> gw
    gw --> svc_aks_hello
    svc_aks_hello --> wl_aks_hello
    wl_aks_hello --> dns
    eso -.->|"Secrets"| wl_aks_hello
    wl_aks_hello -->|"every flow leaving the cluster"| fw
    fw -->|"allowed"| d_api_ipify_org
    fw -->|"allowed"| d_api_open_meteo_com
    fw -.->|"proposed"| d_hello_int_se_dev_onek8s_lol
    classDef ns fill:#e3f2fd,stroke:#1565c0,color:#000
    classDef platform fill:#eceff1,stroke:#455a64,color:#000
    classDef fw fill:#ede7f6,stroke:#5e35b1,color:#000
    classDef allowed fill:#e8f5e9,stroke:#2e7d32,color:#000
    classDef unexplained fill:#fff8e1,stroke:#ef6c00,color:#000
    classDef missing fill:#ffebee,stroke:#c62828,color:#000,stroke-dasharray: 5 5
    classDef proposed fill:#f3e5f5,stroke:#7b1fa2,color:#000,stroke-dasharray: 5 5
    classDef zone fill:#e0f2f1,stroke:#00695c,color:#000
    class gw,dns,eso platform
    class fw fw
    class svc_aks_hello,wl_aks_hello ns
    class d_api_ipify_org,d_api_open_meteo_com allowed
    class d_hello_int_se_dev_onek8s_lol proposed

Legend

The status of a destination, from the traffic flow documentation:

  • allowed - the application’s rule, named in its configuration
  • unexplained - the application’s rule, but its configuration does not name it
  • missing - the configuration names it, the firewall denies it
  • proposed - in an open pull request, not allowed yet
  • zone - open to every application of the zone
  • platform - open from every cell of the exposure

OneK8s – a design under review; country codes and IP ranges are examples.Source of this pageComments and issues