Managing routers¶
Overview¶
A router is the component that exposes an app to its users: it receives the app's address, cnames
and certificates from tsuru and configures load balancers, ingresses or gateways to send traffic to
the app units. Tsuru itself does not route traffic. It talks to a router through the
Router API: every router has type: api and an api-url of a service
implementing that contract, such as kubernetes-router.
The old in-process router implementations (Router V1) were removed (since tsuru 1.20.6); api is the only
registered type, and tsuru router add rejects any other with router type "<type>" is not
registered.
Routers are declared in one of two ways, which coexist and are listed together by
tsuru router list:
- Static routers in
tsuru.conf, underrouters:<name>. - Dynamic routers stored in the database and managed with
tsuru router add/update/remove.
Static routers in tsuru.conf¶
routers:
ingress-router:
type: api
api-url: http://kubernetes-router.tsuru-system:8077/api/ingress-nginx
default: true
multi-cluster: true
debug: false
headers:
X-Custom-Header: my-value
readinessGates:
- tsuru.io/router-ready
| Key | Description |
|---|---|
type |
Required. api. |
api-url |
Required for api. Base URL of the router API. |
default |
Marks this router as the default one when more than one router exists. |
multi-cluster |
Send the pool and cluster headers (X-Tsuru-Pool-Name, X-Tsuru-Pool-Provisioner, X-Tsuru-Cluster-Name, X-Tsuru-Cluster-Provisioner, X-Tsuru-Cluster-Addresses and, with the cluster custom data propagate-kubeconfig=true, X-Tsuru-Cluster-Kube-Config) on every request, so one router instance can serve apps in several clusters. |
debug |
Log every request and response sent to the router. |
headers |
Extra headers (string or list of strings) added to every request. |
readinessGates |
Pod readiness gates added to the web pods of apps using this router, see Readiness gates. |
Each key is also documented in the configuration reference.
Changing a static router requires editing the file and restarting tsurud.
Dynamic routers¶
Dynamic routers take the same configuration as a JSON object and need no restart. They require the
router.create, router.update and router.delete permissions.
$ tsuru router add ingress-router api \
--config '{"api-url": "http://kubernetes-router.tsuru-system:8077/api/ingress-nginx", "multi-cluster": true}' \
--readiness-gate tsuru.io/router-ready
$ tsuru router update ingress-router api --config '{"debug": true}'
$ tsuru router info ingress-router
$ tsuru router remove ingress-router
resource "tsuru_router" "ingress-router" {
name = "ingress-router"
type = "api"
readiness_gates = ["tsuru.io/router-ready"]
config = <<-EOT
api-url: "http://kubernetes-router.tsuru-system:8077/api/ingress-nginx"
multi-cluster: true
EOT
}
tsuru router update merges --config into the stored
configuration key by key; a key set to null is deleted. Rules:
- A dynamic router cannot use the name of a router declared in
tsuru.conf:router named "<name>" already exists in config. Adding a name that is already dynamic answers409 dynamic router already exists. - There is no
defaultfield for dynamic routers. The default router is chosen as described in Default router and routers per pool. tsuru router removeonly removes the definition. Apps still pointing at it fail to deploy (router "<name>" not found), so remove it from every app first withtsuru app router remove.
tsuru router list shows dynamic routers as api (dynamic) and, for every router, the Info the
router API reports; tsuru router info <name> also prints the readiness gates and config. Users
without router.create do not see the config, and users without a global app.create only see the
routers allowed in the pools of their teams.
Readiness gates¶
A readiness gate is a Kubernetes pod condition that must be True before the pod counts as ready.
When a router declares readiness gates (readinessGates in tsuru.conf or --readiness-gate), tsuru
adds them to the web process pods of every app using that router (other processes are not
affected). The router implementation is expected to set the condition once the pod is registered in
the load balancer, so a rollout only proceeds when new units are actually reachable.
kubernetes-router¶
kubernetes-router is the reference implementation of
the Router API. It runs as a pod in the cluster, listens on :8077 and serves the API under
/api and /api/<mode>, where <mode> selects one of the controller modes enabled with
-controller-modes (the first enabled mode is used for plain /api):
| Mode | Resources created per app |
|---|---|
service (or loadbalancer) |
A Service of type LoadBalancer named <app>-router-lb. Router opts can become Service labels through -opts-to-label, and -pool-labels adds labels per pool (POOL={"LABEL":"VALUE"}). |
ingress |
An Ingress per app (and one per cname) with host <app>.<-ingress-domain> (-ingress-domain, default local, takes precedence over the domain-suffix opt; the domain opt replaces the host entirely). Router opts listed in -opts-to-ingress-annotations key=annotation become Ingress annotations; -ingress-class sets the class (-use-ingress-class-name writes spec.ingressClassName instead of the annotation) and -ingress-annotations-prefix prefixes option names that are not mapped. |
ingress-nginx |
Same as ingress with -ingress-class nginx and -ingress-annotations-prefix nginx.ingress.kubernetes.io preset. |
istio-gateway |
An Istio Gateway and VirtualService, selecting the gateway pods with -istio-gateway.gateway-selector. |
gateway-api |
Gateway API HTTPRoutes attached to the Gateway named by -gateway-name/-gateway-namespace; -acme-issuer is the cert-manager ClusterIssuer used when tls-acme=true. |
Every mode points its resource at the Service tsuru creates for the app's web process: tsuru sends
the Service name and namespace of each routable process version with the backend, so the router
never guesses names. Other useful flags: -k8s-namespace (default tsuru) for the router's own
resources, -k8s-labels/-k8s-annotations added to everything it creates, -ingress-http-port and
-ingress-address-scheme for the address reported back to tsuru, -cert-file/-key-file to serve
HTTPS, and -clusters-file to enable multi-cluster mode (a YAML clusters: list with name,
address, token/ca or authProvider/exec; the cluster is picked by the X-Tsuru-Cluster-Name
header, or built from X-Tsuru-Cluster-Kube-Config when present). Requests are authenticated with
HTTP basic auth when ROUTER_API_USER and ROUTER_API_PASSWORD are set; pass them to tsuru through
the router headers (Authorization: Basic ...).
Besides the opts you map yourself, kubernetes-router understands exposed-port, domain,
domain-suffix, domain-prefix, route, tls-acme, tls-acme-cname, http-only,
external-traffic-policy and all-prefixes. Tsuru always adds tsuru.io/app-pool,
tsuru.io/app-teamowner and tsuru.io/app-teams to the opts it sends.
Routers of an app¶
An app can have several routers. The first one is set at creation (tsuru app create --router,
--router-opts key=value, or --router none for an app without any router); the others are
managed with tsuru app router:
$ tsuru app router add ingress-router -a myapp -o tls-acme=true -o domain-suffix=example.com
$ tsuru app router update ingress-router -a myapp -o tls-acme=false
$ tsuru app router list -a myapp
$ tsuru app router remove ingress-router -a myapp
resource "tsuru_app_router" "ingress" {
app = tsuru_app.myapp.name
name = "ingress-router"
options = {
"tls-acme" = "true"
"domain-suffix" = "example.com"
}
}
-o/--optsvalues are passed verbatim to the router implementation;app router updatereplaces the whole set of opts of that router.- The router must be allowed in the app's pool (
router "<name>" is not available for pool "<pool>". Available routers are: ...) and cannot be added twice (router already linked to this app). Adding a router whose cname is already registered by another app in the same router is refused. - The permissions are
app.update.router.add,app.update.router.updateandapp.update.router.remove. tsuru app updatedoes not change routers anymore: the API answersupdating router was deprecated, please add the wanted router and remove the old one. Theapp swapAPI is also deprecated (swapping is deprecated) and has no client command.
tsuru app router list and tsuru app info show one row per router with its opts, the addresses
reported by the router and the backend Status the router returns (for example ready, or
not ready with a detail), so a misconfigured ingress shows up there before users notice.
Default router and routers per pool¶
When an app is created without --router, tsuru picks:
- the first value listed in the
routerpool constraint of the app's pool, when the constraint is an allowlist without globs; - with a glob or denylist constraint, the only allowed router if exactly one remains;
- otherwise the router flagged
default: trueintsuru.conf, or the only router when just one exists. If none matches, creation fails withNo default router found.
The same router constraint also restricts which routers tsuru app router add accepts in that
pool:
$ tsuru pool constraint set prod router ingress-router
$ tsuru pool constraint set "dev_*" router internal-router --denylist
Rebuilding routes¶
Tsuru resends the whole backend (addresses of every routable version, cnames, certificate issuers, team, tags, healthcheck and opts) to each router of the app after deploys, unit changes and app or router updates. If a router lost state, for example after its resources were deleted by hand, force it with:
$ tsuru app routes rebuild -a myapp
It requires the app.admin.routes permission and reports one Updating router [<name>] section per
router.