Kubernetes troubleshooting

Did requests reach my Kubernetes service?

Published September 24, 2026 · Fortem engineering

A visible Ingress → Service → ready endpoint route tells you where Kubernetes is configured to send matching traffic. It does not count requests or prove that a workload processed them. For observed HTTP counts and errors, Fortem needs an existing Prometheus source that already scrapes a supported controller. Application-side metrics or traces are a separate proof of what the application received.

Three different observations

Route exists

Kubernetes Ingress, Service and EndpointSlice objects show the declared host/path → backend relationship and current endpoint readiness. They do not expose HTTP request counters.

Controller saw traffic

A compatible Prometheus counter estimates requests observed at the ingress/controller during a named window. Fortem shows a five-minute count, RPS, separate 4xx and 5xx percentages, and p95 when its histogram exists.

Application handled it

Only application-side telemetry or a trace can establish receipt and processing beyond the ingress measurement point. A ready endpoint and an ingress counter are not that evidence.

Check an existing metrics source

First identify an already available Prometheus-compatible HTTP API. If it is private to the cluster, use an approved local port-forward or another authorized endpoint. Fortem does not install Prometheus. Its --prometheus-url takes the API's base URL; Fortem appends /api/v1/query. Do not pass a controller's /metrics exporter. This example assumes that API is reachable at 127.0.0.1:9091 and that the controller is ingress-nginx.

Check whether Prometheus has a request series for the namespace:

curl -fsSG 'http://127.0.0.1:9091/api/v1/query' \
  --data-urlencode 'query=count(nginx_ingress_controller_requests{namespace="YOUR_NAMESPACE"})'

In the JSON response, a non-empty data.result means a matching series exists. It does not prove that requests arrived in the last five minutes. An empty result can mean the metric, namespace label, scrape, or selected controller profile does not match; it is not evidence of zero traffic.

Then let Fortem test its actual five-minute queries without starting the UI:

fortem --doctor --context YOUR_CONTEXT --namespaces YOUR_NAMESPACE \
  --prometheus-url http://127.0.0.1:9091 \
  --traffic-profile ingress-nginx

Look for the Prometheus line: it is labeled Prometheus HTTP traffic (optional) before configuration and Prometheus / ingress-nginx with this profile. not configured means no URL was passed; unavailable or partial after configuration means the endpoint, series, labels or query results need inspection. Kubernetes API and metrics-server being available do not make HTTP metrics available.

If the check succeeds, start Fortem with the same arguments and open the namespace's Traffic tab:

fortem --context YOUR_CONTEXT --namespaces YOUR_NAMESPACE \
  --prometheus-url http://127.0.0.1:9091 \
  --traffic-profile ingress-nginx

What the numbers mean—and do not mean

The request count uses Prometheus increase(...[5m]) over a controller counter; RPS uses rate(...[5m]). Prometheus adjusts for counter resets and extrapolates to the window boundaries, so a five-minute estimate can be fractional. Fortem labels the source and window, and keeps 4xx separate from 5xx. If the duration histogram is missing, p95 remains unavailable rather than zero.

These are observations at the selected controller, not a count of successful application transactions. Retries, scrape gaps and different measurement points can change totals. Do not subtract an ingress counter from an application counter and call the difference “lost requests” without a matching measurement model.

Fortem supports Prometheus profiles for ingress-nginx, Traefik and Istio. Pick --traffic-profile from the metric family you actually have—not from whether the cluster is GKE, EKS, AKS, kind or k3s. The current adapter does not accept Prometheus credentials or custom headers, and does not provide a managed GKE Ingress, AWS ALB or Azure Application Gateway counter adapter by itself. See Fortem's traffic source boundary.

Need recent request rows?

Fortem can separately read recent ingress-nginx controller access logs through Kubernetes pods/log, if your identity can read those pods and the controller uses its supported default log format. That is a bounded, recent log view—not the source of historical request totals, and not an application trace. Review the read-only permission boundary before enabling access to potentially sensitive ingress logs.

Related check: if it is CPU/RAM rather than HTTP traffic that is missing, use the node metrics permission guide.

Sources and scope

Checked September 24, 2026. Commands and Fortem behavior were verified against the implementation and synthetic adapter tests; this article did not query a reader's live cluster.