on this page
After I gave a talk based on the last field guide, the questions kept circling back to the same thing. The parts of an AI estate were clear enough. People knew about runtimes, gateways, serving systems, tracking platforms, data stores, and agents. What they could not see was how those parts became a system. Where was the wiring?
The field guide had given them an inventory when what they wanted was a map.
A scanner can list the services in an afternoon, but it cannot explain how authority or data moves between them. Much of the risk follows routes hidden in configuration, environment variables, service accounts, and the quiet agreements that let one service rely on another. Reconstructing those routes takes more than enumeration.
To make those routes visible, I keep coming back to one image.
There is a surface, and there are tunnels.
On the surface sit the services a scan can reach, each one like a building whose doors can be tested from the street. Beneath them run the passages carrying credentials, trust, and data from one building to the next.
From the street, every building appears to stand alone. Once someone gets through an open door, however, the connections inside determine how far they can travel. The next building may let them in simply because it was designed to trust the first.
Eight services appear as a compact two-row inventory above a street line. Below it, the Ray to MLflow to TGI credential chain leads, followed by a provider credential path, a shared-corpus data path, and two trust paths from MCP and Kubernetes.
What a scan misses
A network sweep finds listeners and fingerprints products. In the lab, that produces Ray on port 8265, MLflow on 5000, Qdrant on 6333, and several agent endpoints. A scanner can test whether each service accepts unauthenticated requests and assign a severity to what it finds.
What the scan cannot see is the runtime state connecting them. A Ray job receives credentials for MLflow through its environment. An MLflow run records a token accepted by the TGI gateway. The RAG application sends retrieval queries to Qdrant. Those relationships live in job definitions, run metadata, application configuration, and cloud identity rather than in the response banner from any single port.
Assessing each service alone therefore understates the path. An exposed Ray dashboard and a weakly protected MLflow server may each deserve a medium rating, yet the credential passed between them can turn the pair into one route toward the model gateway.
Three kinds of tunnel
Most of these connections use ordinary HTTP or storage APIs. Their security effect depends on what crosses the connection. In practice, the consequential paths carry credentials, extend trust, or provide direct access to shared data.
Three panels compare credential, trust, and data tunnels with a lab example and production analogue for each.
Credential tunnels form when one service stores a credential that another service accepts. In the lab, Ray distributes environment variables from a job’s runtime_env to its workers so the training code can authenticate to MLflow and log a run. Recovering that Basic credential turns access to Ray into authenticated access to the tracking server. One MLflow run then records the Hugging Face token accepted by the TGI gateway, creating the next hop in the chain.
LiteLLM uses the same pattern at a gateway boundary. Its route configuration maps a model name to a provider endpoint, then resolves the provider’s API key from an environment variable or secret store before forwarding the request. To map this kind of connection, record where the credential is stored, which service accepts it, what permissions it grants, and how long it remains valid. Runtime environments, container environment variables, secret references, run parameters, and gateway configuration are the places to inspect.
Trust tunnels appear when one component accepts an identity or capability supplied by another. Kubernetes commonly projects a service-account token into a pod. The pod presents that token to the Kubernetes API, which uses RBAC to decide what the workload may read or change. Code execution inside the pod therefore inherits whatever API authority the service account was given, even when no credential is copied out of the cluster.
MCP and A2A create a different kind of trust path. An MCP client requests tool definitions from a server and makes those descriptions available to the model. An A2A client reads an Agent Card to discover an agent’s capabilities and advertised security schemes. Neither description proves that the remote party should be trusted. MCP clients still need approval and policy around tool calls, while A2A clients need authenticated requests and should verify signed cards when signatures are present. Mapping these paths means recording who supplies the claim, who validates it, and which action becomes possible when the claim is accepted.
Data tunnels connect services through storage rather than through a direct request between them. A training job writes a model artifact to S3 or Google Cloud Storage, MLflow records the artifact URI with the run, and a serving system later reads the same object. A Kubernetes InferenceService may also name that storage location in its model specification. Training, tracking, and serving are now connected through the bucket even if none calls the others directly.
The access path depends on the MLflow configuration. In proxied mode, the tracking server holds the storage credentials and transfers artifacts on behalf of clients. With direct artifact access, clients receive the storage URI and use their own cloud identity to reach the bucket. A compromised job or pod with broad storage permissions can then read or replace the model without going through MLflow’s application controls.
RAG systems follow a similar pattern. The application converts a question into an embedding and queries a vector database for matching records. Those records often include document chunks and metadata alongside the vectors. If the database is reachable and its credential is exposed, someone can enumerate collections and query those records directly, bypassing the authorization and filtering implemented by the chat application.
The categories can overlap on one connection. A Ray worker calling MLflow is ordinary HTTP traffic, a credential tunnel because authentication material crosses it, and potentially the first step toward a data tunnel if the run reveals a storage location. Labeling the edge by what it carries makes the next security boundary explicit.
Every service is a junction
Drawing those connections changes the way each service looks. What first appeared to be an isolated box becomes a junction where writers, readers, storage, and authority meet. MLflow brings all four together clearly.
MLflow sits at the center with writers above, readers to the right, object storage below, an inbound credential to the left, and an inference token inside its run parameters.
Ray jobs and training pipelines use MLflow’s tracking API to write parameters, metrics, tags, and artifact locations. Registry and deployment systems read model versions back out. The artifact path connects both sides to object storage, either through the tracking server in proxied mode or directly from the client in direct-access mode. One service now sits between compute, storage, and serving.
A useful map records four facts on every edge. It names the caller, the operation, the credential, and the resource being accessed. Apply the same method to Ray, a RAG application, or the Kubernetes API and the dependencies become concrete enough to test. The diagram stops being a collection of product names and starts showing which authority can move where.
The map
Set the important junctions and their connections on one canvas, and the inventory becomes an attack graph. Read from that perspective, the useful question changes from what is here to what can lead to what.
A prominent verified path runs from Ray to the MLflow gateway and then to the TGI gateway. Four smaller supporting groups show Jupyter reaching MCP, LiteLLM reaching model providers, the RAG application reaching Qdrant and its corpus, and Kubernetes and A2A control planes reaching their workloads.
The lit route comes from the aipostex lab, running from Ray through the MLflow gateway and on to the TGI gateway. Its labels record the evidence behind each transition. “Read-confirmed” means the material needed for the next hop was recovered, while “execution-confirmed” means the final authority was exercised in the lab rather than inferred from configuration.
With the route visible, the tidy report of medium findings reads differently. The exposed Ray dashboard also serves as the entrance to a credential path, while the vector store opens onto the private corpus the product was built to protect. The map supplies the relationships that the surface scan left out.
The same wire, two directions
During normal operation, a RAG application sends a prompt through a gateway to a model runtime. In the lab attack path, access begins at Ray, moves to MLflow with a recovered credential, and reaches TGI with the token found in the run. Both flows use services and protocols the product already needs.
A request descends from a RAG application through a gateway to a model runtime. A stolen credential ascends from Ray through MLflow to the same gateway. The paths meet at the gateway.
There is no CVE for a Ray job carrying the credential it needs to log an MLflow run because the job is supposed to reach MLflow. Removing the connection would also break the workflow. A practical defense instead narrows the MLflow credential, keeps gateway tokens out of run metadata, limits storage permissions, and alerts when a service uses its legitimate access in an unexpected way.
A diagram is not a boundary
Architecture diagrams often imply boundaries that the network never enforces. Five hostnames do not create five security zones when every host can still reach every other host.
Five lab hosts sit along one bidirectional network line. Every host can reach the others because no east-west boundary is enforced.
Shadow AI often grows on whatever network was already available because the immediate goal was to ship a model. Kubernetes does not change the underlying rule. A NetworkPolicy isolates traffic only when it selects the pod for that direction and the cluster network plugin enforces it. Without both conditions, the boundary remains a label on a diagram.
The lab keeps the shape
A lab becomes misleading when it simplifies away the relationships under study. Replacing a production service with a smaller stand-in causes no problem as long as the same roles still exchange credentials, trust, and data.
Three compact comparisons pair production and lab credential, data, and trust paths. Production compute, tracking, and gateway roles map to Ray, MLflow, and TGI. Production tracking and storage map to MLflow and the object-store stand-in. Production orchestration and workloads map to the Kubernetes API and pods.
Managed platforms may add stronger identity, artifact proxying, encryption, network policy, and audit trails, so the diagram does not claim that every deployment has the same defaults or controls. The topology still persists. Compute logs to tracking, tracking resolves artifacts, gateways hold downstream authority, applications query data stores, and orchestrators issue workload identity. The lab preserves those relationships in a form that can be inspected.
What the map is for
Taken together, the diagrams show why a service inventory can never describe the whole estate. A scanner reports what answers on the network. An operator also needs to know what each service can reach, what it trusts, and what authority it carries into the next system.
Once those relationships are added, a stack of medium findings may resolve into a single path from an open port to a private model. Both attackers and defenders work in the gap between those two views.
After every finding, ask one more question. What does this connect to?
The next post follows the credential chain lit in Figure 4 from end to end, using the actual commands that turn the route on the map into a working path.
Operator references
- Ray runtime environments, including per-job environment variables.
- MLflow artifact stores and the distinction between backing storage and tracking metadata.
- LiteLLM proxy configuration, including environment-backed and centrally managed provider credentials.
- Kubernetes service accounts and projected workload credentials.
- Kubernetes NetworkPolicy behavior, including the default non-isolated state.
- MCP tools, sampling, and elicitation.
- A2A protocol security and Agent Card signing.