0
I Traced a Multi-Step LLM Agent With Self-Hosted SigNoz. One Feature Sold Me.
TL;DR: Self-hosting SigNoz for an LLM multi-step agent turned out to be surprisingly easy and revealing. The standout feature was seamless OpenTelemetry observability, not SigNoz-specific code.
The author set up a four-step LLM pipeline (plan, retrieve, generate, synthesize) and wired it to SigNoz using a minimal 8-line YAML with Foundry. Telemetry is collected via OpenTelemetry OTLP without any app changes, and GenAI semantic conventions are attached to each LLM call. Admin signup is required to start ingestion, after which traces, metrics, and logs flow through the system. The result highlighted that traditional debugging misses what step saw what, how much it cost, and which model answered. The workload runs entirely over standard OTLP to localhost:4318, with the UI at localhost:8080. The setup emphasizes ease of self-hosting and non-intrusive instrumentation best practices.
Question for the room: What small telemetry or observability change did you make to your LLM pipeline that yielded the biggest debugging or cost insight?
— via dev.to
Add a comment
0/2000