Observability

People always say there are no stupid questions, and then you read the title of this post and you’re not so sure anymore. You start to doubt my sanity, or at least suspect that I’m a troll. However, as it is with most apparently stupid questions, there is something to learn from the answer if you explore it. To spare you from reading the rest of this, the short answer is “Yes, but…” and the long answer is more of a theoretical observation with some linguistic subtleties. So if you’re not interested in that, you can leave and do something fun, otherwise don’t say I didn’t warn you! ...

I did an experiment on splitting out a monolithic application into multiple “virtual services” in OpenTelemetry to have modules visualized independently on service maps. I am not sure if this is a good idea and something you should replicate in practice, since it might violate some best practices. However, I wanted to see how I can do it. Since (as far as I know) all otel backends are only able to provide such a map/graph visualization using service.name from the resource attributes, I tried out what happens if I create one TracerProvider per module with module-specific service.* attributes. ...

When you’re heads-down in your own area of expertise, it’s easy to forget that what’s obvious to you might not be to others. As you might have seen in previous posts, I learned that for me using pen and paper from time to time helps uncover unknown knowns in my head. Last time, it was why the three pillars need to go. This time, it’s context propagation, and its surprising relationship to metrics. ...

I just read another post introducing traces, metrics, and logs using that analogy, which reminded me to re-share that excellent piece by Ted Young on The New Stack from a few years ago: Modern Observability Is a Single Braid of Data Ted argued the pillars are no longer load‑bearing and suggests a better framing: the “Single Braid of Data”. So let’s wheel the pillars into the museum, rope off the exhibit, and hang a small plaque: “Historic framing.” As we do with once‑cherished pillars that are no longer load‑bearing. ...

I have a confession to make: I struggled for a long time with the statement: “Observability helps you uncover unknown unknowns.” It never quite sat right with me. If you can uncover an “unknown unknown” using Observability, wasn’t it always, in some sense, a known unknown? Or perhaps even an unknown known you just hadn’t noticed? That logical loop kept bothering me. Recently, I realized where my understanding failed, and thought this might be worth sharing: ...