Open Source Observability Comes of Age in 2022

Observability is increasingly important for meeting perceived business goals. Despite the benefits to performance data, organizations have faced challenges in realizing the full potential of observation based on implementation, distributed software tools, and other technological hurdles. But with the new innovation, the era of true observation is upon us.

innovative changes

The open source surveillance landscape, in particular, is going through a phase of innovation. Powered by Distributed Tracking Coming of Age with OpenTelemetry, and eBPF takes center stage in terms of restructuring as the next frontier of Monitor and Monitor on LinuxInnovation at an all-time high.

kernel technology

The extended Berkeley Packet Filter (eBPF) introduced an impressive technology that was native to the Linux kernel. Born with the goal of having a programmable way of dealing with networking, eBPF has evolved into a fairly generic and efficient facility for secure and dynamic access to the Linux kernel, which, among many other things, enables telemetry extraction for monitoring and security purposes. eBPF is strong and growing in popularity, and Microsoft is working on a port to Windows.


In terms of observability, eBPF has proven to be remarkably versatile in 2021. For example, there are tools that use eBPF to perform distributed tracing, such as pixie project which was opened in mid-2021 by New Relic. But the most promising aspect of eBPF for 2022 is the fact that, using eBPF, it is possible to carry out continuous profiling of applications in production. Examples include the above-mentioned Pixie project and garden. It’s also proven by tools like Google Tools cloud profile And for many commercial monitoring tools, having a low-cost, always-on production profile is a powerful way to troubleshoot latency and memory issues in production environments, which are known to be difficult and time-consuming to reproduce in test labs.

obvious differences

While production is now profiled with eBPF, it is not a generic capability for use with most applications. The way profiling works varies surprisingly across different runtimes and programming languages. While nearly all languages ​​behave similarly with regard to CPU usage and memory allocation (and even in such foundational computing matters, there are differences), the method of practical synchronization differs markedly, pretty much across the board.

For example, Java uses threads within the Java Virtual Machine that are mapped to threads in the operating system (to be fair, there are other options as well, such as NIO, libraries like RxJava, Reactor, Vert.x as well as Blame Project will eventually ship). In Node.js, on the other hand, concurrency is mostly handled with event loop, and the “real” threads are extracted into the OS away from the developer. Such differences are important for the meta-analyst, because the data presented to the person troubleshooting the problem must be “translated” into the concurrency model used by the programming language, and this requires explicit support in the profile for specific runtimes.

Open source monitoring starts in 2022

Fortunately, 2022 will be the year that brings us full-featured, open-source, eBPF-based production definition tools that understand the details of many runtimes prevalent in today’s cloud-native applications, such as Java, Node.js, Python, .NET as well as compiled languages Like Go and Rust. This will be a game changer for software optimization and faster problem solving, for all DevOps, SRE, and Operators.

As we look to the year ahead, revolutionary technology like eBPF means that the rate of innovation happening at the OS level is only setting the stage for a year of true observation.

Leave a Comment