Search

Observability for AI Applications | COMPEL Framework

The article outlines a minimum observability plan for AI systems, covering traces, prompt and response capture, and evaluation signals. It recommends OpenTelemetry conventions and privacy controls so teams can investigate incidents and quality changes.

What Is AI Observability? A Guide for Developers

A guide on AI observability explains how it extends traditional monitoring by capturing AI-specific data to explain why AI outputs differ, are slow, or cost more, covering four AI system types and methods to spot non-determinism, silent failures, and cost spikes. It also contrasts AI observability with traditional monitoring and provides setup ideas, including tracing and tooling.

[2611.04487] Reconstruction-Based Diagnosis through Gödel-Regularized Virtual Observability

The work tackles fault isolation in sensor-limited systems by using virtual sensors and a normal-data-trained neural observer. It detects deviations from nominal behavior, derives local symptoms and onset times, and yields root-cause explanations, guided by Gödel logic, with evaluations on synthetic networks and real production datasets.

[2610.21108] Planning Commitment of LLM under Partial Observability

This work compares aggressive and conservative LLM planners in partially observable environments, showing a safety-efficiency trade-off with similar accuracy across three models; priors influence performance and reliability, and hybrid methods with tunable parameters are explored to balance risk and cost.

[2610.28611] Semantic Observability for Autonomous Systems Diagnostics

A multi-agent system called Semantic Observability unifies heterogeneous observability logs into a graph-based model to isolate root causes of failures. It uses causal graphs and temporal tracing to determine blast radii, then a LLM provides contextual RCAs and fixes, with open-source validation and plans for quantitative benchmarking.

The outer loop is designed around “when to start, when to stop, and what to hand over” — based on the concrete example of Sentry → Webhook → Codex.

Translated title. Original: 外ループは「いつ動かし、いつ止め、何を引き継ぐか」で設計する ― Sentry→Webhook→Codexの実例から

A practical blueprint for designing outer (external) loops that trigger AI agents like Codex via events or schedules, with inner loops detailing how the AI processes tasks. It covers start timing, stop conditions, hand-offs, and when to use open vs. closed loop feedback, illustrated through Sentry/Webhook workflows.

GitHub - B2ZO0/netcockpit: Your whole infrastructure, live, in one cockpit: auto-discovered topology, VMware, switches, alerts, logs and AI, even without access to your network gear. Prometheus · Grafana · Loki · Zabbix. EN/FR. · GitHub

NetCockpit combines available network and virtualization data to build a live infrastructure map, show device problems, and route alerts from Prometheus, Loki, and Zabbix. It supports VMware and Proxmox, optional AI diagnosis, and a Docker-based setup.

GitHub - hohieuai/colab-monitor: A local dashboard for the Colab sessions you run with the Colab CLI. · GitHub

Colab Monitor is a local dashboard for Colab CLI sessions that reads CLI data, shows compute units and machines, and streams job logs. It features sections for overview, machine details, logs, history, and settings, and runs read-only on the user’s machine.

Grafana guide, dashboards, logs, and alerts from scratch

Translated title. Original: Grafana rehberi, sıfırdan panolar, loglar ve uyarılar

Grafana is an open source visualization and monitoring platform that reads metrics, logs, and traces from various data sources via connectors, keeping data in the source systems. It comes in OSS (free), Enterprise (paid features and support), and Cloud (managed service with a free plan); OSS is AGPLv3. A working stack includes Prometheus, Loki, Tempo, Mimir, Alloy, and Pyroscope, with Grafana Play offering sample dashboards and Docker enabling quick setup. The document also presents a seven-step deployment map for end-to-end monitoring and alerting of a simple web service.

Introducing Azure Monitor dashboards with Grafana MCP: Telemetry for AI Agents, Signed In as You | Microsoft Community Hub

Microsoft introduced an Azure Monitor MCP server that lets AI agents query monitoring and analytics data and manage dashboards using the signed-in user's Azure permissions. The article describes setup, available tools, and a Copilot example analyzing storm damage data.