LLM Observability with Arize Phoenix : Open-Source Tracing, Evaluation, and Debugging for AI Applications

"LLM Observability with Arize Phoenix: Open-Source Tracing, Evaluation, and Debugging for AI Applications"

Modern LLM systems fail in ways that ordinary logs, metrics, and dashboards cannot explain. This book is written for experienced engineers, platform teams, ML practitioners, and technical leads who need rigorous visibility into agent workflows, RAG pipelines, prompt behavior, and production-quality regressions. Centered on Arize Phoenix OSS, it shows how to make complex AI applications observable, debuggable, and operationally trustworthy.

Readers will learn how Phoenix fits into the contemporary LLM stack, how OpenTelemetry and OpenInference shape trace data, and how to instrument both framework-driven and highly customized systems. The book then moves into trace-driven debugging, retrieval analysis, evaluator design, dataset curation, experiment workflows, and programmatic automation for CI and release gates. By the end, readers will be able to connect execution traces to quality signals, isolate failure modes precisely, and build a disciplined feedback loop for continuous improvement.

The treatment is version-aware, operationally grounded, and aimed at advanced usage rather than introductory walkthroughs. It also extends beyond local experimentation to cover self-hosting, storage architecture, scaling, security, and governance, making it especially valuable for teams adopting Phoenix as shared infrastructure rather than a single-developer tool.

Over dit boek

"LLM Observability with Arize Phoenix: Open-Source Tracing, Evaluation, and Debugging for AI Applications"

Modern LLM systems fail in ways that ordinary logs, metrics, and dashboards cannot explain. This book is written for experienced engineers, platform teams, ML practitioners, and technical leads who need rigorous visibility into agent workflows, RAG pipelines, prompt behavior, and production-quality regressions. Centered on Arize Phoenix OSS, it shows how to make complex AI applications observable, debuggable, and operationally trustworthy.

Readers will learn how Phoenix fits into the contemporary LLM stack, how OpenTelemetry and OpenInference shape trace data, and how to instrument both framework-driven and highly customized systems. The book then moves into trace-driven debugging, retrieval analysis, evaluator design, dataset curation, experiment workflows, and programmatic automation for CI and release gates. By the end, readers will be able to connect execution traces to quality signals, isolate failure modes precisely, and build a disciplined feedback loop for continuous improvement.

The treatment is version-aware, operationally grounded, and aimed at advanced usage rather than introductory walkthroughs. It also extends beyond local experimentation to cover self-hosting, storage architecture, scaling, security, and governance, making it especially valuable for teams adopting Phoenix as shared infrastructure rather than a single-developer tool.

Begin vandaag nog met dit boek voor € 0

  • Krijg volledige toegang tot alle boeken in de app tijdens de proefperiode
  • Geen verplichtingen, op elk moment annuleren
Probeer nu gratis
Meer dan 52.000 mensen hebben Nextory 5 sterren gegeven in de App store en op Google Play.

Meer van Caelan Wexford

Sla de lijst over

  1. Type-Safe Databases with Drizzle ORM : Schema, Queries, and Migrations for Modern TypeScript Applications

    Caelan Wexford

  2. Valkey : The Open-Source Redis Successor — Caching, Data Structures, and High-Performance In-Memory Systems at Scale

    Caelan Wexford

  3. Edge-First Web Development with Hono : Building Fast, Portable APIs Across Cloudflare, Deno, Bun, and Node.js

    Caelan Wexford

  4. Retrieval-Augmented Generation with RAGFlow : Building Document-Aware, Deep-Search RAG Engines for Production

    Caelan Wexford

  5. Fast LLM Fine-Tuning with Unsloth : Train and Optimize Open Models Faster and with Less Memory

    Caelan Wexford

  6. Serverless AI and Data with Modal : Running GPU Workloads, Pipelines, and Apps Without Managing Infrastructure

    Caelan Wexford

  7. Evaluating Generative AI with Galileo : Measuring Quality, Detecting Hallucinations, and Monitoring LLMs in Production

    Caelan Wexford

  8. High-Speed Inference with Groq : Deploying Ultra-Low-Latency LLM Applications on the LPU

    Caelan Wexford

  9. Modern Python Workflows with uv : Lightning-Fast Packaging, Dependency, and Project Management

    Caelan Wexford

  10. Structured LLM Outputs with Instructor : Reliable, Type-Safe Data Extraction and Validation from Language Models

    Caelan Wexford

  11. Structured Generation with Outlines : Constrained Decoding for Reliable, Schema-Driven LLM Output

    Caelan Wexford

Gerelateerde categorieën