Skip to main content

Introduction

Quick Summary​

Confident AI was designed for teams to bring LLM evaluations from development to production. It is an all-in-one platform that unlocks deepeval's full potential by allowing you to:

  • evaluate LLM applications continously in production
  • centralize and standardize evaluation datasets on the cloud
  • trace and debug LLM applications during evaluation
  • keep track of the evaluation history of your LLM application
  • generate evaluation-based summary reports for relevant stakeholders

Continuous Evaluation​

Continuous evaluation refers to the process of evaluating LLM applications in not just development, but also in production and throughout the lifetime of your LLM application. Here's a quick diagram outlining how Confident AI enables this process:

Everything in deepeval is already automatically integrated with Confident AI, including deepeval's custom metrics. To start using Confident AI with deepeval, simply login in the CLI:

deepeval login

Follow the instructions displayed on the CLI (to create an account, get your Confident API key, paste it in the CLI), and you're good to go.

tip

You can also login directly in Python if you already have a Confident API Key:

deepeval.login_with_confident_api_key("your-confident-api-key")

Or, via the CLI:

deepeval login --confident-api-key "your-confident-api-key"