Skip to content
FlowHubFluxonLab
AI Agents & Chatbotsfree

Evaluate AI agent response relevance using OpenAI and cosine similarity

by JimleukUpdated Aug 2026
RequiresOpenAI
Share Post Share
OpenAI Chat Model1OpenAI Chat Mod…EvWhen fetching a dataset rowWhen fetching a…SeRemap InputEvEvaluationSeSet Input FieldsNONo Operation, do nothingNo Operation, d…AgAI AgentChWhen chat message receivedWhen chat messa…EvUpdate OutputEvUpdate MetricsOpenAI Chat ModelOpenAI Chat Mod…OPStructured Output ParserStructured Outp…CLAnswer RelevanceHRGet EmbeddingsCoCalculate Similarity ScoreCalculate Simil…CoQuestions to ItemsQuestions to It…SeCalculate Relevance ScoreCalculate Relev…12345678910111213141516
1/5
STEPS · 16
Starts on a Evaluation event

On an evaluation trigger, runs an OpenAI agent on chat inputs and scores response relevance against expected answers using cosine similarity.

Tags

eventcomplexOpenAIEngineeringAI Summarizationdiscoveredpending-review
Connects
OpenAI
CategoryAI Agents & Chatbots
Triggerevent
Complexitycomplex
Nodes17
AddedJun 27, 2026