LLM agents flunk CRM and confidentiality tasks

LLM agents flunk CRM and confidentiality tasks

: 6-in-10 success rate for single-step tasks

AI in CRM: Promise and Pitfalls Revealed by Salesforce Study

The rapid evolution of artificial intelligence, particularly in the realm of Large Language Models (LLMs), has promised transformative potential for business applications. Yet, a recent benchmark study led by Salesforce AI researcher Kung-Hsiang Huang has cast a shadow over the readiness of LLM-based agents for real-world customer relationship management (CRM) tasks. Published in a detailed academic paper on arXiv and reported by The Register, t…

Salesforce Study Finds LLM Agents Flunk CRM and Confidentiality Tests

A new Salesforce-led study found that LLM-based AI agents struggle with real-world CRM tasks, achieving only 58% success on simple tasks and dropping to 35% on multi-step ones. They also demonstrated poor confidentiality awareness. "Agents demonstrate low confidentiality awareness, which, while impr...

Salesforce study finds LLM agents flunk CRM and confidentiality tests

6-in-10 success rate for single-step tasks A new benchmark developed by academics shows that LLM-based AI agents perform below par on standard CRM tests and fail to understand the need for customer confidentiality. . . .