Lead photo by Reshape Automation

General-purpose AI models struggled to answer industrial parts questions reliably in Reshape Automation’s Industrial AI Accuracy Index 2026, announced Oct. 7.

The company tested 100 questions across 14 manufacturers’ catalogs, making 1,500 API calls. Without tools, GPT-6 Astra, Claude and Gemini scored 12-14%. With web search, GPT-6 Astra reached 52% and Claude 40.5%. Gemini was tested without tools only.

Scores included half-credit for partially correct responses. GPT-6 Astra’s leading result included 46 fully correct answers out of 100.

Repeated questions produced different correctness verdicts in 13.2% of question-and-model-configuration combinations. An estimated 36.5% of incorrect answers supplied a specific fact, figure or code without qualifying the response.

Reshape, which sells industrial AI technology, wrote and graded the questions without third-party review. Its manufacturer-verified answers served as the reference set; the study did not establish the accuracy of its own system’s unreviewed responses. The findings represent the tested models and conditions.

MDM Analysis

The findings give practical context to a central challenge in NAW’s 2026 AI Adoption Index: 68.1% of respondents ranked data quality and integration among their top three adoption barriers. Only 39.4% reported a formal data inventory, while 32.2% had formal AI governance.

For distributors, those gaps matter when AI moves into product selection, substitutions and technical support. Current specifications, compatibility rules and manufacturer-approved alternatives need to be accessible and verifiable before an automated recommendation reaches a customer.

The two studies measure different things and do not establish a statistical correlation. Still, Reshape’s results help explain why broader adoption must be accompanied by stronger controls.

NAW’s Index also found that 47.4% of respondents reported moderate or significant back-office impact from AI. That value can coexist with the need for expert review of product recommendations.

Distributors should test prospective tools against their own catalogs and difficult customer questions, require traceable sources and establish when answers must escalate to a specialist. Track accuracy, returns and rework alongside response speed. Faster quoting creates value only when the quoted product meets the customer’s requirements.