Up front: this is an ongoing test, and we don't have results to share yet. This article records what we want to verify and why we designed the test this way. We'll update it once results are in.
Warehouse and sales staff at stone plants answer the same questions every day: "What material do we have right now?" "How many slabs does this block have, and are they available?" "Do we have polished slabs between 250 and 300 long?" The answers are all in the ERP, but spread across different pages and several layers of pop-ups, so each lookup takes many clicks.
The ERP we built for primary stone processing plants already includes an AI assistant that answers questions from the employee handbook and operating manuals. What we want to know next is: can an AI agent with computer-use capability (able to see the screen, click and type) operate the ERP directly like a person, complete these lookups, and give trustworthy answers?
If it works, many legacy systems wouldn't need new APIs first — AI could help through the existing user interface.
The first round is read-only: no creating, editing or deleting. Each task maps to a real work scenario and has acceptance criteria, so results can be checked item by item.
Describe the scenario and expected output, not the clicks. We want to test whether the agent can understand the interface on its own, not follow a script.
Treat honesty as its own test. No matter how good the lookups are, an agent that makes things up can't be used for quotes or shipping decisions. This task costs the least, so it runs early.
A person handles login. Credentials aren't given to the agent; a person logs in, and the agent takes over from the home screen.
Once testing is complete, we'll summarize which tasks the agent handled reliably, which it couldn't yet, and why — and update this article.
If your company also wants AI to help operate existing systems, let's talk about your situation. Until the test results are in, we'll be honest about what we can do today and won't take on projects with unproven solutions.