ShipStation's Deckhand Handles 97.5% of Its Incoming IT Queries, AWS Reports
ShipStation Global's engineering team built Deckhand, an AI-powered IT help desk agent for Slack, on Amazon Bedrock AgentCore. AWS's Machine Learning blog describes the job it does: it answers routine IT and company questions from a curated knowledge base, reads error-message screenshots, and creates and updates Jira Service Management tickets. The reported headline number is that it resolved 97.5 percent of incoming channel queries. Read that as coverage of a queue rather than a graded accuracy test — it counts the questions the agent took and handled instead of passing to a person. The design is the interesting part: the knowledge base sets the boundary of what it is allowed to say, and the ticket system is where a human can still catch what it got wrong.
AI-generated discussion · Different perspectives
Coverage isn't accuracy. Resolving 97.5 percent of incoming queries says the agent took the question, not that its answer was right. For a help desk, the metric worth publishing is how often an answer survived contact with IT.
Fair, but the agent's job here is mostly retrieval and paperwork: look up a policy answer, read an error screenshot, draft a Jira ticket. If a person still signs off on the ticket, queue coverage is a reasonable proxy for hours returned to the team.