---
title: "Avid Solutions International uses IBM Bob to build Elmer in Winston-Salem, NC; accelerates pilots across three use cases"
sdDatePublished: "2026-08-28T18:08:00Z"
source: "https://www.ibm.com/new/product-blog/how-ibm-bob-helped-avid-solutions-connect-spatial-perception-to-agentic-control"
topics:
  - name: "artificial intelligence"
    identifier: "medtop:20001298"
  - name: "agriculture"
    identifier: "medtop:20000210"
  - name: "education"
    identifier: "medtop:05000000"
locations:
  - "Winston-Salem"
---


Avid Solutions International uses IBM Bob to build Elmer in Winston-Salem, NC; accelerates pilots across three use cases

How IBM Bob helped Avid Solutions connect spatial perception to agentic control

Learn how IBM Bob helped Avid Solutions build an AI architecture that interprets visual, spatial and operational context before proposing an action, and can support an impressively broad range of use cases. A startup founder reviewing a busy day, a field operator monitoring crops and a robotics student moving a project from simulation to hardware face the same problem: data is separated from the physical and temporal context needed to act. Elmer, an agentic operating system developed by Avid Solutions International, combines voice, gesture, vision and operational data before it reasons and proposes an action for approval. The system supports enterprise email and calendar visualization, agricultural operations and robotic lab work for students. The engineering challenge faced by Winston-Salem, N.C.-based Avid was to unify on-device processing of voice, camera and gesture inputs, cloud reasoning, digital workflows and physical control into a single platform. Without a shared architecture, expanding Elmer across new industries and use cases would require maintaining disconnected systems and duplicated development effort, so a common architecture is a critical requirement. These three use cases were not chosen for variety. They are the three environments where Avid already had customers, and together they stress every part of the same problem. Enterprise email and calendar work tests digital context and reversible actions. Agriculture tests spatial and temporal context where the consequences are physical. Robotics education tests the handoff from simulation to hardware under supervision. If one interaction model holds all three, it holds most of what sits between them. The Avid team used IBM Bob™ to understand Elmer’s repository, plan the perception-first redesign and coordinate changes across application logic, interfaces, integrations and deployment configuration. Bob accelerated development by helping to connect perception, reasoning, approval and execution into one draft-and-approve pattern. In this article, we’ll explore how OnticWorks.io used Bob to build a shared spatial reasoning pattern for enterprise productivity, agriculture and robotics education. The three use cases show how one interaction model can adapt to different inputs, constraints and users. Most agents return abstractions such as an email thread, sensor reading or simulator log. Users must reconstruct where an event is happening, what it affects and which action is safe. Elmer keeps spatial and temporal context inside the agent loop so recommendations include the relevant place, relationship and constraint. Elmer isn’t built for a single use case; it learns a domain rather than being hardwired to it. IBM Bob was used to stand up and improve each version, so a new pilot in a new use case can reach a working product quickly, without requiring a separate team each time. For enterprise users, Elmer combines Microsoft Graph or Google Workspace data with identity, location and timing to visualize email and calendar activity as a connected daily flow. It can associate an urgent email with an upcoming onsite meeting, flag travel time between appointments or propose a reschedule after a cancellation. The user approves any email or calendar change before execution. For agricultural users, Elmer combines drone position, multispectral imagery, soil moisture, weather and field history to locate crop stress or equipment problems. It can identify the affected zone, account for geofences and nearby equipment and recommend an irrigation change, treatment window or drone route for approval. For students at RoboLabWorks, a robotics learning platform, Elmer preserves the workflow as projects move from physics-aware simulation to classroom robotics kits or field hardware. Students can see why a drone encountered an obstacle or an irrigation mission missed a section, then revise the plan before an instructor approves physical execution. The common observe-propose-approve pattern connects software to machinery while protecting equipment from unreviewed commands. Elmer’s architecture separates fast local perception from heavier cloud reasoning, then separates proposals from execution. This boundary keeps the interface responsive and prevents a model output from becoming an action without an explicit control step. Whisper speech-to-text captures the voice locally. A webcam and MediaPipe track hands, faces and poses for gesture-aware interaction. Drone position, field sensors and multispectral data supply operational telemetry. Microsoft Graph, Google Workspace and GovPulse.io add enterprise context for email, calendar and opportunity workflows (read how Avid built the government procurement platform GovPulse.io on IBM watsonx Orchestrate®). Keeping voice and gesture processing near the user supports subsecond feedback and reduces the amount of raw interaction data that must leave the device. The perception layer converts multimodal input into objects, relationships, environmental changes and real-time anomalies. IBM watsonx.ai® supports multimodal reasoning with that spatial context. Bob helped the development team design and build this layer as part of the core loop, replacing the earlier pattern in which visual information primarily served as post-process output. Elmer and AvidRAG run a nine-layer agentic retrieval-augmented generation (RAG) system built on Red Hat® Openshift®. The system follows a ReAct loop—plan, act and observe—and uses NVIDIA Llama inference for chat, decisions and planning. It can support agents for morning briefings, inbox triage, opportunity discovery and fleet coordination without giving each workflow a separate interaction model. The control boundary is draft-and-approve. Elmer presents a proposed email, calendar change, drone route or payload action through a visual card, voice summary or gesture prompt. A person approves, changes or rejects the proposal. The system logs that decision. Safety controls block dangerous actions (a drone route that flies over people, or overwriting a locked calendar entry), enforce geofences and GPS checks and prevent conflicting physical operations. Digital controls can protect locked calendar entries and reduce the risk of sending a message to the wrong recipient. A Flask-based digital orchestrator carries out approved email, calendar and vendor actions. A physical orchestrator uses robotic software framework ROS 2 and MAVLink to sequence drone routes and payload operations. JavaScript 3D library Three.js renders the Elmer interface, Amazon Polly provides voice output and augmented-reality or hologram displays can present feedback. The same approval semantics apply on a macOS workstation, an iPad or a kiosk. The deployment plane matches computation to latency and complexity. Whisper and MediaPipe run locally, while NVIDIA Jetson supports field inference at the edge. Red Hat OpenShift hosts cloud reasoning and complex planning. This architecture supports local processing of gestures and voice events for latency and speed and scheduling on elastic infrastructure for workloads that benefit from larger models and multi-step planning. A key design choice is the boundary between reasoning and execution. Here, a “safety model” means the consistent set of controls that determines when Elmer can proceed automatically, when it must request human approval and which rules constrain an action before execution. An operator learns one pattern: Elmer observes, proposes and waits. That consistency helps users develop an accurate mental model of the system whether they’re reviewing an email or a drone route. It also gives product teams a clear place to add recipient checks, confidence thresholds, geofences and domain-specific rules. The agricultural deployment provides a concrete example. Multispectral imagery, soil-moisture readings, weather and drone telemetry can flag crop stress or equipment problems, then identify the affected location and propose a response. Across a 20-hectare field validation, the operation reduced irrigation water use by 58%, improved yields by 23% in managed zones and reduced routine monitoring labor by an estimated 40%. A sensor-fusion alert also identified the risk of fungal infection four days before visual symptoms appeared, enabling earlier intervention. RoboLabWorks shows the same architecture from a learning perspective. Its simulator introduces sensor noise, weather variability, geofences and equipment constraints instead of giving students idealized (and unrealistic) conditions. Thanks to the updated Elmer platform, the program launched in 2026, 12 months earlier than its original 2027 target. Twenty-four of 27 participating students successfully deployed autonomous agents to physical hardware. By keeping irreversible actions behind an approval step, the platform lets students test ambitious ideas without turning every error into equipment damage. In the calendar and email use case example, Elmer integrates with Microsoft 365 (or any other cloud-based productivity workspace). This approach enables it to turn a cluttered inbox and calendar into a clear visual picture of where time and attention go. Users can see their week at a glance and act on it, spotting overloaded days, meeting-heavy stretches and email that needs a reply, without digging through folders. All three use cases are essentially the same loop with different sensors: observe, propose, wait for approval and execute. Elmer is one assistant that learns a domain instead of being hardwired to one. For Avid, shared components lower the cost of supporting several markets. The team can reuse perception adapters, the reasoning loop, approval controls and deployment patterns instead of maintaining independent applications for administration, field robotics and education. For customers, that reuse can shorten implementation cycles and reduce training costs because workflows change while the familiar safety model—observe, propose, validate against applicable rules and obtain approval before execution—remains consistent. Elmer began as a set of useful but separate abstractions. Building it into a perception-first system required coordinated changes across local input processing, agent logic, safety controls, cloud deployment and physical integrations. IBM Bob helped Avid evolve a collection of independent capabilities into a reusable engineering platform. By combining repository-wide understanding with architectural reasoning and coordinated implementation, Bob accelerated development while creating a foundation that can scale across enterprise workflows, robotics and future AI applications. Avid used Bob to build a modular architecture and keep execution behind explicit approval. It turned one spatial-agent concept into a platform that can serve office workflows, field robotics and education without rebuilding the foundation for each use case. Users get faster access to relevant context and a consistent way to supervise consequential actions. The development team gets reusable components, a clearer path to new deployments and a lower long-term cost of change.