Solution · Computer Vision Services
AI Video Analytics & Video Intelligence
AI video analytics turns camera and drone footage into searchable events and verified alerts. How it works in 2026, where it fails, and how to test a vendor.
Updated:
AI video analytics is software that watches video so people do not have to: it detects objects and events in live or recorded footage, raises alerts, and makes hours of footage searchable. Defense and security technology companies now sell the same category as video intelligence, the step from flagging events to answering questions about them. DataWise designs video analytics pipelines that run where the footage must stay, on your servers, next to the camera, or inside a closed network, and validates them on your own footage before anyone commits to a build.
What is AI video analytics, and what is video intelligence?
Classic video analytics answers questions fixed at installation: is there a person in this zone, did a vehicle stop here. NVIDIA describes this generation as “fixed-function, limited models that are designed to detect and identify only a select set of predefined objects” (NVIDIA).
Video intelligence lets an operator ask a new question after the fact. Conntour, an Israeli company, describes an “AI video intelligence platform” that searches any camera network in natural language, “operating fully on-premises”, with example queries such as “a van with a print of fruits on it” (PR Newswire). Maris-Tech presents “edge-based video intelligence” for “contested and bandwidth-constrained environments” (GlobeNewswire).
Defense intelligence uses its own vocabulary for the same work. ISR is the mission: intelligence, surveillance and reconnaissance. Exploitation is the analysis step: the US Department of Defense’s Project Maven began in 2017 by applying computer vision to the processing, exploitation and dissemination (PED) of full-motion video (FMV) from unmanned aircraft (GlobalSecurity). A buyer who uses these terms correctly gets better answers from vendors, because each term implies a different workflow and a different person at the end of it.
What can the state of the art do in 2026, and where does it stop?
Three generations of models now coexist, and each has a known limit.
- Dedicated detectors and trackers, trained on your classes. Fast and predictable per frame, cheap enough to run on every stream, blind to anything they were not trained on.
- Open-vocabulary detection and segmentation, where you name the object in words. Grounding DINO reaches 52.5 AP on the COCO benchmark without COCO training data, but a mean of 26.1 AP on ODinW, a benchmark built from real-world domains (arXiv). SAM 3 detects, segments and tracks every instance of a concept named by a short noun phrase or an example image, in images and in video (arXiv).
- Vision-language models (VLMs), which describe, summarize and answer questions. Qwen2.5-VL processes video of up to hours with second-level event localization (arXiv). Yet four leading VLMs averaged 58.07% on seven visual tasks a person finds trivial, such as counting where two lines intersect (arXiv).
The practical reading: language makes video searchable, not reliable. Counting, small or distant objects and exact spatial relations still belong to dedicated models whose error you have measured on your footage.
Which architecture fits: rules, detectors, or vision-language models?
| Approach | Good at | Costs and limits | Fits when |
|---|---|---|---|
| Rules on motion and zones | Cheap, explainable, runs inside the camera | Many false alarms from weather, foliage and lighting changes | Simple perimeter events in stable scenes |
| Dedicated detector and tracker | Fast, predictable, runs at the edge | Needs labeled footage; sees only its trained classes | A fixed set of events that matter every day |
| Open-vocabulary detector or segmenter | New classes without retraining | Accuracy drops outside web-like imagery; heavier compute | Rare classes, exploration, bootstrapping labels |
| Vision-language model | Search, summaries, questions about a clip | Slow and costly per frame; can describe things that are not there | Investigation, review and alert verification |
Most production systems are cascades. A light detector runs on every frame, often at the edge, and only its candidate events reach a heavier model for review. NVIDIA’s reference design follows this pattern, sending clips triggered by a computer vision pipeline to a VLM for review, alerts and questions (NVIDIA). A cascade keeps cost proportional to events rather than to hours of footage, and keeps the expensive model out of the real-time path.
Where each stage runs follows from latency, bandwidth and data rules, covered in edge AI for computer vision. When footage cannot leave the building, every stage, including the language model, has to be self-hosted, the subject of secure and air-gapped AI. What an AI agent may then do with an event is a separate design question, covered in vision AI agents.
Why do video analytics deployments fail?
- False alarms, not misses, end adoption. An operator who clears dozens of false alerts a shift stops looking. Track false alarms per camera per day alongside detection rates.
- Pixels on target. A model cannot recognize what the camera does not resolve. Placement, focal length and resolution set the ceiling before any model does.
- Domain shift. Night, rain, thermal imagery, glare and a new camera angle. A model validated in daylight is unvalidated at night.
- Compression. Streams are compressed for storage and bandwidth, and small, fast objects lose detail first.
- Drift. Seasons, construction and repainted lines. Accuracy at handover is not accuracy in month six.
- Latency. An alert that arrives after the event is a report.
- Fluent errors. A VLM can describe an object or an action that is not in the clip, so its output needs a check before it triggers anything.
How should you evaluate a video analytics vendor?
- Test on your footage, including the hard cases: night, rain, the camera everyone complains about, and lookalikes such as animals, shadows and reflections.
- Ask for precision and recall per event type at the threshold you will run, and for false alarms per camera per day.
- Measure latency end to end, from the event to the operator’s screen.
- Ask where every stage runs and what leaves your network, including model updates and telemetry.
- Ask how the system improves: who reviews errors, who labels, how often models are retrained, and how a bad update is rolled back.
- Ask who owns the models trained on your footage, and whether the system works with your existing cameras and video management system (VMS).
- Ask for the cost per camera per month at your scale, compute included.
Where does DataWise fit?
DataWise designs the analysis layer: the models, the cascade, the review interface and the evaluation on your footage. We are not a camera or VMS vendor, so the design starts from the cameras and systems you already run. The closest part of our record, in the verbs we use across this site: for a security-sensitive organization, DataWise architected vision object-detection as part of a fully isolated AI enclave, everything running inside the closed network with nothing leaving it. Every engagement starts with feasibility on your own footage, staged as described in how we work, and the wider practice is on our computer vision services page. For defense and critical infrastructure buyers, the constraints we design for are set out in AI for defense and critical infrastructure.
FAQ
Is video intelligence different from video analytics?
Mostly in what you can ask. Video analytics usually means detectors and rules fixed at installation. Video intelligence, the name defense and security technology companies now use, adds search and questions in plain language over live and recorded footage. A sound system combines both: dedicated detectors for the events that matter every day, language models for investigation and review.
Can it run on our existing cameras?
Usually, if the cameras resolve what you need to detect. The analysis reads standard video streams. The limit is pixels on target, lighting and angle, which feasibility checks camera by camera before any build.
Can the footage stay inside our network?
Yes. Every stage, including vision-language models, can be self-hosted. That constraint is familiar to us: DataWise architected vision object-detection inside a fully isolated AI enclave, with nothing leaving the network.
Do we need labeled footage to start?
Not to start. Open-vocabulary and vision-language models let feasibility begin without labels, and they speed up labeling later. Events that must be caught reliably every day usually still justify a dedicated model trained on labeled examples from your own cameras.