A systematic review in Mathematics maps the techniques that squeeze AI models onto industrial hardware — and the trade-offs quality engineers need to understand before they deploy one.
The distance between what a state-of-the-art AI model demands and what a device on a production line can actually supply is not a gap so much as a canyon. A review published in the MDPI journal Mathematics puts a hard number on it: training the OPT-175B model required close to a thousand high-end 80GB GPUs, each running at roughly 147 TFLOP/s. The best smartphone-class silicon available today delivers under 3 TFLOP/s.
That mismatch is the central problem of “edge intelligence” — the discipline of making AI models small enough, fast enough, and frugal enough to run where the data is generated rather than in a distant data center. The review, led by Tianyu Wang and Dong Li of the State Key Laboratory of Robotics at the Shenyang Institute of Automation, Chinese Academy of Sciences, with co-authors from Beihang University, surveys where the field stands and where it keeps getting stuck.
What the review covers
The authors screened roughly 1,870 papers across IEEE Xplore, Scopus, ACM Digital Library and Google Scholar, narrowing to 170 for full review under Kitchenham and PRISMA methodology. The resulting picture is heavily weighted toward the shop floor: industrial applications account for about 45% of the included studies, ahead of healthcare and autonomous systems. Model compression dominates the technique mix at 58%, with neural architecture search and federated learning splitting most of the remainder. Roughly 70% of the literature was published after 2021.
Notably, the team used ChatGPT-4 to independently re-screen a fifth of the candidate papers as a consistency check, reporting 92% agreement with their manual screening — a small methodological detail that says something about how quickly AI-assisted review is becoming normal practice.
The toolkit
Four techniques carry most of the load, and each buys efficiency with a different currency.
Sparsity and pruning strip out redundant parameters. The lineage runs from Optimal Brain Damage and Optimal Brain Surgeon through Han and colleagues’ work, which cut model size by a factor of 35 without meaningful accuracy loss — the result the authors treat as the field’s founding moment.
Quantization drops numerical precision, converting 32-bit floating point weights to 8-bit integers or lower. INT8 has become the de facto standard for edge deployment, trimming model size by as much as 75% while staying close to full-precision accuracy. Push to very low bit-widths, though, and accuracy starts to give.
Knowledge distillation trains a small “student” model on the output distribution of a large “teacher.” Because the teacher’s soft probabilities carry information about relationships between classes that hard labels do not, the student generalizes better than its parameter count suggests. The catch is the dependency: you need a well-aligned teacher.
Neural architecture search and federated learning round out the set. NAS automates model design against specific hardware constraints. Federated learning lets multiple plants or production lines jointly train a model without sensitive process data ever leaving the site — at the cost of communication overhead.
What it looks like in a plant
The review positions the edge layer as the connective tissue in a four-tier manufacturing stack, sitting between the device layer of sensors, actuators and PLCs and the operations layer running MES and SCADA. That is where local inference, data filtering and real-time analytics happen.
A case study drawn from prior work by Lee and colleagues sketches the payoff. An advanced machining facility instrumented critical equipment with vibration, temperature and acoustic emission sensors, ran preprocessing and anomaly detection on embedded edge modules, and pushed features to fog nodes for failure prediction. The reported outcomes: 25% less unplanned downtime, a 30% reduction in defect rates via edge-deployed visual inspection, and sub-10 millisecond response latency. Sensitive production data stayed on site.
Worth flagging for readers evaluating vendor claims: these are figures from a case study the authors cite, not from an experiment they ran themselves. Treat them as an illustration of the ceiling rather than a benchmark.
The four things that still break
The review is blunt about what remains unsolved, and two of the four should matter especially to anyone responsible for quality.
Interpretability degrades as models shrink. Pruning and quantization remove exactly the structure that made a network’s reasoning traceable. For a defect classifier whose calls feed a disposition decision, or a diagnostic model in a regulated environment, a black box that got smaller is a black box that got harder to audit. Explainable-AI techniques adapted for lightweight models — attention visualization, rule extraction, interpretability-aware pruning that preserves semantically important pathways — are active research, not settled practice.
Edge devices are physically exposed. A cloud server sits behind layered institutional security. An edge gateway sits on a machine. The review catalogues model inversion, membership inference and side-channel attacks aimed at deployed lightweight models, plus sensor-fusion attacks in autonomous vehicles where crafted obstacles go undetected. Compressed models, with fewer parameters and simpler structure, are more vulnerable than their full-size counterparts, not less.
The remaining two are large-model deployment — compressing billion-parameter architectures without accuracy collapse, where recent methods like SparseGPT and OmniQuant have made progress but still fight heterogeneous hardware — and energy efficiency, which brings thermal management along with it on battery-powered and continuously-operating devices.
No universal answer
The authors’ most practically useful contribution is a decision framework that refuses to name a winner. Latency-critical systems favor quantization plus hardware-aware NAS. Memory-constrained deployments want sparse models with aggressive quantization. Privacy-sensitive applications lean on distillation and federated approaches. Their stated conclusion is that no single technique optimizes every metric, and that hybrid methods — quantized sparse models, federated distillation — plus dynamic adaptation are where the field should go next.
For anyone specifying an edge inspection or monitoring system, that translates cleanly enough: decide which constraint actually binds before choosing a compression strategy, and budget separately for the interpretability and security work that the compression itself will make harder.