Accuracy–Latency Frontier

Every latency-measured model below was re-benchmarked under one unified protocol: median CUDA-event latency of the network forward pass — FP16, batch one, RTX 4090, torch.compile + CUDA Graphs, CUDA NMS included for models that need it. Baseline accuracy values follow the sources identified in the paper's full tables. Hover any point for details, switch task, or filter by model scale — GTR variants (green squares) trace the upper-left frontier. Exact numbers are in the results tables.

How GTR Works

GTR keeps the same stride-16 patch grid through all 12 blocks — no hierarchy, no window partitioning. Each block combines key-only gated linear attention along one scan order with a Spatial SwiGLU channel mixer; consecutive blocks cycle through left→right, right→left, top→bottom and bottom→top orders, so context accumulates from complementary directions across depth. Features tapped at blocks 4/8/12 feed a lightweight stride-8/16/32 pyramid and a DETR-style decoder with 300 queries. Four scales — S/M/L/X at 12.1/22.7/37.2/46.5 M detector parameters — share this exact layout.

A Single ℓ₂ Loss for Cross-Architecture Distillation

The teacher is a DINOv3 backbone specialized for detection following the EdgeCrafter recipe, then frozen. Teacher and student both process the same complete image, so their patch tokens correspond spatially. A learned linear projection aligns feature dimensions, and the entire distillation objective is

There is no input masking, no intermediate-layer loss, and no token-affinity target — only the student and the projection are optimized. Prior cross-architecture transfer pipelines bridge different token mixers with several stages or complementary objectives; with the same teacher, student and downstream COCO schedule, the single final-output loss outperforms our adaptations of both:

A Kernel That Makes Recurrence Fast

Chunkwise GLA evaluates the exact recurrence through matrix contractions over chunks of tokens. The reference schedule propagates state while traversing chunks serially; GTR's inference-only operator instead computes all per-chunk summaries in parallel across the chunk–head grid and leaves only a lightweight, coalesced scan for the boundary states. The output kernel fuses the within-chunk score–value product with the boundary-state readout and holds the causal tile in shared memory. Execution is specialized to the deployment regime — FP16, , , zero initial state — while training keeps the reference FLA kernels. Same state update, different execution schedule.

Results

One Backbone Design, Six Dense Tasks

Accuracy Scales with Input Resolution

Linear complexity is only useful if the extra spatial detail is actually exploited. Sweeping the Objects365-pre-trained GTR-S from 640² to 1280², accuracy improves monotonically from 53.6 to 57.0 AP, with the largest gain on small objects (AP_S 36.4 → 42.4) — while median latency grows only from 1.225 to 2.529 ms and peak memory from 0.096 to 0.249 GB. That is 2.06× latency for 4× as many image tokens under the reported benchmark: GTR does not merely tolerate larger inputs; it converts them into accuracy.

What Actually Matters

Local Mixing Beats Positional Embeddings

Spatial SwiGLU replaces learned positional embeddings entirely — and wins: on GTR-S it adds +1.2 / +1.3 AP at 768² / 1024² for only 0.2 / 0.5 extra GFLOPs. The interaction with scanning is where it gets dramatic: with a single scan direction, swapping S-SwiGLU for positional embeddings collapses GTR-L from 54.4 to 45.6 AP (−8.8) — local spatial mixing is what makes causal scans workable on images.

Softmax Layers Don't Pay Their Way

Would sprinkling softmax attention back in help? Replacing three of GTR-L's twelve GLA layers with softmax attention — first three, last three, or uniformly spaced — yields 55.4–55.6 AP versus 55.5 for pure GLA, while median latency rises from 1.908 to 2.094–2.117 ms. At this scale the hybrids buy no reliable accuracy and cost ~10% latency; run-to-run variation is not reported, so the 0.1-AP differences should not be over-read. Independently, Objects365 detector pre-training adds +2.9 / +3.3 / +3.4 / +3.2 AP at S/M/L/X — the headline numbers reflect both the distilled representation and supervised pre-training.

Edge Deployment on DRIVE AGX Thor

GTR deploys in TensorRT on NVIDIA DRIVE AGX Thor with graph transformations and fused CUDA kernels for GLA and S-SwiGLU, built for Thor's sm_110 GPU. Across all six tasks and four scales, batch-one median latency spans 2.282–8.769 ms — detection runs at 2.282–4.080 ms — and FP16 TensorRT outputs keep a cosine similarity of at least 0.9989 with their FP32 PyTorch references across all 48 task–scale–batch configurations.

Our Mission

Our mission is to build powerful, efficient AI models and make them effortless to deploy. 🚀🚀🚀

Citation

If you find GTR useful, please consider citing:

@article{gtr2026feng,
    title   = {{GTR}: Gated Token Recurrence for Efficient Dense Prediction},
    author  = {Feng, Zhe and Liu, Longfei and Liu, Wei and Chen, Kai and Kong, Jiangang and Zhou, Wei and Qian, Yifeng and Chen, Dexiong and Yu, Xuanlong and Shen, Xi},
    journal = {arXiv preprint arXiv:2609.26590},
    year    = {2026}
}