In a controlled comparison on one RTX 4090, attention pooling reduced batch-one GPU memory use by 62.0% while maintaining ...
A knowledge-distilled pyramid transformer with reduced attention achieved F1-scores of 90.92%, 97.76% and 94.59% on three 32 ...