Power-Law Attention Module: An Adaptive Non-Linear Approach to Feature Map Refinement
Abstract
Attention mechanisms are used in Convolutional Neural Networks to focus on salient features. Unlike conventional attention modules that rely on linear scaling and straightforward activation functions, the proposed Power-Law Attention Module integrates trainable power-law exponents directly into the feature maps transformation as an attention mechanism. In this strategy, the power exponents are predicted and constrained within appropriate, learnable bounds. We then perform a power-law transformation on the intensity values of the respective feature maps. Finally, the gating mechanism adaptively combines the original features with their power-law transformed features, allowing the model to capture complex features. The experimental results on multiple real-world medical imaging datasets demonstrate the effectiveness of the Power-Law Attention Module for medical image analysis.
Authors: Thant Lwin Oo, Praisan Padungweang
Published in: International Conference on Knowledge and Smart Technology (2026)