Macdrop Net: ((exclusive))

MacDrop identifies "massive weights"—a few highly influential parameters—that tend to dominate during pre-training. Instead of updating all parameters equally, it applies targeted dropout specifically to these massive weights during the fine-tuning process.