Shop : Details
Shop
Details
July 2026
In preparation
Alessio Quercia
On Data and Knowledge Transfer Efficiency in Deep Learning
Deep Learning (DL) has advanced rapidly with larger models and datasets, driven by the belief that scale improves performance. However, this overlooks energy efficiency and environmental impact. This work introduces data- and knowledge-efficient training to address scaling inefficiencies in Computer Vision.
We propose a data-efficient SGD variant that biases optimization toward important samples identified after few epochs—no extra overhead unlike SOTA methods. We apply it to super-resolution of low-resolution CT scans (reducing patient radiation/costs), enabling accurate nasal cavity flow simulations.
For knowledge transfer, we develop an alternating training scheme using auxiliary vision datasets to boost the main MDE task via weighted steps, improving performance without full retraining.
Finally, ILoRA (Feature-Integral LoRA) offers compute-/memory-efficient fine-tuning: it uses fixed feature integrals for compression and a single trainable vector for decompression—fewer parameters per layer than SOTA PEFT.
Results show faster training with no performance loss by reducing data reliance, plus gains from auxiliary tasks and foundation model transfer. Combined with hardware advances, these strategies mitigate DL scaling's energy costs for sustainable high-performance AI.
We propose a data-efficient SGD variant that biases optimization toward important samples identified after few epochs—no extra overhead unlike SOTA methods. We apply it to super-resolution of low-resolution CT scans (reducing patient radiation/costs), enabling accurate nasal cavity flow simulations.
For knowledge transfer, we develop an alternating training scheme using auxiliary vision datasets to boost the main MDE task via weighted steps, improving performance without full retraining.
Finally, ILoRA (Feature-Integral LoRA) offers compute-/memory-efficient fine-tuning: it uses fixed feature integrals for compression and a single trainable vector for decompression—fewer parameters per layer than SOTA PEFT.
Results show faster training with no performance loss by reducing data reliance, plus gains from auxiliary tasks and foundation model transfer. Combined with hardware advances, these strategies mitigate DL scaling's energy costs for sustainable high-performance AI.
Keywords: Transfer Learning; Deep Learning; Efficiency
Aachener Informatik-Berichte, Software Engineering
Edited by Prof. Dr. rer. nat. Bernhard Rumpe, Aachen
Volume 66
Export of bibliographic data
Shaker Verlag GmbH
Am Langen Graben 15a
52353 Düren
Germany
Am Langen Graben 15a
52353 Düren
Germany
Mon. - Thurs. 8:00 a.m. to 4:00 p.m.
Fri. 8:00 a.m. to 3:00 p.m.
Fri. 8:00 a.m. to 3:00 p.m.
Contact us. We will be happy to help you.
