Symmetry 2024, 16(12), 1608; https://doi.org/10.3390/sym16121608 - 3 Dec 2024
Abstract
The text-to-image task, a critical branch of computer vision and image processing, has witnessed remarkable advancements fueled by the abundance of realistic data and rapid AI innovation. However, existing research often overlooks scenarios involving sparse textual input and fails to incorporate human personalized
[...] Read more.
The text-to-image task, a critical branch of computer vision and image processing, has witnessed remarkable advancements fueled by the abundance of realistic data and rapid AI innovation. However, existing research often overlooks scenarios involving sparse textual input and fails to incorporate human personalized preferences into the generative process. To address these gaps, we propose a novel AI methodology: personalized short-text-to-image generation through aesthetic assessment and human insights. Our approach introduces a symmetry between personalized aesthetic preferences and the generated images by leveraging a data-driven personality encoder (PE) to extract personal information and embed it into a Big Five personality trait-based image aesthetic assessment (BFIAA) model. This model harmonizes aesthetic preferences with the generative process by adapting the stable diffusion framework to align with personalized assessments. Experimental results demonstrate the effectiveness of our method: the PE module achieves an accuracy of 98.1%, while the BFIAA model surpasses the baseline by 13% on the PLCC metric, accurately reflecting human aesthetic preferences. Furthermore, our adapted generation model improves convergence loss by over 10% compared to the base model, consistently producing personalized images that are more aligned with human preferences.
Full article
(This article belongs to the Special Issue Model-Driven, Data-Driven and Symmetry Methods in Hyperspectral Image Processing)
►
Show Figures
Figure 1
Figure 1
<p>Generated images within short text easily contain discomfort regions surrounded by white frames, which conflict with human preference.</p> Full article ">Figure 2
<p>Overall framework of our proposed method.</p> Full article ">Figure 3
<p>Comparison of generated images from our method and baseline model in [<a href="#B4-symmetry-16-01608" class="html-bibr">4</a>]. The white frame denotes the discomfort region.</p> Full article ">Figure 4
<p>Image comparison between personality trait O (aesthetically preferred) and personality trait C (aesthetically preferred).</p> Full article ">Figure 5
<p>Generated image comparison between personality trait E (aesthetically preferred) and personality trait A (aesthetically preferred).</p> Full article ">Figure 6
<p>Generated image comparison between personality trait C (aesthetically preferred) and personality trait N (aesthetically preferred).</p> Full article ">Figure 7
<p>Generated image comparison between personality trait A (aesthetically preferred) and personality trait N (aesthetically preferred).</p> Full article ">Figure 8
<p>Generated personalized image comparison between the five personality traits.</p> Full article ">
<p>Generated images within short text easily contain discomfort regions surrounded by white frames, which conflict with human preference.</p> Full article ">Figure 2
<p>Overall framework of our proposed method.</p> Full article ">Figure 3
<p>Comparison of generated images from our method and baseline model in [<a href="#B4-symmetry-16-01608" class="html-bibr">4</a>]. The white frame denotes the discomfort region.</p> Full article ">Figure 4
<p>Image comparison between personality trait O (aesthetically preferred) and personality trait C (aesthetically preferred).</p> Full article ">Figure 5
<p>Generated image comparison between personality trait E (aesthetically preferred) and personality trait A (aesthetically preferred).</p> Full article ">Figure 6
<p>Generated image comparison between personality trait C (aesthetically preferred) and personality trait N (aesthetically preferred).</p> Full article ">Figure 7
<p>Generated image comparison between personality trait A (aesthetically preferred) and personality trait N (aesthetically preferred).</p> Full article ">Figure 8
<p>Generated personalized image comparison between the five personality traits.</p> Full article ">
