The Cycle-consistent Adversarial Semantics-Texture Learning For Image Synthesis In Internet Of Medical Things

With the wide deployment of Internet of Medical Things, a great number of incomplete multi-modal medical data that violate the data integrity are collected in many applications, producing negative impacts on diagnosis for patients. AI-driven medical image synthesis is of great significance in recovering the missing images with complementary information. However, existing methods depend on single-modal schemes devised for wild image synthesis, which neglect information loss caused by high heterogeneity between modalities of medical images in transferring semantics of source domains. Meanwhile, they cannot take into account semanticstexture consistencies in generating medical images, which causes distortions of lesions in synthesizing medical images. To address challenges above, the cycle-consistent adversarial semantics-texture learning (Cycle-STAR) is proposed for medical image synthesis via defining a cycle-consistent adversarial auto-encoder for medical image synthesis. In detail, a hierarchical disentangled cycle-consistent adversarial paradigm is designed to learn the semantics-texture consistencies of medical images. Then, a cycle alignment fusion loss is proposed for the network training, which encourages the alignment of semantics between domains to prevent information loss in transferring semantics of source domain. Finally, extensive experimental results in two typical medical scenarios illustrate that Cycle-STAR achieves the superior performance to thirteen methods.

Recommended citation: Zhang J, Chen Z, Liu Y, et al. The Cycle-consistent Adversarial Semantics-Texture Learning For Image Synthesis In Internet Of Medical Things[J]. Journal of Applied Science and Engineering, 2024, 28(5): 1029-1040.
Download Paper