Fashion Florence: Fine-Tuning Florence-2 for Structured Fashion Attribute Extraction
Researchers have introduced Fashion Florence, a specialized vision-language model designed to extract structured fashion attributes from clothing images. Built by fine-tuning the Florence-2 model with LoRA techniques, it processes single photographs to generate JSON objects containing category, color, material, style, and occasion tags. This structured output is optimized for direct integration into downstream recommendation and retrieval systems. The model was trained on a refined subset of the iMaterialist Fashion dataset, utilizing rule-based label engineering to create a compact schema. Performance evaluations on a held-out test set demonstrate that Fashion Florence achieves 94.6% category accuracy and 63.0% material accuracy, significantly outperforming general-purpose models like GPT-4o-mini and Gemini 2.5 Flash in both accuracy and style tag F1 scores. Notably, it produces valid JSON in 99.8% of cases while operating efficiently on a single GPU with zero marginal inference cost. The model has been deployed as a Hugging Face Space and integrated into Loom, an open-source outfit recommendation system, highlighting its practical applicability in e-commerce and fashion technology sectors.
Wire timeline
Fashion Florence: Fine-Tuning Florence-2 for Structured Fashion Attribute Extraction
Researchers have introduced Fashion Florence, a specialized vision-language model designed to extract structured fashion attributes from clothing images. Built by fine-tuning the Florence-2 model with LoRA techniques, it processes single photographs to generate JSON objects containing category, color, material, style, and occasion tags. This structured output is optimized for direct integration into downstream recommendation and retrieval systems. The model was trained on a refined subset of the iMaterialist Fashion dataset, utilizing rule-based label engineering to create a compact schema. Performance evaluations on a held-out test set demonstrate that Fashion Florence achieves 94.6% category accuracy and 63.0% material accuracy, significantly outperforming general-purpose models like GPT-4o-mini and Gemini 2.5 Flash in both accuracy and style tag F1 scores. Notably, it produces valid JSON in 99.8% of cases while operating efficiently on a single GPU with zero marginal inference cost. The model has been deployed as a Hugging Face Space and integrated into Loom, an open-source outfit recommendation system, highlighting its practical applicability in e-commerce and fashion technology sectors.
cs.AI updates on arXiv.org