Abstract
E-commerce fashion faces a significant challenge with high return rates, often attributed to poor fit. This research presents a method to automatically estimate a user's 3D body shape from everyday social media photos to predict clothing preferences. By leveraging the consistency of a person's body shape across multiple poses and camera angles, the system achieves robust shape estimation. The study demonstrates that detailed 3D body silhouettes are strong predictors of clothing choices, outperforming traditional manual labels and offering a path toward highly personalized online shopping experiences.
Introduction
The global e-commerce fashion industry is hindered by a staggering 50% return rate, with over half of these returns cited as "it doesn't fit." Current online shopping experiences rely on coarse numeric sizes and photos of professional models who rarely represent the diverse body shapes of the general public. Because "fit" is subjective and highly dependent on silhouette, there is a critical need to link detailed 3D body shapes to actual clothing preferences. This paper bridges this gap by extracting 3D shape data from unconstrained social media photos to model what people actually choose to wear.
Methodology
Estimating 3D shape from a single 2D image is difficult due to depth ambiguity and clothing occlusions. To overcome this, the researchers developed a robust multi-photo optimization approach. They observed that while a user's pose and camera angle change across different photos, their underlying body shape remains constant.
The system utilizes the SMPL (Skinned Multi-Person Linear model), a statistical model representing the human body through principal components for shape and a skeleton rig for pose. For each photo, the SMPL model is fitted by minimizing an objective function that accounts for 2D joint detections and silhouettes. A "depth search" strategy is employed to resolve the ambiguity between a person's physical size and their distance from the camera. Finally, the system calculates a median shape across a user's photo gallery and jointly optimizes a single shape across all valid poses, making the model resilient to the "noise" of baggy clothing or awkward angles.
Insights into Body Shape and Preference
The researchers introduced the Fashion Takes Shape (FTS) dataset, containing over 18,000 images from 181 female users on Chictopia.com. A key finding was the significance of the second shape parameter ($\beta_2$), which correlates most strongly with human perception of body mass and girth.
Interestingly, the clothing preference model built using automatically estimated 3D shapes performed better than one built using manual human labels. This suggests that 3D models capture fine-grained nuances of shape that simple "average" or "above-average" labels miss. The data confirmed intuitive fashion trends: for example, users with higher $\beta_2$ values were statistically more likely to wear cardigans and jackets, while those with lower values showed a higher preference for shorts and skirts.
Key Results and Future Work
The multi-photo approach significantly improved accuracy, reducing shape estimation error from 1.05 to 0.84 on synthetic data compared to single-view baselines. An SVM trained on the estimated shape parameters achieved 87.17% accuracy in identifying a user's body type group. Furthermore, the shape-conditioned preference model consistently achieved the lowest Negative Log-Likelihood (19.81), proving that body shape is a statistically significant predictor of clothing choice.
While promising, the study currently focuses on female users from a single platform and uses broad clothing categories. Future work will aim to include more diverse demographics and finer-grained clothing styles (e.g., specific dress cuts). Scaling this technology could enable personalized AI fashion assistants that understand the unique relationship between a user's silhouette and their style preferences.
Summary updated