Traditional computer vision and natural language processing models for food analysis excel at categorical identification—answering the question, what is on the plate or in the recipe? As noted by GaYeon Koh et al., standard frameworks remain largely limited to these categorical labels. However, knowing that a dish contains olive oil, chicken, and rice is insufficient for health applications. To calculate caloric intake or macronutrient composition, we must know how much of each ingredient is present.
Nutritional value is fundamentally a function of mass. Without quantitative metrics, dietary assessment tools—ranging from Food Frequency Questionnaires (Guiomar Masip et al.) to automated recipe editors (S. X. Li et al.)—suffer from severe accuracy bottlenecks, forcing users to manually input weights and volumes. Emerging research addresses this by shifting from discrete classification to continuous quantitative estimation across images, text, and sensor data.
Visual Estimation: Mapping Pixels to Mass
How can a machine look at a single image of a mixed dish and infer the physical mass of its components? The core intuition relies on moving from discrete classification heads to continuous regression frameworks that capture spatial, volumetric, and contextual cues.
To ground this problem in real-world physics, Quin Thames et al. introduced the Nutrition5k: Towards Automatic Nutritional Understanding of Generic Food dataset. By using a robotic scanning rig to measure incremental weights and 3D structures during food assembly, the authors enabled models to directly map visual features to absolute macronutrient and caloric values, bypassing the need for manual portion estimation.
[ Food Image ] ───> [ Spatial / Volumetric Features ] ───> [ Continuous Mass Regression ] ───> [ Calories / Macros ]
When absolute weight is difficult to infer from a single 2D perspective, predicting relative ingredient amounts provides a powerful alternative. In the PITA framework, Jiatong Li et al. leveraged cross-modal embeddings and a domain-driven Wasserstein loss to predict the proportions of ingredients in Picture-to-Amount (PITA): Predicting Relative Ingredient Amounts from Food Images. By framing the problem as a distribution matching task rather than independent regressions, the model enforces a global understanding of the dish's composition.
In parallel work, Jiatong Li et al. explored dense and sparse prediction architectures in Deep Cooking: Predicting Relative Food Ingredient Amounts from Images to capture how ingredients are distributed spatially. This spatial awareness allows retrieval systems to eventually match recipes not just by ingredient presence, but by volumetric composition.
Textual Estimation: Numerical Reasoning in Recipe Contexts
The challenge of quantitative estimation extends to textual data. Recipes found online are often unstructured, use inconsistent units, or omit quantities entirely. Traditional NLP models treat numerical values as simple text tokens, failing to capture the underlying mathematical and culinary relationships.
As demonstrated by Devansh Batra et al. through RecipeDB: a resource for exploring recipes, the precision of nutritional profile estimation is directly tied to how accurately ingredient units and quantities can be parsed and standardized. When units are missing, simple heuristics introduce massive errors.
To solve this, Donghee Choi et al. developed KitchenScale, a system detailed in both their conference paper and journal publication. Instead of forcing a language model to directly predict a raw number, KitchenScale decomposes the problem into a multi-stage pipeline:
- Measurement Type Classification: Determining if the ingredient is measured by volume, weight, or count.
- Unit Classification: Standardizing the unit (e.g., grams, cups, pieces).
- Quantity Regression: Utilizing specialized regression techniques (such as DExp) to predict the continuous numerical value based on the surrounding cooking context.
This structured approach allows the model to leverage culinary common sense—for instance, inferring that "two cloves" refers to garlic and mapping that contextually to an approximate weight, rather than treating the number "two" as an isolated text token.
Quantity-Aware Dietary Optimization
Why does this shift from qualitative to quantitative modeling matter? It transforms dietary applications from passive logs into active, optimizing agents.
For example, in recipe recommendation, the NutRec framework by Meng Chen et al. uses a neural network-based amount predictor to construct a "pseudo-recipe" in Eating Healthier: Exploring Nutrition Information for Healthier Recipe Recommendation. By explicitly predicting the mass of ingredients required to meet health guidelines, the system can accurately evaluate a meal's true nutritional profile and recommend healthier real-world alternatives.
Furthermore, this quantitative paradigm is reshaping multi-modal health monitoring. As Chunzhuo Wang et al. pointed out in Detection of Eating Gestures in Older Persons Using IMU Sensors With Multistage Temporal Convolutional Network, simply detecting the occurrence of an eating gesture using wearable sensors does not reveal caloric or nutrient intake. True dietary assessment requires fusing motion data with complementary modalities like computer vision and text parsing. By integrating these systems, future frameworks can transform raw behavioral signals into precise, quantity-aware nutritional insights.
Key Foundational Work
Deep Cooking: Predicting Relative Food Ingredient Amounts from Images (2019) — contributed 4 concept elements to this direction.
Picture-to-Amount (PITA): Predicting Relative Ingredient Amounts from Food Images (2020) — contributed 3 concept elements to this direction.
Eating Healthier: Exploring Nutrition Information for Healthier Recipe Recommendation (2020) — contributed 2 concept elements to this direction.