The Use of Mobile Devices in Aiding Dietary Assessment and Evaluation

Abstract

There is a growing concern about chronic diseases and other health problems related to diet including obesity and cancer. The need to accurately measure diet (what foods a person consumes) becomes imperative. Dietary intake provides valuable insights for mounting intervention programs for prevention of chronic diseases. Measuring accurate dietary intake is considered to be an open research problem in the nutrition and health fields. In this paper, we describe a novel mobile telephone food record that will provide an accurate account of daily food and nutrient intake. Our approach includes the use of image analysis tools for identification and quantification of food that is consumed at a meal. Images obtained before and after foods are eaten are used to estimate the amount and type of food consumed. The mobile device provides a unique vehicle for collecting dietary information that reduces the burden on respondents that are obtained using more classical approaches for dietary assessment. We describe our approach to image analysis that includes the segmentation of food items, features used to identify foods, a method for automatic portion estimation, and our overall system architecture for collecting the food intake information.

Figures

Fig. 1

Overall system architecture for dietary assessment.

Fig. 2

Ideal food image analysis system.

Fig. 3

Food portion estimation process

Fig. 4

Volume reconstruction of scrambled eggs using our prismatic approximation model. (a) An input food image, (b) feature points, (c) a base plane constructed using the feature points, and (d) the food volume shape for the scrambled eggs.

Fig. 5

User refinement. More accurate estimate is produced by translating and scaling the spherical volume. (a) shows the initial reconstructed sphere for the orange and (b) and (c) show the translated and scaled estimates, respectively. The initial estimated radius is 1.649 inches (original: 1.45) and the final estimate is 1.5 inches.

Fig. 6

Sample results of connect component labeling. (a) A typical image of a meal, (b) food item segmented using a fix threshold (T=127), and (c) additional food item segmented using color information.

Fig. 7

Sample results of active contours. (a) and (b) each contains the original image (upper left), initial contour (upper right), segmented object boundary (lower left), and binary mask (lower right).

Fig. 8

Sample results of normalized cut. (a) and (c) are the original images, (b) and (d) show the segmented object boundary, and (e)–(h) are the extracted objects, respectively.

Fig. 9

Examples of classified food items, each item label is shown for corresponding food mask. (a) All food items are successfully classified using SVM. (b) Some food items are misclassified by SVM, i.e., scrambled eggs is misclassified as margarine.