feat: USDA FoodData Central per-ingredient nutrition lookup (v0.70.0)

estimateNutrition() (lib/ai/features/estimate-nutrition.ts) is now a hybrid: for each ingredient, estimateGrams() (lib/ingredient-grams.ts) converts its quantity+unit to grams where possible (weight units exactly; volume units -- cup/tbsp/tsp/ml/l/etc -- via a 1ml~=1g water- density approximation, documented as a real simplification but far better than skipping them). Convertible ingredients get looked up in USDA FoodData Central (lib/usda.ts, SR Legacy + Survey (FNDDS) datasets -- the two suited to generic/raw ingredients, not Branded packaged products or narrower Foundation) and summed. Whatever's left (count-based units like "2 cloves", no USDA match, or USDA_API_KEY unset entirely) goes through one AI call asking for just that subset's total contribution, which sums directly with the USDA totals -- no whole-recipe AI estimate to reconcile against. Same exported signature and return shape as before (NutritionEstimate / {perServing: {...}}), so every existing caller (the nutrition route, meal-plan generation) gets more accurate results automatically once USDA_API_KEY is set in Admin -> Settings, with zero behavior change when it isn't. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-22 00:41:12 +02:00
parent 592c86f8d4
commit f0632cce95
12 changed files with 254 additions and 30 deletions
@@ -65,6 +65,10 @@ GITEA_REPO=owner/repo
 # events) to sync issue close/reopen/comments back into Epicure support tickets.
 GITEA_WEBHOOK_SECRET=

+# USDA FoodData Central (optional, free — get a key at https://fdc.nal.usda.gov/api-key-signup)
+# Improves recipe nutrition estimates with real per-ingredient data instead of AI-only guesses.
+USDA_API_KEY=
+
 # Stripe (optional — webhook stub only)
 STRIPE_WEBHOOK_SECRET=

@@ -2,6 +2,11 @@

 All notable changes to Epicure are documented here. This file is mirrored in-app at `/changelog` (and in the admin dashboard) via `apps/web/lib/changelog.ts` — update both together.

+## 0.70.0 — 2026-07-22 09:30
+
+### Added
+- Recipe nutrition estimation now looks up real per-ingredient data from USDA FoodData Central (for ingredients whose quantity/unit converts to grams — e.g. "200g flour", "2 tbsp oil") and only asks AI to estimate whatever's left (count-based units like "2 cloves", or ingredients with no USDA match). Configure `USDA_API_KEY` in Admin → Settings (free, no partnership needed) — unset, it falls back to the previous fully-AI-estimated behavior exactly as before.
+
 ## 0.69.0 — 2026-07-22 09:00

 ### Added
@@ -66,7 +66,7 @@ Status legend: **Exists** (fully working) · **Partial** (works but with a real
 | Photo scan (pantry) | Exists | Vision-model based | `apps/web/app/api/v1/pantry/scan/photo` |
 | Nutrition goals | Exists | | `apps/web/app/api/v1/users/me/nutrition-goals` |
 | Nutrition diary | Exists | Single-day diary view, plus a Trend tab (7/30/90-day calorie line chart + daily macro averages) — the same endpoint switches modes via a `range` query param | `apps/web/app/api/v1/users/me/nutrition-diary`, `apps/web/components/nutrition/nutrition-trend.tsx` |
-| **Barcode/USDA nutrition-facts database** | **Missing** | Barcode scan only IDs the product name, doesn't pull nutrition facts; all nutrition numbers are AI-estimated, never database-sourced | — |
+| USDA nutrition-facts database | Exists | `estimateNutrition` now looks up each ingredient in USDA FoodData Central (SR Legacy + Survey (FNDDS) datasets) after converting its quantity/unit to grams, sums the matches, and only falls back to AI estimation for ingredients that couldn't be converted (count-based units like "2 cloves") or had no USDA match — including every ingredient, transparently, when `USDA_API_KEY` isn't configured. Barcode scan (pantry) still only IDs product names via Open Food Facts, unrelated to this. | `apps/web/lib/usda.ts`, `apps/web/lib/ingredient-grams.ts`, `apps/web/lib/ai/features/estimate-nutrition.ts` |
 | Batch cooking | Exists | Shared prep steps grouped by which dish they serve, per-dish fridge/freezer countdown independent of the whole-batch pantry deduction | `apps/web/components/recipe/batch-cook-*.tsx` |

 ---
@@ -143,7 +143,7 @@ Ranked roughly by likely value:
 1. **Stripe checkout + billing portal** — the webhook consumer is production-ready but nothing produces the checkout event or lets a user self-serve upgrade/cancel/view invoices. Biggest gap if monetizing seriously.
 2. **Recipe video** — no support at all.
 3. ~~Nutrition trend/history view~~ — closed 2026-07-22 (7/30/90-day calorie chart + macro averages).
-4. **USDA/nutrition-database lookup** — all nutrition numbers are AI-estimated; barcode scan only gets product name, not nutrition facts.
+4. ~~USDA/nutrition-database lookup~~ — closed 2026-07-22 (per-ingredient, gram-convertible ingredients only; AI fills the rest).
 5. ~~OS Share Target~~ — closed 2026-07-21 (Chromium/Android only, no Safari equivalent exists).
 6. ~~Push click-through handling~~ — closed 2026-07-21 (`push`/`notificationclick` listeners added to `sw.js`).
 7. ~~Anonymous public link for meal plans~~ — closed 2026-07-21 (read-only, with a QR code).
@@ -23,6 +23,14 @@ const SETTING_GROUPS = [
      "never touches code or repo settings, just creates issues.",
    keys: ["GITEA_URL", "GITEA_TOKEN", "GITEA_REPO"] as const,
  },
+  {
+    title: "USDA FoodData Central",
+    description:
+      "Improves recipe nutrition estimates with real per-ingredient data instead of AI-only guesses, " +
+      "for ingredients whose quantity/unit can be converted to grams. Free — get a key at " +
+      "fdc.nal.usda.gov/api-key-signup. Unset falls back to the previous AI-only estimation for every ingredient.",
+    keys: ["USDA_API_KEY"] as const,
+  },
 ];

 export default async function AdminSettingsPage() {
@@ -26,6 +26,7 @@ const ALLOWED_KEYS: SiteSettingKey[] = [
  "GITEA_TOKEN",
  "GITEA_REPO",
  "GITEA_WEBHOOK_SECRET",
+  "USDA_API_KEY",
 ];

 export async function PUT(req: NextRequest) {
@@ -1,7 +1,12 @@
 import { generateObject } from "ai";
 import { z } from "zod";
 import { resolveModel, type AiConfig } from "../factory";
+import { estimateGrams } from "@/lib/ingredient-grams";
+import { lookupUsdaNutrients, type UsdaNutrientsPer100g } from "@/lib/usda";

+// Only used for its inferred type below (estimateNutrition builds this
+// shape by hand from USDA + AI sub-totals, not via generateObject anymore).
+// eslint-disable-next-line @typescript-eslint/no-unused-vars
 const NutritionSchema = z.object({
  perServing: z.object({
    calories: z.number().describe("Total calories per serving (kcal)"),
@@ -15,6 +20,81 @@ const NutritionSchema = z.object({

 export type NutritionEstimate = z.infer<typeof NutritionSchema>;

+type Totals = { calories: number; proteinG: number; carbsG: number; fatG: number; fiberG: number; sodiumMg: number };
+
+const ZERO_TOTALS: Totals = { calories: 0, proteinG: 0, carbsG: 0, fatG: 0, fiberG: 0, sodiumMg: 0 };
+
+function addTotals(a: Totals, b: Totals): Totals {
+  return {
+    calories: a.calories + b.calories,
+    proteinG: a.proteinG + b.proteinG,
+    carbsG: a.carbsG + b.carbsG,
+    fatG: a.fatG + b.fatG,
+    fiberG: a.fiberG + b.fiberG,
+    sodiumMg: a.sodiumMg + b.sodiumMg,
+  };
+}
+
+function scalePer100g(per100g: UsdaNutrientsPer100g, grams: number): Totals {
+  const factor = grams / 100;
+  return {
+    calories: per100g.calories * factor,
+    proteinG: per100g.proteinG * factor,
+    carbsG: per100g.carbsG * factor,
+    fatG: per100g.fatG * factor,
+    fiberG: per100g.fiberG * factor,
+    sodiumMg: per100g.sodiumMg * factor,
+  };
+}
+
+const IngredientTotalSchema = z.object({
+  calories: z.number().describe("Total calories (kcal) contributed by these ingredients, as used"),
+  proteinG: z.number().describe("Total protein in grams"),
+  carbsG: z.number().describe("Total carbohydrates in grams"),
+  fatG: z.number().describe("Total fat in grams"),
+  fiberG: z.number().describe("Total dietary fiber in grams"),
+  sodiumMg: z.number().describe("Total sodium in milligrams"),
+});
+
+/** AI fallback for whichever ingredients didn't get a USDA match — asked
+ * for the total contribution of just this subset, at the quantities as
+ * used (not per serving), so it composes directly with the USDA-derived
+ * totals via addTotals rather than needing to be reconciled against a
+ * separate whole-recipe estimate. */
+async function estimateIngredientsTotal(
+  ingredients: Array<{ rawName: string; quantity?: string | number | null; unit?: string | null }>,
+  config?: AiConfig
+): Promise<Totals> {
+  const model = resolveModel(config);
+  const ingredientList = ingredients
+    .map((i) => {
+      const parts: string[] = [];
+      if (i.quantity) parts.push(String(i.quantity));
+      if (i.unit) parts.push(i.unit);
+      parts.push(i.rawName);
+      return `- ${parts.join(" ")}`;
+    })
+    .join("\n");
+
+  const { object } = await generateObject({
+    model,
+    schema: IngredientTotalSchema,
+    system:
+      "You are an expert nutritionist with deep knowledge of food composition. Estimate the total nutritional contribution of the given ingredient list at the quantities specified, not per serving. When quantities are not specified, use typical amounts. Provide realistic estimates based on standard nutritional databases.",
+    prompt: `Estimate the total nutrition contributed by these ingredients, as used:\n\n${ingredientList}`,
+  });
+
+  return object;
+}
+
+/**
+ * Per-ingredient USDA FoodData Central lookup (converting quantity+unit to
+ * grams first) summed across matches, with AI estimation as the fallback
+ * for whichever ingredients couldn't be converted to grams or had no
+ * usable USDA match — including every ingredient, transparently, when
+ * USDA_API_KEY isn't configured at all. Divides the combined total by
+ * baseServings for the final per-serving figures callers expect.
+ */
 export async function estimateNutrition(
  recipe: {
    title: string;
@@ -27,33 +107,36 @@ export async function estimateNutrition(
  },
  config?: AiConfig
 ): Promise<NutritionEstimate> {
-  const model = resolveModel(config);
-
-  const ingredientList = recipe.ingredients
-    .map((i) => {
-      const parts: string[] = [];
-      if (i.quantity) parts.push(String(i.quantity));
-      if (i.unit) parts.push(i.unit);
-      parts.push(i.rawName);
-      return `- ${parts.join(" ")}`;
+  const usdaAttempts = await Promise.all(
+    recipe.ingredients.map(async (ingredient) => {
+      const grams = estimateGrams(ingredient.quantity, ingredient.unit);
+      if (grams == null) return { ingredient, totals: null };
+      const per100g = await lookupUsdaNutrients(ingredient.rawName);
+      return { ingredient, totals: per100g ? scalePer100g(per100g, grams) : null };
    })
-    .join("\n");
+  );

-  const { object } = await generateObject({
-    model,
-    schema: NutritionSchema,
-    system:
-      "You are an expert nutritionist with deep knowledge of food composition and nutritional values. Estimate nutrition data accurately based on ingredients, quantities, and typical cooking methods. When quantities are not specified, use typical serving amounts. Provide realistic estimates based on standard nutritional databases.",
-    prompt: `Estimate the nutritional content per serving for the following recipe.
+  const usdaTotal = usdaAttempts.reduce(
+    (sum, attempt) => (attempt.totals ? addTotals(sum, attempt.totals) : sum),
+    ZERO_TOTALS
+  );
+  const remaining = usdaAttempts.filter((a) => !a.totals).map((a) => a.ingredient);

-Recipe: ${recipe.title}
-Servings: ${recipe.baseServings}
+  const remainingTotal = remaining.length > 0
+    ? await estimateIngredientsTotal(remaining, config)
+    : ZERO_TOTALS;

-Ingredients:
-${ingredientList}
+  const total = addTotals(usdaTotal, remainingTotal);
+  const servings = recipe.baseServings > 0 ? recipe.baseServings : 1;

-Calculate nutrition values per single serving (total divided by ${recipe.baseServings} servings).`,
-  });
-
-  return object;
+  return {
+    perServing: {
+      calories: total.calories / servings,
+      proteinG: total.proteinG / servings,
+      carbsG: total.carbsG / servings,
+      fatG: total.fatG / servings,
+      fiberG: total.fiberG / servings,
+      sodiumMg: total.sodiumMg / servings,
+    },
+  };
 }
@@ -1,5 +1,5 @@
 // Mirrors CHANGELOG.md at the repo root — update both together.
-export const APP_VERSION = "0.69.0";
+export const APP_VERSION = "0.70.0";

 export type ChangelogEntry = {
  version: string;
@@ -11,6 +11,13 @@ export type ChangelogEntry = {
 };

 export const CHANGELOG: ChangelogEntry[] = [
+  {
+    version: "0.70.0",
+    date: "2026-07-22 09:30",
+    added: [
+      "Recipe nutrition estimation now looks up real per-ingredient data from USDA FoodData Central (for ingredients whose quantity/unit converts to grams — e.g. \"200g flour\", \"2 tbsp oil\") and only asks AI to estimate whatever's left (count-based units like \"2 cloves\", or ingredients with no USDA match). Configure USDA_API_KEY in Admin → Settings (free, no partnership needed) — unset, it falls back to the previous fully-AI-estimated behavior exactly as before.",
+    ],
+  },
  {
    version: "0.69.0",
    date: "2026-07-22 09:00",
@@ -0,0 +1,40 @@
+const WEIGHT_TO_GRAMS: Record<string, number> = {
+  g: 1, gram: 1, grams: 1,
+  kg: 1000, kilogram: 1000, kilograms: 1000,
+  oz: 28.3495, ounce: 28.3495, ounces: 28.3495,
+  lb: 453.592, lbs: 453.592, pound: 453.592, pounds: 453.592,
+};
+
+// Volume converted via a 1ml ≈ 1g water-density approximation — a real
+// simplification (oil is ~0.92g/ml, honey ~1.4g/ml) but the best available
+// without a per-ingredient density table, and still far better than
+// skipping every cup/tbsp/tsp-measured ingredient's USDA lookup entirely.
+const VOLUME_TO_ML: Record<string, number> = {
+  ml: 1, milliliter: 1, milliliters: 1, millilitre: 1, millilitres: 1,
+  l: 1000, liter: 1000, liters: 1000, litre: 1000, litres: 1000,
+  tsp: 4.92892, teaspoon: 4.92892, teaspoons: 4.92892,
+  tbsp: 14.7868, tablespoon: 14.7868, tablespoons: 14.7868,
+  cup: 236.588, cups: 236.588,
+  "fl oz": 29.5735, "fl. oz": 29.5735, floz: 29.5735,
+  pt: 473.176, pint: 473.176, pints: 473.176,
+  qt: 946.353, quart: 946.353, quarts: 946.353,
+  gal: 3785.41, gallon: 3785.41, gallons: 3785.41,
+};
+
+function normalize(unit: string): string {
+  return unit.trim().toLowerCase().replace(/\.$/, "");
+}
+
+/** Best-effort quantity+unit -> grams, for feeding a USDA per-100g lookup.
+ * Returns null for count-based units ("2 cloves", "1 can") or missing
+ * quantity/unit — those ingredients fall back to AI estimation instead. */
+export function estimateGrams(quantity: string | number | null | undefined, unit: string | null | undefined): number | null {
+  if (quantity == null || !unit) return null;
+  const value = typeof quantity === "number" ? quantity : parseFloat(quantity);
+  if (!isFinite(value) || value <= 0) return null;
+
+  const key = normalize(unit);
+  if (key in WEIGHT_TO_GRAMS) return value * WEIGHT_TO_GRAMS[key]!;
+  if (key in VOLUME_TO_ML) return value * VOLUME_TO_ML[key]!;
+  return null;
+}
@@ -22,7 +22,8 @@ export type SiteSettingKey =
  | "GITEA_URL"
  | "GITEA_TOKEN"
  | "GITEA_REPO"
-  | "GITEA_WEBHOOK_SECRET";
+  | "GITEA_WEBHOOK_SECRET"
+  | "USDA_API_KEY";

 const SECRET_KEYS: SiteSettingKey[] = [
  "OPENAI_API_KEY",
@@ -31,6 +32,7 @@ const SECRET_KEYS: SiteSettingKey[] = [
  "VAPID_PRIVATE_KEY",
  "GITEA_TOKEN",
  "GITEA_WEBHOOK_SECRET",
+  "USDA_API_KEY",
 ];

 export function isSecretKey(key: SiteSettingKey): boolean {
@@ -74,6 +76,7 @@ export async function getAllSiteSettings(): Promise<Record<string, { value: stri
    "GITEA_TOKEN",
    "GITEA_REPO",
    "GITEA_WEBHOOK_SECRET",
+    "USDA_API_KEY",
  ];

  const result: Record<string, { value: string | null; isSecret: boolean; fromDb: boolean }> = {};
@@ -0,0 +1,73 @@
+import { getSiteSetting } from "@/lib/site-settings";
+
+export type UsdaNutrientsPer100g = {
+  calories: number;
+  proteinG: number;
+  carbsG: number;
+  fatG: number;
+  fiberG: number;
+  sodiumMg: number;
+};
+
+// Standard USDA nutrient numbers (stable across FoodData Central datasets,
+// unlike nutrientId which can vary) — see
+// https://fdc.nal.usda.gov/portal-data/external/nutrientsList
+const NUTRIENT_NUMBERS: Record<string, keyof UsdaNutrientsPer100g> = {
+  "208": "calories",
+  "203": "proteinG",
+  "205": "carbsG",
+  "204": "fatG",
+  "291": "fiberG",
+  "307": "sodiumMg",
+};
+
+/** Looks up an ingredient's nutrition facts per 100g via USDA FoodData
+ * Central. Restricted to the SR Legacy and Survey (FNDDS) datasets — the
+ * two best suited to generic/raw recipe ingredients ("flour", "olive
+ * oil"), unlike Branded (packaged products, already covered by the
+ * Open Food Facts barcode lookup in pantry scanning) or Foundation
+ * (narrower coverage). Returns null if unconfigured, no usable match, or
+ * the request fails — callers must treat this as best-effort and fall
+ * back to AI estimation. */
+export async function lookupUsdaNutrients(query: string): Promise<UsdaNutrientsPer100g | null> {
+  const apiKey = await getSiteSetting("USDA_API_KEY");
+  if (!apiKey) return null;
+
+  try {
+    const url = new URL("https://api.nal.usda.gov/fdc/v1/foods/search");
+    url.searchParams.set("api_key", apiKey);
+    url.searchParams.set("query", query);
+    url.searchParams.set("pageSize", "3");
+    url.searchParams.set("dataType", "SR Legacy,Survey (FNDDS)");
+
+    const res = await fetch(url.toString(), { signal: AbortSignal.timeout(8000) });
+    if (!res.ok) return null;
+
+    const data = (await res.json()) as {
+      foods?: { foodNutrients?: { nutrientNumber?: string; value?: number }[] }[];
+    };
+    const food = data.foods?.[0];
+    if (!food?.foodNutrients) return null;
+
+    const result: Partial<UsdaNutrientsPer100g> = {};
+    for (const n of food.foodNutrients) {
+      const key = n.nutrientNumber ? NUTRIENT_NUMBERS[n.nutrientNumber] : undefined;
+      if (key && typeof n.value === "number") result[key] = n.value;
+    }
+    // Energy is the load-bearing field — a "match" with no calorie value
+    // isn't usable, treat it the same as no match.
+    if (result.calories == null) return null;
+
+    return {
+      calories: result.calories,
+      proteinG: result.proteinG ?? 0,
+      carbsG: result.carbsG ?? 0,
+      fatG: result.fatG ?? 0,
+      fiberG: result.fiberG ?? 0,
+      sodiumMg: result.sodiumMg ?? 0,
+    };
+  } catch (err) {
+    console.error("[usda] lookupUsdaNutrients failed", { query, message: err instanceof Error ? err.message : err });
+    return null;
+  }
+}
@@ -1,6 +1,6 @@
 {
  "name": "@epicure/web",
-  "version": "0.69.0",
+  "version": "0.70.0",
  "private": true,
  "scripts": {
    "dev": "next dev",
@@ -1,6 +1,6 @@
 {
  "name": "epicure",
-  "version": "0.69.0",
+  "version": "0.70.0",
  "private": true,
  "scripts": {
    "dev": "pnpm --filter web dev",