Skip to content
File

Blob: docs/deferred/face-aware-passport.md

Markdown64 lines

Deferred: face-aware ID/passport framing

Goal

Automatically position and size the face so an identity photo meets a country's spec: head height within a ratio range and eyes on a given line. Today the app ships the manual half of this - a head/eye-line guide overlay driven by the template's framing metadata. The deferred half is detecting the face and computing the crop for the user.

What already exists

  • Template.framing = { faceHeightRatio: [min, max], eyeLineFromTop }.
  • The US passport template populates it (faceHeightRatio [0.5, 0.69], eyeLineFromTop ~0.43, output 600x600 = 2x2 inch at 300 DPI).
  • EditorCanvas renders a dashed head ellipse and eye line over the crop box for any template that has framing.

Recommended approach

Run a face detector in a Web Worker, then solve for the crop:

  1. Detect the face on the loaded image. Options:
    • MediaPipe Face Detector (the tasks-vision package) - small, fast, runs in a worker. Gives a face box and key points (eyes).
    • A small ONNX face model via onnxruntime-web if MediaPipe is undesirable.
  2. From the detected eye positions and chin/crown estimate, compute the zoom and crop offset so:
    • head height / output height falls in faceHeightRatio, and
    • the eye midpoint lands on eyeLineFromTop. Convert that target into react-easy-crop's crop ({x,y}) and zoom, or compute croppedAreaPixels directly for the export.
  3. Offer "Auto-frame" as a button; keep the manual guide for fine-tuning.

Per-country specs

Passport/ID rules differ by country (US, UK, EU/Schengen, India, China, and others differ on size, head ratio, background, and DPI). Start with one or two countries. Model each as its own Template row (plus framing), so adding a country is just data. Keep the numbers sourced and commented.

The hard part

  • Same cross-origin isolation constraint as background removal: self-host the detector's model/wasm so it loads under COEP require-corp. See background-removal.md.
  • Mapping detector output to react-easy-crop's crop/zoom is fiddly; it may be cleaner to bypass the interactive state and compute croppedAreaPixels for a one-shot "auto-frame to export" while leaving manual adjustment available.
  • Real compliance (background color, expression, shadows) is out of scope; framing geometry is the achievable, useful part.

Effort

Medium-to-large, and best paired with background removal (passport photos need both the framing and a plain background).

Acceptance checks

  • For a portrait, Auto-frame places eyes on the eye line and head height inside the ratio range; the export is exactly the template size.
  • The manual guide still works and matches.
  • Adding a second country is only a new template row.