Blob: docs/deferred/face-aware-passport.md
Deferred: face-aware ID/passport framing
Goal
Automatically position and size the face so an identity photo meets a country's
spec: head height within a ratio range and eyes on a given line. Today the app
ships the manual half of this - a head/eye-line guide overlay driven by the
template's framing metadata. The deferred half is detecting the face and
computing the crop for the user.
What already exists
- Template.framing = { faceHeightRatio: [min, max], eyeLineFromTop }.
- The US passport template populates it (faceHeightRatio [0.5, 0.69], eyeLineFromTop ~0.43, output 600x600 = 2x2 inch at 300 DPI).
- EditorCanvas renders a dashed head ellipse and eye line over the crop box for
any template that has
framing.
Recommended approach
Run a face detector in a Web Worker, then solve for the crop:
- Detect the face on the loaded image. Options:
- MediaPipe Face Detector (the tasks-vision package) - small, fast, runs in a worker. Gives a face box and key points (eyes).
- A small ONNX face model via onnxruntime-web if MediaPipe is undesirable.
- From the detected eye positions and chin/crown estimate, compute the
zoom and crop offset so:
- head height / output height falls in faceHeightRatio, and
- the eye midpoint lands on eyeLineFromTop. Convert that target into react-easy-crop's crop ({x,y}) and zoom, or compute croppedAreaPixels directly for the export.
- Offer "Auto-frame" as a button; keep the manual guide for fine-tuning.
Per-country specs
Passport/ID rules differ by country (US, UK, EU/Schengen, India, China, and
others differ on size, head ratio, background, and DPI). Start with one or two
countries. Model each as its own Template row (plus framing), so adding a
country is just data. Keep the numbers sourced and commented.
The hard part
- Same cross-origin isolation constraint as background removal: self-host the detector's model/wasm so it loads under COEP require-corp. See background-removal.md.
- Mapping detector output to react-easy-crop's crop/zoom is fiddly; it may be cleaner to bypass the interactive state and compute croppedAreaPixels for a one-shot "auto-frame to export" while leaving manual adjustment available.
- Real compliance (background color, expression, shadows) is out of scope; framing geometry is the achievable, useful part.
Effort
Medium-to-large, and best paired with background removal (passport photos need both the framing and a plain background).
Acceptance checks
- For a portrait, Auto-frame places eyes on the eye line and head height inside the ratio range; the export is exactly the template size.
- The manual guide still works and matches.
- Adding a second country is only a new template row.