
What I learned building FaceSculpt AI: using Vision Camera for live face detection, validating frames before capture instead of after, measuring distance with a face-to-frame ratio, and writing camera permission strings that pass App Store review.
When I started building FaceSculpt AI, an app that scans a face and measures jawline, symmetry, and posture, I assumed the hard part would be the analysis. It wasn't. The hard part was making sure the photo going into the analysis was worth analysing.
Here's what I learned shipping real-time face detection in React Native to both app stores.
The first instinct is to reach for a standard camera component, capture a photo, and send it off. That works right up until you need to know what's in the frame before the user taps the shutter.
For live detection you need frame-by-frame access at a usable frame rate. In React Native that means react-native-vision-camera with its face detector plugin:
npm install react-native-vision-camera react-native-vision-camera-face-detector
import { useCameraDevice, useCameraPermission } from 'react-native-vision-camera';
import { Camera as FaceCamera, type Face } from 'react-native-vision-camera-face-detector';
const device = useCameraDevice('front');
const { hasPermission, requestPermission } = useCameraPermission();
const [faces, setFaces] = useState<Face[]>([]);
The detector hands you face bounding boxes on every frame. That's the raw material for everything else.
This is the single decision that improved my analysis quality more than anything on the AI side.
Most apps capture whatever is in frame and let the backend deal with it. The backend then returns a confident-sounding result built on a photo where the user was three metres away in bad lighting. Garbage in, confident garbage out.
Instead, model the camera as a state machine with explicit failure states:
type FaceStatus =
| 'NO_FACE'
| 'MULTIPLE_FACES'
| 'TOO_FAR'
| 'TOO_CLOSE'
| 'FACE_NOT_CLEAR'
| 'OK';
Capture is only permitted in OK. Every other state shows the user a specific instruction — "move closer", "only one face please" — instead of a generic error after the fact.
Tagged with
Leave a comment
You don't need depth sensors to know whether someone is too far away. The ratio of the detected face box to the frame size is a reliable proxy:
const MIN_FACE_RATIO = 0.20;
const MAX_FACE_RATIO = 0.35;
const ratio = face.bounds.width / frameWidth;
if (ratio < MIN_FACE_RATIO) setFaceStatus('TOO_FAR');
else if (ratio > MAX_FACE_RATIO) setFaceStatus('TOO_CLOSE');
else setFaceStatus('OK');
Those two numbers took real device testing to settle on. Below 0.20 the face occupies too few pixels for landmarks to be stable. Above 0.35 the frame starts cropping the jaw and hairline — exactly the features you need for proportion measurements. Your ideal window depends on what you're measuring, so tune it on real hardware rather than trusting a value from a blog post (including this one).
If two people are in frame, which one is the user? There's no good answer, so don't guess:
if (faces.length === 0) setFaceStatus('NO_FACE');
else if (faces.length > 1) setFaceStatus('MULTIPLE_FACES');
Picking the largest face seems clever until someone scans themselves with a friend leaning in and gets their friend's results.
Once conditions are met, users need to know the app is working and roughly how long to stay put. A circular progress indicator filling over the capture window solves it, and it doubles as a natural stabilisation period — the frame gets steadier while the ring fills.
Keep the interval handle in a ref and clear it on unmount and on focus loss, or you'll leak timers when users navigate away mid-scan:
const intervalRef = useRef<NodeJS.Timeout | null>(null);
useEffect(() => () => {
if (intervalRef.current) clearInterval(intervalRef.current);
}, []);
Uploading face images demands authentication, and you don't want each screen remembering to add a header. Do it once in an axios interceptor:
const apiClient = axios.create({ baseURL: API_BASE, timeout: 10000 });
apiClient.interceptors.request.use(async (config) => {
const token = await AsyncStorage.getItem('token');
if (token) {
config.headers = { ...config.headers, Authorization: `Bearer ${token}` };
}
return config;
});
Every request is authenticated by default, and adding a new endpoint can't accidentally ship an unauthenticated upload.
Apple reads these. Vague camera descriptions get apps rejected, and reviewers are specifically alert to face data. Write them for a human being who wants to know what happens to their face:
This app uses the camera to capture facial images for AI-based face analysis, such as estimating facial symmetry, jawline structure, and visual appearance metrics. The captured images are processed only for providing personalized analysis results to the user.
Say what you capture, why, and what you do with it. Vague strings like "needs camera access" are a rejection waiting to happen.
Back the description with actual behaviour: no permanent face-image storage on the device, HTTPS only, and images used solely for analysis. If your privacy policy and your code disagree, the code is what matters.
Real-time face detection in React Native is very achievable in 2026 — Vision Camera does the heavy lifting. The engineering that determines whether your app feels trustworthy isn't the detection itself. It's the rules you write around it: what you refuse to capture, what you tell the user when you refuse, and what you do with the image afterwards.
FaceSculpt AI is live on Google Play and the App Store.
Building something similar? Get in touch and tell me about it.