Hand gesture control in web maps is a niche but growing area, especially for kiosk and touchless UI scenarios. The map-gesture-controls repo offers a pragmatic and privacy-focused take: it runs entirely in the browser, uses MediaPipe’s WASM hand landmark detection, and supports major mapping libraries with a clean abstraction.
What map-gesture-controls offers and its architecture
This TypeScript monorepo provides hand gesture control for three major web map libraries: OpenLayers, Google Maps, and Leaflet. At its core, it uses MediaPipe’s Hand Landmarker in WASM form to track 21 3D hand landmarks from the webcam feed directly in the browser, avoiding any backend processing or data sharing.
The architecture is split into a map-agnostic core package (@map-gesture-controls/core) that handles the gesture recognition, smoothing, and state machine logic, while the three integration packages (@map-gesture-controls/ol, @map-gesture-controls/google-maps, and @map-gesture-controls/leaflet) act as thin wrappers that re-export the core and add map-specific interaction classes.
The core pipeline processes the hand landmarks frame-by-frame through a gesture classification state machine. This state machine incorporates configurable “dwell” timers (default 80ms) to confirm gestures and “release grace periods” (default 150ms) to avoid flickering between states. It detects discrete gestures such as fist or pinch to map to pan, zoom, or rotate operations on the map.
Before applying map interactions, the raw hand movement deltas are filtered through a dead-zone filter to ignore small jitters and an exponential smoothing filter to produce smooth, natural-feeling pan and zoom motions.
All this processing occurs on the client side, ensuring user privacy and eliminating latency from server roundtrips.
The gesture classification pipeline and smoothing — technical strengths and tradeoffs
The heart of the repo lies in GestureStateMachine.classifyGesture(), which transforms raw 21-point 3D hand landmark data into discrete map control commands. The use of a state machine with dwell and grace timers is a thoughtful approach to combat noisy input and flickering that commonly plague gesture detection systems.
The state machine waits to confirm a gesture is stable for a minimum dwell time before switching states, and holds onto gestures briefly after release to avoid flickering. This adds a small delay but greatly improves UX stability.
Dead-zone filtering discards small hand movements below a threshold, preventing micro jitters from moving the map unintentionally. Exponential smoothing further dampens abrupt changes in hand movement velocity, yielding smooth pan and zoom transitions that feel responsive but not jumpy.
The separation between the core recognition logic and map-specific adapters means the gesture pipeline does not depend on map internals, making it reusable or extendable to other map libraries or custom renderers.
The tradeoff here is that the smoothing and timing parameters (dwell, grace period, dead-zone thresholds) are opinionated defaults that may not suit every use case, requiring tuning for specific environments or user preferences.
Also, the reliance on webcam and MediaPipe WASM limits usage to modern browsers with WebGL and getUserMedia support, and the gesture set is relatively basic (fist, pinch, idle) which might not cover advanced gestures.
Quick start with map-gesture-controls
This repo provides straightforward npm install commands for each supported map library, making initial integration simple.
OpenLayers:
npm install @map-gesture-controls/ol ol
Google Maps:
npm install @map-gesture-controls/google-maps @googlemaps/js-api-loader
npm install -D @types/google.maps
Leaflet:
npm install @map-gesture-controls/leaflet leaflet
npm install -D @types/leaflet
The README also notes that the repo does not commit built artifacts (dist/), so maintainers need to build before publishing with:
npm run build
Once installed, you can instantiate the appropriate map gesture controller class for your map instance and start receiving pan/zoom/rotate controls driven entirely by hand gestures detected in the browser.
Verdict
map-gesture-controls is a solid open source solution for adding hand gesture interaction to mainstream web maps without sacrificing privacy or adding backend complexity. Its architecture cleanly separates core gesture logic from map-specific bindings, making it easier to maintain and extend.
Its main audience is developers building kiosk or touchless UI apps around maps, or accessibility-focused projects that want to offer an alternative input mode. The gesture set and tuning parameters are relatively minimal and opinionated — you may need to adjust or extend the code for more complex gestures or different UX requirements.
The browser-only approach using MediaPipe WASM is a good tradeoff for privacy and responsiveness but requires modern browser capabilities and webcam access, which might limit where you can deploy this reliably.
Overall, the repo’s code quality is practical and focused on UX stability through smoothing and state machine timers. It’s worth understanding if you want to explore gesture-driven map interactions or build touchless kiosks with common web mapping libraries.
→ GitHub Repo: sanderdesnaijer/map-gesture-controls ⭐ 92 · TypeScript