PinPoint/docs/architecture.md

Architecture

Technology stack

Layer	Choice
Platform	Android (min SDK 24, target/compile SDK 36)
Languages	Kotlin (feature code), Java (legacy MainActivity entry point), C++ (native game shell)
Build	Gradle 9.x (version catalog), CMake 3.22.1 via Android's externalNativeBuild
Camera capture	CameraX (core, camera2, lifecycle, video, view, effects)
Pose detection	Google ML Kit Pose Detection (accurate model)
Concurrency	Kotlin Coroutines
UI (bowling feature)	Android Views + ViewBinding, custom Views for the pose overlay
UI (menu shell)	Native OpenGL ES 3.0, rendered from C++ via a GLSurfaceView
Rendering (native)	Custom C++ GLRenderer / UIRenderer

High-level component map

The app is really two loosely-coupled subsystems living in one Gradle module (app/), connected by a single JNI call:

flowchart TB
    subgraph Native["Native game shell (C++, JNI, OpenGL ES 3.0)"]
        GSM[GameStateManager]
        States[States: MainMenu / Menu1 / Menu2 / Menu3 / Settings]
        UIR[UIRenderer / GLRenderer]
        PB[PlatformBridge]
        GSM --> States
        GSM --> UIR
        States --> PB
    end

    subgraph AndroidHost["Android host (Java/Kotlin)"]
        MA[MainActivity\nGLSurfaceView host]
    end

    subgraph Bowling["Bowling capture + analysis (Kotlin)"]
        BCA[BowlingCameraActivity]
        CVM[CameraViewModel]
        CXC[CameraXController]
        PA[PoseAnalyzer]
        LSD[LiveStepDetector]
        PPD[PosePhaseDetector]
        PAC[PoseAngleCalculator]
        PLS[PoseLandmarkSmoother /\nAnkleHipMovingAverageFilter]
        POV[PoseOverlayView /\nPoseSkeletonRenderer]
        FUI[FeedbackUI / StepCounterUiController]
        DSL[DebugSessionLogger]
        PEA[ParameterEditorActivity /\nDetectorSettings]
        AA[AdminAuth /\nAdminLoginPrompt]
    end

    MLKit[(ML Kit Pose Detection)]

    PB -- "JNI: launchBowlingCamera()" --> MA
    MA -- "startActivity()" --> BCA
    BCA --> CVM --> CXC
    CXC -- camera frames --> PA
    PA -- landmarks --> MLKit
    MLKit -- pose result --> PA
    PA --> PLS --> LSD
    PA --> PAC
    LSD --> PPD
    PPD --> FUI
    PAC --> FUI
    PA --> POV
    CVM --> DSL
    PEA --> CVM
    AA --> PEA

Native game shell

• GameStateManager (singleton) owns the current GameState and drives Update() / Render() each frame. State transitions are deferred (RequestStateChange) so a state can safely trigger its own replacement mid-frame (e.g. from a button click handled during Render()).
• States (MainMenuState, Menu1State, Menu2State, Menu3State, SettingsState) implement the actual menu screens; Menu3State is the entry point into the bowling feature.
• PlatformBridge is the seam between shared state-machine code and platform-specific "launch a native feature" hooks, so state code doesn't need #ifdefs for Android vs. desktop. On Android, LaunchBowlingCamera() calls back into MainActivity over JNI (caching a JavaVM + global activity ref from initGL()); on the Windows/GLFW build it's a no-op log.
• MainActivity (Java) hosts a GLSurfaceView (OpenGL ES 3.0, RENDERMODE_CONTINUOUSLY, setPreserveEGLContextOnPause(true) so launching BowlingCameraActivity on top doesn't tear down and have to re-init the GL context/UI). Touch events are forwarded to native code (nativeOnTouch) for in-engine hit-testing.
• Cross-platform note: Platform.h already defines both PLATFORM_ANDROID and PLATFORM_WINDOWS (GLFW) branches, and main.cpp has a GLFW desktop loop. Only the Android CMake/Gradle build is currently wired up — there is no standalone desktop build target yet, but the state-machine/rendering code is written to be portable.

Bowling capture & pose analysis pipeline

1. BowlingCameraActivity hosts the camera preview and UI chrome (switch camera, back, recording indicator); orientation follows the device sensor.
2. CameraXController wraps CameraX use cases (preview, video, image analysis) and exposes camera frames.
3. PoseAnalyzer runs each frame through ML Kit's accurate Pose Detection model and produces a PoseFrame.
4. Landmarks are smoothed (PoseLandmarkSmoother, AnkleHipMovingAverageFilter) before being consumed by:
   • LiveStepDetector / StepDetector / StepCountingSession — step counting during the approach.
   • PosePhaseDetector — segments the approach into delivery phases.
   • PoseAngleCalculator — computes joint angles at points of interest.
5. PoseOverlayView + PoseSkeletonRenderer draw the live skeleton over the camera preview.
6. FeedbackUI / StepCounterUiController surface step count, phase, and feedback to the user.
7. CameraViewModel coordinates the above and survives configuration changes; DebugSessionLogger records session data for offline tuning.
8. DetectorSettings (via ParameterEditorActivity, gated by AdminAuth/AdminLoginPrompt) allows adjusting detection thresholds without a rebuild, for tuning during development/testing.

Known architectural gap

BowlingCameraActivity is reachable two ways today: directly (e.g. via adb/launcher shortcut) and from the native menu's Menu3State via PlatformBridge. Per in-code comments, the direct-launch path exists because the feature isn't yet fully wired into the menu's visual flow — this should converge as the menu integration matures.

Build & deployment

• Single Gradle module (app), AGP + CMake (externalNativeBuild) for the native library, targeting arm64-v8a, armeabi-v7a, x86, x86_64.
• No CI/CD pipeline exists yet — see docs/deliverables.md for the CI/CD deliverable and milestone target.
• No backend/server component exists; the app is fully on-device (no network calls in the current codebase).