new stuff added yes
This commit is contained in:
@@ -0,0 +1,102 @@
|
||||
# Architecture
|
||||
|
||||
## Technology stack
|
||||
|
||||
| Layer | Choice |
|
||||
|---|---|
|
||||
| Platform | Android (min SDK 24, target/compile SDK 36) |
|
||||
| Languages | Kotlin (feature code), Java (legacy `MainActivity` entry point), C++ (native game shell) |
|
||||
| Build | Gradle 9.x (version catalog), CMake 3.22.1 via Android's `externalNativeBuild` |
|
||||
| Camera capture | CameraX (`core`, `camera2`, `lifecycle`, `video`, `view`, `effects`) |
|
||||
| Pose detection | Google ML Kit Pose Detection (accurate model) |
|
||||
| Concurrency | Kotlin Coroutines |
|
||||
| UI (bowling feature) | Android Views + ViewBinding, custom `View`s for the pose overlay |
|
||||
| UI (menu shell) | Native OpenGL ES 3.0, rendered from C++ via a `GLSurfaceView` |
|
||||
| Rendering (native) | Custom C++ `GLRenderer` / `UIRenderer` |
|
||||
|
||||
## High-level component map
|
||||
|
||||
The app is really two loosely-coupled subsystems living in one Gradle module (`app/`), connected by a single JNI call:
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph Native["Native game shell (C++, JNI, OpenGL ES 3.0)"]
|
||||
GSM[GameStateManager]
|
||||
States[States: MainMenu / Menu1 / Menu2 / Menu3 / Settings]
|
||||
UIR[UIRenderer / GLRenderer]
|
||||
PB[PlatformBridge]
|
||||
GSM --> States
|
||||
GSM --> UIR
|
||||
States --> PB
|
||||
end
|
||||
|
||||
subgraph AndroidHost["Android host (Java/Kotlin)"]
|
||||
MA[MainActivity\nGLSurfaceView host]
|
||||
end
|
||||
|
||||
subgraph Bowling["Bowling capture + analysis (Kotlin)"]
|
||||
BCA[BowlingCameraActivity]
|
||||
CVM[CameraViewModel]
|
||||
CXC[CameraXController]
|
||||
PA[PoseAnalyzer]
|
||||
LSD[LiveStepDetector]
|
||||
PPD[PosePhaseDetector]
|
||||
PAC[PoseAngleCalculator]
|
||||
PLS[PoseLandmarkSmoother /\nAnkleHipMovingAverageFilter]
|
||||
POV[PoseOverlayView /\nPoseSkeletonRenderer]
|
||||
FUI[FeedbackUI / StepCounterUiController]
|
||||
DSL[DebugSessionLogger]
|
||||
PEA[ParameterEditorActivity /\nDetectorSettings]
|
||||
AA[AdminAuth /\nAdminLoginPrompt]
|
||||
end
|
||||
|
||||
MLKit[(ML Kit Pose Detection)]
|
||||
|
||||
PB -- "JNI: launchBowlingCamera()" --> MA
|
||||
MA -- "startActivity()" --> BCA
|
||||
BCA --> CVM --> CXC
|
||||
CXC -- camera frames --> PA
|
||||
PA -- landmarks --> MLKit
|
||||
MLKit -- pose result --> PA
|
||||
PA --> PLS --> LSD
|
||||
PA --> PAC
|
||||
LSD --> PPD
|
||||
PPD --> FUI
|
||||
PAC --> FUI
|
||||
PA --> POV
|
||||
CVM --> DSL
|
||||
PEA --> CVM
|
||||
AA --> PEA
|
||||
```
|
||||
|
||||
## Native game shell
|
||||
|
||||
- **`GameStateManager`** (singleton) owns the current `GameState` and drives `Update()` / `Render()` each frame. State transitions are deferred (`RequestStateChange`) so a state can safely trigger its own replacement mid-frame (e.g. from a button click handled during `Render()`).
|
||||
- **States** (`MainMenuState`, `Menu1State`, `Menu2State`, `Menu3State`, `SettingsState`) implement the actual menu screens; `Menu3State` is the entry point into the bowling feature.
|
||||
- **`PlatformBridge`** is the seam between shared state-machine code and platform-specific "launch a native feature" hooks, so state code doesn't need `#ifdef`s for Android vs. desktop. On Android, `LaunchBowlingCamera()` calls back into `MainActivity` over JNI (caching a `JavaVM` + global activity ref from `initGL()`); on the Windows/GLFW build it's a no-op log.
|
||||
- **`MainActivity`** (Java) hosts a `GLSurfaceView` (OpenGL ES 3.0, `RENDERMODE_CONTINUOUSLY`, `setPreserveEGLContextOnPause(true)` so launching `BowlingCameraActivity` on top doesn't tear down and have to re-init the GL context/UI). Touch events are forwarded to native code (`nativeOnTouch`) for in-engine hit-testing.
|
||||
- **Cross-platform note:** `Platform.h` already defines both `PLATFORM_ANDROID` and `PLATFORM_WINDOWS` (GLFW) branches, and `main.cpp` has a GLFW desktop loop. Only the Android CMake/Gradle build is currently wired up — there is no standalone desktop build target yet, but the state-machine/rendering code is written to be portable.
|
||||
|
||||
## Bowling capture & pose analysis pipeline
|
||||
|
||||
1. **`BowlingCameraActivity`** hosts the camera preview and UI chrome (switch camera, back, recording indicator); orientation follows the device sensor.
|
||||
2. **`CameraXController`** wraps CameraX use cases (preview, video, image analysis) and exposes camera frames.
|
||||
3. **`PoseAnalyzer`** runs each frame through ML Kit's accurate Pose Detection model and produces a `PoseFrame`.
|
||||
4. Landmarks are smoothed (**`PoseLandmarkSmoother`**, **`AnkleHipMovingAverageFilter`**) before being consumed by:
|
||||
- **`LiveStepDetector`** / **`StepDetector`** / **`StepCountingSession`** — step counting during the approach.
|
||||
- **`PosePhaseDetector`** — segments the approach into delivery phases.
|
||||
- **`PoseAngleCalculator`** — computes joint angles at points of interest.
|
||||
5. **`PoseOverlayView`** + **`PoseSkeletonRenderer`** draw the live skeleton over the camera preview.
|
||||
6. **`FeedbackUI`** / **`StepCounterUiController`** surface step count, phase, and feedback to the user.
|
||||
7. **`CameraViewModel`** coordinates the above and survives configuration changes; **`DebugSessionLogger`** records session data for offline tuning.
|
||||
8. **`DetectorSettings`** (via **`ParameterEditorActivity`**, gated by **`AdminAuth`**/**`AdminLoginPrompt`**) allows adjusting detection thresholds without a rebuild, for tuning during development/testing.
|
||||
|
||||
## Known architectural gap
|
||||
|
||||
`BowlingCameraActivity` is reachable two ways today: directly (e.g. via `adb`/launcher shortcut) and from the native menu's `Menu3State` via `PlatformBridge`. Per in-code comments, the direct-launch path exists because the feature isn't yet fully wired into the menu's visual flow — this should converge as the menu integration matures.
|
||||
|
||||
## Build & deployment
|
||||
|
||||
- Single Gradle module (`app`), AGP + CMake (`externalNativeBuild`) for the native library, targeting `arm64-v8a`, `armeabi-v7a`, `x86`, `x86_64`.
|
||||
- No CI/CD pipeline exists yet — see `docs/deliverables.md` for the CI/CD deliverable and milestone target.
|
||||
- No backend/server component exists; the app is fully on-device (no network calls in the current codebase).
|
||||
Reference in New Issue
Block a user