unplayable01/PinPoint
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Squashed commit of the following:

Browse Source 
commit f0b9b2af67
Merge: 373d65a bbaa7bb
Author: Gabriel Low <73009315+gabriellow1111@users.noreply.github.com>
Date:   Sat Sep 12 18:17:04 2026 +0800

    Merge branch 'merge-testing'

commit 373d65a9d5
Merge: 28fc652 dd0e6fd
Author: harine <harinesumen@gmail.com>
Date:   Sat Sep 12 17:34:27 2026 +0800

    Merge branch 'master' into Harine

commit 28fc652097
Author: harine <harinesumen@gmail.com>
Date:   Sat Sep 12 17:34:07 2026 +0800

    new stuff added yes

commit bbaa7bb04f
Merge: 1c6ced2 610bfd7
Author: Gabriel Low <73009315+gabriellow1111@users.noreply.github.com>
Date:   Fri Sep 11 11:18:35 2026 +0800

    Merge branch 'Gabriel' into merge-testing

commit 1c6ced2dce
Author: Gabriel Low <73009315+gabriellow1111@users.noreply.github.com>
Date:   Fri Sep 11 11:12:30 2026 +0800

    this good

commit dd0e6fdc39
Author: DefiantWanderer <yongwei1349@gmail.com>
Date:   Fri Sep 11 10:32:34 2026 +0800

    Revert "fixing compatibility issues"

    This reverts commit 564e7d83f8.

commit 564e7d83f8
Author: DefiantWanderer <yongwei1349@gmail.com>
Date:   Fri Sep 11 10:28:40 2026 +0800

    fixing compatibility issues

commit 610bfd7991
Author: Gabriel Low <73009315+gabriellow1111@users.noreply.github.com>
Date:   Fri Sep 11 10:13:25 2026 +0800

    Merged Khalil

commit cfe76da1d3
Merge: 584818b 9fcdc13
Author: Gabriel Low <73009315+gabriellow1111@users.noreply.github.com>
Date:   Fri Sep 11 10:03:51 2026 +0800

    Merge branch 'Khalil' into Gabriel

commit 584818b95a
Author: Gabriel Low <73009315+gabriellow1111@users.noreply.github.com>
Date:   Thu Sep 10 12:47:06 2026 +0800

    added back phase toggle button

commit 17fad430c1
Author: Gabriel Low <73009315+gabriellow1111@users.noreply.github.com>
Date:   Thu Sep 10 12:09:46 2026 +0800

    Cleaned up merge UI, added reset step button

commit ac4366d04d
Author: Gabriel Low <73009315+gabriellow1111@users.noreply.github.com>
Date:   Thu Sep 10 00:31:47 2026 +0800

    Added some UI to recording

commit d601efcf43
Author: Gabriel Low <73009315+gabriellow1111@users.noreply.github.com>
Date:   Thu Sep 10 00:13:06 2026 +0800

    Corrected 5 step approach phases & added placeholders for future logic

commit 7cf217488a
Author: Gabriel Low <73009315+gabriellow1111@users.noreply.github.com>
Date:   Wed Sep 9 22:48:06 2026 +0800

    Cleared warnings

commit 981071ef74
Author: Gabriel Low <73009315+gabriellow1111@users.noreply.github.com>
Date:   Wed Sep 9 20:09:18 2026 +0800

    Fix Merge, cleaned up UI

commit f30bbe2a34
Merge: 2f45115 7518594
Author: Gabriel Low <73009315+gabriellow1111@users.noreply.github.com>
Date:   Tue Sep 8 14:11:21 2026 +0800

    Merge branch 'au-au' into Gabriel

commit 9fcdc13699
Author: Khalil Belabadia <belabakhalil@gmail.com>
Date:   Mon Sep 7 22:16:00 2026 +0800

    Audio feedback update

    Added the following files :

    - AudioCue.kt
    - AudioFeedbackEngine
    - AudioFeedbackSettings

    Edited files :

    - BowlingCameraActivity (added the triggerAudiofeedback function)

commit 7518594440
Merge: e7a9c2b 7af128d
Author: midnight-masala <2401021@sit.singaporetech.edu.sg>
Date:   Mon Sep 7 21:26:18 2026 +0800

    Merge remote-tracking branch 'origin/au-au' into au-au

    # Conflicts:
    #	app/src/main/java/com/example/jnicpp/bowling/BowlingCameraActivity.kt
    #	app/src/main/res/layout-land/activity_bowling_camera.xml
    #	app/src/main/res/layout/activity_bowling_camera.xml

commit e7a9c2b140
Author: midnight-masala <2401021@sit.singaporetech.edu.sg>
Date:   Mon Sep 7 21:22:11 2026 +0800

    Add live per-step form feedback based on joint angles

    Extends pose angle tracking with knee bend (hip-knee-ankle), then uses
    it alongside the existing elbow/shoulder angles in a new
    PoseStageAdvisor to give a short live cue for whichever step of the
    approach is in progress: push-away on step 2, downswing on step 3,
    backswing on step 4, and knee-bend/arm-extension on the final step.
    Thresholds are starting defaults, not measured coaching data, and are
    expected to be retuned against real approach footage.

    Also fixes a layout bug found while testing this live: the new
    feedback text was chained via ConstraintLayout's toBottomOf to the
    step banner above it, so whenever that banner was hidden (GONE) the
    feedback text rendered at its collapsed zero-height position instead
    of staying put, landing on top of the recording indicator. Both now
    sit in a plain vertical LinearLayout, which collapses GONE children
    correctly.

    Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

commit 7af128d575
Author: midnight-masala <2401021@sit.singaporetech.edu.sg>
Date:   Mon Sep 7 20:58:03 2026 +0800

    steps

    identify the steps

commit 2bb7c6ca7b
Author: midnight-masala <2401021@sit.singaporetech.edu.sg>
Date:   Mon Sep 7 20:58:03 2026 +0800

    Add live final-position feedback and fix step detection accuracy

    Shows a live banner as each step of the approach is counted, ending in
    "FINAL POSITION - RELEASE!" once the 5th footfall lands. Also fixes
    LiveStepDetector's peak-prominence check, which compared a candidate
    footfall only to its immediate neighboring frame and silently dropped
    real steps that landed across several closely-spaced frames; it now
    tracks the true rise/fall trough on each side instead. Switches
    PoseAnalyzer to ML Kit's faster base pose model (the accurate model was
    starving the peak detector of frames on unaccelerated hardware), and
    makes LiveStepDetector's stillness-based reset toggleable, currently
    off for easier testing against recorded reference clips.

    Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
jingwen121
2026-09-12 21:43:10 +08:00
parent b907747755
commit c6501d00f2
44 changed files with 1757 additions and 1063 deletions
Download Patch File Download Diff File Expand all files Collapse all files
app/src/main/java/com/example/jnicpp/bowling/CameraViewModel.kt
+44 -27
View File
@@ -17,6 +17,7 @@ import kotlinx.coroutines.flow.asSharedFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.isActive
import kotlinx.coroutines.launch
import kotlin.time.Duration.Companion.seconds
/**
* @brief Holds camera/recording UI state so it survives configuration
@@ -51,7 +52,7 @@ class CameraViewModel(application: Application) : AndroidViewModel(application)
// recordingState is Idle -- the UI disables the toggle otherwise (see
// BowlingCameraActivity#renderRecordingState) since the recording
// pipeline picks its pose mode once at start.
private val _poseEnabled = MutableStateFlow(false)
private val _poseEnabled = MutableStateFlow(value = false)
/** @brief Whether pose detection/overlay is currently enabled. */
val poseEnabled: StateFlow<Boolean> = _poseEnabled.asStateFlow()
@@ -62,6 +63,7 @@ class CameraViewModel(application: Application) : AndroidViewModel(application)
// confidence bar -- see PoseAngleCalculator).
private val _poseAngles = MutableStateFlow<PoseAngles?>(null)
/** @brief Joint angles computed for the most recent analyzed frame, or null if none available. */
@Suppress("unused")
val poseAngles: StateFlow<PoseAngles?> = _poseAngles.asStateFlow()
// Pose-frame buffering and live step counting for the current/most
@@ -73,18 +75,6 @@ class CameraViewModel(application: Application) : AndroidViewModel(application)
/** @brief Time-ordered pose samples buffered for the current/most recent recording session. */
val poseFrames: List<PoseFrame> get() = stepCountingSession.poseFrames
// Extra SMA smoothing for ankle/hip landmarks specifically, on top of
// PoseLandmarkSmoother's per-frame EMA -- see AnkleHipMovingAverageFilter.
// Reset (not replaced) alongside the buffer so a new session's window
// doesn't lerp in from the previous one's last few frames.
private val ankleHipSmoother = AnkleHipMovingAverageFilter()
// Live, incremental step counting for the current recording -- see
// LiveStepDetector. Resets itself mid-recording when the bowler holds
// a stationary "ready" stance again, so one recording can capture
// several practice approaches back to back.
private val liveStepDetector = LiveStepDetector()
// Live delivery-phase classification (starting stance, approach, etc)
private val posePhaseDetector = PosePhaseDetector()
private val _posePhase = MutableStateFlow<BowlingPhase?>(null)
@@ -98,20 +88,20 @@ class CameraViewModel(application: Application) : AndroidViewModel(application)
//This frame's torso/knee/elbow angle readings, or null if pose detection is off.
val poseMetrics: StateFlow<PosePhaseDetector.Metrics?> = _poseMetrics.asStateFlow()
// Steps detected so far in the current attempt (since the last reset,
// whether that reset was a new recording starting or the bowler
// returning to a stationary stance mid-recording -- see
// onPoseFrameUpdated and LiveStepDetector). The UI reads events.size
// as the "Step N" counter. Stays populated after recording stops so
// the last attempt's count remains visible.
private val _stepEvents = MutableStateFlow<List<StepEvent>>(emptyList())
/** @brief Steps detected so far in the current attempt, since the last reset. */
val stepEvents: StateFlow<List<StepEvent>> get() = stepCountingSession.stepEvents
/** @brief Progress toward the hold-to-reset gesture, from 0 to 1. */
val handRaiseProgress: StateFlow<Float> get() = stepCountingSession.handRaiseProgress
private val _permissionsGranted = MutableStateFlow(false)
// Live "how's my form right now" cue for whichever step is currently in
// progress -- see PoseStageAdvisor. Recomputed every frame alongside
// stepEvents so it's always tied to the same step count the UI already
// shows, and cleared on the same resets stepEvents is.
private val _poseStageFeedback = MutableStateFlow<String?>(null)
/** @brief Live form feedback for the current step, or null if there's nothing to say yet. */
val poseStageFeedback: StateFlow<String?> = _poseStageFeedback.asStateFlow()
private val _permissionsGranted = MutableStateFlow(value = false)
/** @brief Whether all required camera/microphone/storage permissions are currently granted. */
@Suppress("unused")
val permissionsGranted: StateFlow<Boolean> = _permissionsGranted.asStateFlow()
// One-shot user-facing error messages (camera unavailable, detector
@@ -158,14 +148,37 @@ class CameraViewModel(application: Application) : AndroidViewModel(application)
*/
fun onPoseFrameUpdated(landmarks: Map<Int, SmoothedLandmark>, angles: PoseAngles) {
_poseAngles.value = angles
val phaseResult = posePhaseDetector.update(landmarks, angles) //for pose detector
// Update pose phase detector with this frame's landmarks and angles (independent of step counter)
val phaseResult = posePhaseDetector.update(landmarks, angles)
_posePhase.value = phaseResult.phase
_poseMetrics.value = phaseResult.metrics
// Check if the current pose matches the Starting Stance (pure posture query)
val isStartingStance = (phaseResult.phase == BowlingPhase.STARTING_STANCE)
|| posePhaseDetector.isStartingStanceValid(phaseResult.metrics)
if (_recordingState.value is RecordingState.Recording) {
stepCountingSession.onFrame(landmarks, angles, System.currentTimeMillis())
stepCountingSession.onFrame(
landmarks = landmarks,
angles = angles,
timestampMs = System.currentTimeMillis(),
isStartingPosition = isStartingStance,
)
val currentStepCount = stepEvents.value.size
_poseStageFeedback.value = PoseStageAdvisor.feedback(
stepNumber = currentStepCount.takeIf { it > 0 },
angles = angles,
)
}
}
/** @brief Manually resets step counting state for the current recording session. */
fun resetStepCounter() {
stepCountingSession.resetStepCounter()
_poseStageFeedback.value = null
}
/**
* @brief Marks a recording as being requested and resets all
* per-session buffering/detection state.
@@ -178,6 +191,11 @@ class CameraViewModel(application: Application) : AndroidViewModel(application)
fun onRecordingStarting() {
_recordingState.value = RecordingState.Starting
stepCountingSession.startNewSession(DetectorSettings.load(getApplication()))
/*poseFrameBuffer.clear()
ankleHipSmoother.reset()
liveStepDetector.reset()
_stepEvents.value = emptyList()
_poseStageFeedback.value = null*/
}
/** @brief Marks a recording as actively writing and starts the elapsed-time timer. */
@@ -187,7 +205,7 @@ class CameraViewModel(application: Application) : AndroidViewModel(application)
var seconds = 0L
while (isActive) {
_recordingState.value = RecordingState.Recording(seconds)
delay(1000)
delay(1.seconds)
seconds++
}
}
@@ -226,7 +244,6 @@ class CameraViewModel(application: Application) : AndroidViewModel(application)
/** @brief Cancels the elapsed-time timer when this ViewModel is destroyed. */
override fun onCleared() {
super.onCleared()
timerJob?.cancel()
}
}