11 Commits
Author SHA1 Message Date
jingwen121 2260585cfe Merge branch 'master' into jingwen 2026-09-17 10:09:14 +08:00
Khalil Belabadia e89188a34e Merge branch 'master' into auwk2 2026-09-16 20:23:11 +08:00
DefiantWanderer 39eac8e70b Update proposal.md 2026-09-16 17:40:58 +08:00
jingwen121 3c682c1b30 adjusted order 2026-09-14 21:38:10 +08:00
midnight-masala bb6698a028 updated live feedback
correct the user live
- Starting position
- Finishing position
- Second step
2026-09-14 02:01:00 +08:00
midnight-masala 3227d08e2d Fix wrong Gradle 9.6.0 distribution checksum
distributionSha256Sum was a typo/copy-paste error from whoever bumped
the Gradle version on master, not the real checksum for gradle-9.6.0-bin.zip.
Verified the correct value directly against Gradle's own published
checksum at downloads.gradle.org, and it matches what a fresh download
of the distribution actually hashes to. The wrong value was blocking
gradlew from building at all.
2026-09-14 01:24:43 +08:00
jingwen121 f30bf103be Added slide and follow through 2026-09-13 00:33:32 +08:00
jingwen121 c461bcc433 Merge branch 'master' into jingwen 2026-09-12 21:43:28 +08:00
jingwen121 c6501d00f2 Squashed commit of the following:
commit f0b9b2af67
Merge: 373d65a bbaa7bb
Author: Gabriel Low <73009315+gabriellow1111@users.noreply.github.com>
Date:   Sat Sep 12 18:17:04 2026 +0800

    Merge branch 'merge-testing'

commit 373d65a9d5
Merge: 28fc652 dd0e6fd
Author: harine <harinesumen@gmail.com>
Date:   Sat Sep 12 17:34:27 2026 +0800

    Merge branch 'master' into Harine

commit 28fc652097
Author: harine <harinesumen@gmail.com>
Date:   Sat Sep 12 17:34:07 2026 +0800

    new stuff added yes

commit bbaa7bb04f
Merge: 1c6ced2 610bfd7
Author: Gabriel Low <73009315+gabriellow1111@users.noreply.github.com>
Date:   Fri Sep 11 11:18:35 2026 +0800

    Merge branch 'Gabriel' into merge-testing

commit 1c6ced2dce
Author: Gabriel Low <73009315+gabriellow1111@users.noreply.github.com>
Date:   Fri Sep 11 11:12:30 2026 +0800

    this good

commit dd0e6fdc39
Author: DefiantWanderer <yongwei1349@gmail.com>
Date:   Fri Sep 11 10:32:34 2026 +0800

    Revert "fixing compatibility issues"

    This reverts commit 564e7d83f8.

commit 564e7d83f8
Author: DefiantWanderer <yongwei1349@gmail.com>
Date:   Fri Sep 11 10:28:40 2026 +0800

    fixing compatibility issues

commit 610bfd7991
Author: Gabriel Low <73009315+gabriellow1111@users.noreply.github.com>
Date:   Fri Sep 11 10:13:25 2026 +0800

    Merged Khalil

commit cfe76da1d3
Merge: 584818b 9fcdc13
Author: Gabriel Low <73009315+gabriellow1111@users.noreply.github.com>
Date:   Fri Sep 11 10:03:51 2026 +0800

    Merge branch 'Khalil' into Gabriel

commit 584818b95a
Author: Gabriel Low <73009315+gabriellow1111@users.noreply.github.com>
Date:   Thu Sep 10 12:47:06 2026 +0800

    added back phase toggle button

commit 17fad430c1
Author: Gabriel Low <73009315+gabriellow1111@users.noreply.github.com>
Date:   Thu Sep 10 12:09:46 2026 +0800

    Cleaned up merge UI, added reset step button

commit ac4366d04d
Author: Gabriel Low <73009315+gabriellow1111@users.noreply.github.com>
Date:   Thu Sep 10 00:31:47 2026 +0800

    Added some UI to recording

commit d601efcf43
Author: Gabriel Low <73009315+gabriellow1111@users.noreply.github.com>
Date:   Thu Sep 10 00:13:06 2026 +0800

    Corrected 5 step approach phases & added placeholders for future logic

commit 7cf217488a
Author: Gabriel Low <73009315+gabriellow1111@users.noreply.github.com>
Date:   Wed Sep 9 22:48:06 2026 +0800

    Cleared warnings

commit 981071ef74
Author: Gabriel Low <73009315+gabriellow1111@users.noreply.github.com>
Date:   Wed Sep 9 20:09:18 2026 +0800

    Fix Merge, cleaned up UI

commit f30bbe2a34
Merge: 2f45115 7518594
Author: Gabriel Low <73009315+gabriellow1111@users.noreply.github.com>
Date:   Tue Sep 8 14:11:21 2026 +0800

    Merge branch 'au-au' into Gabriel

commit 9fcdc13699
Author: Khalil Belabadia <belabakhalil@gmail.com>
Date:   Mon Sep 7 22:16:00 2026 +0800

    Audio feedback update

    Added the following files :

    - AudioCue.kt
    - AudioFeedbackEngine
    - AudioFeedbackSettings

    Edited files :

    - BowlingCameraActivity (added the triggerAudiofeedback function)

commit 7518594440
Merge: e7a9c2b 7af128d
Author: midnight-masala <2401021@sit.singaporetech.edu.sg>
Date:   Mon Sep 7 21:26:18 2026 +0800

    Merge remote-tracking branch 'origin/au-au' into au-au

    # Conflicts:
    #	app/src/main/java/com/example/jnicpp/bowling/BowlingCameraActivity.kt
    #	app/src/main/res/layout-land/activity_bowling_camera.xml
    #	app/src/main/res/layout/activity_bowling_camera.xml

commit e7a9c2b140
Author: midnight-masala <2401021@sit.singaporetech.edu.sg>
Date:   Mon Sep 7 21:22:11 2026 +0800

    Add live per-step form feedback based on joint angles

    Extends pose angle tracking with knee bend (hip-knee-ankle), then uses
    it alongside the existing elbow/shoulder angles in a new
    PoseStageAdvisor to give a short live cue for whichever step of the
    approach is in progress: push-away on step 2, downswing on step 3,
    backswing on step 4, and knee-bend/arm-extension on the final step.
    Thresholds are starting defaults, not measured coaching data, and are
    expected to be retuned against real approach footage.

    Also fixes a layout bug found while testing this live: the new
    feedback text was chained via ConstraintLayout's toBottomOf to the
    step banner above it, so whenever that banner was hidden (GONE) the
    feedback text rendered at its collapsed zero-height position instead
    of staying put, landing on top of the recording indicator. Both now
    sit in a plain vertical LinearLayout, which collapses GONE children
    correctly.

    Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

commit 7af128d575
Author: midnight-masala <2401021@sit.singaporetech.edu.sg>
Date:   Mon Sep 7 20:58:03 2026 +0800

    steps

    identify the steps

commit 2bb7c6ca7b
Author: midnight-masala <2401021@sit.singaporetech.edu.sg>
Date:   Mon Sep 7 20:58:03 2026 +0800

    Add live final-position feedback and fix step detection accuracy

    Shows a live banner as each step of the approach is counted, ending in
    "FINAL POSITION - RELEASE!" once the 5th footfall lands. Also fixes
    LiveStepDetector's peak-prominence check, which compared a candidate
    footfall only to its immediate neighboring frame and silently dropped
    real steps that landed across several closely-spaced frames; it now
    tracks the true rise/fall trough on each side instead. Switches
    PoseAnalyzer to ML Kit's faster base pose model (the accurate model was
    starving the peak detector of frames on unaccelerated hardware), and
    makes LiveStepDetector's stillness-based reset toggleable, currently
    off for easier testing against recorded reference clips.

    Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-12 21:43:10 +08:00
jingwen121 b907747755 Merge branch 'master' into jingwen 2026-09-08 11:53:55 +08:00
jingwen121 18fc5596ef fixed version 2026-09-08 11:53:31 +08:00
16 changed files with 242 additions and 190 deletions
-37
View File
@@ -1,37 +0,0 @@
# Git LFS tracking for BowlEye
# Binary/large-asset types are stored via LFS and lockable, since binaries
# can't be merged. Run `git lfs lock <file>` before editing one of these.
# See docs/ and the "So you got your VM" onboarding deck for context.
# ML models (pose detection, custom-trained)
*.tflite filter=lfs diff=lfs merge=lfs -text lockable
*.onnx filter=lfs diff=lfs merge=lfs -text lockable
*.pt filter=lfs diff=lfs merge=lfs -text lockable
*.pb filter=lfs diff=lfs merge=lfs -text lockable
# Video / audio (bowling capture footage, test clips)
*.mp4 filter=lfs diff=lfs merge=lfs -text lockable
*.mov filter=lfs diff=lfs merge=lfs -text lockable
*.wav filter=lfs diff=lfs merge=lfs -text lockable
*.mp3 filter=lfs diff=lfs merge=lfs -text lockable
# Images
*.png filter=lfs diff=lfs merge=lfs -text lockable
*.jpg filter=lfs diff=lfs merge=lfs -text lockable
*.jpeg filter=lfs diff=lfs merge=lfs -text lockable
*.webp filter=lfs diff=lfs merge=lfs -text lockable
*.psd filter=lfs diff=lfs merge=lfs -text lockable
# Android build / signing artifacts
*.apk filter=lfs diff=lfs merge=lfs -text lockable
*.aab filter=lfs diff=lfs merge=lfs -text lockable
*.jar filter=lfs diff=lfs merge=lfs -text lockable
*.aar filter=lfs diff=lfs merge=lfs -text lockable
*.so filter=lfs diff=lfs merge=lfs -text lockable
*.keystore filter=lfs diff=lfs merge=lfs -text lockable
*.jks filter=lfs diff=lfs merge=lfs -text lockable
# Misc large/binary
*.zip filter=lfs diff=lfs merge=lfs -text lockable
*.fbx filter=lfs diff=lfs merge=lfs -text lockable
*.blend filter=lfs diff=lfs merge=lfs -text lockable
@@ -89,17 +89,13 @@ class BowlingCameraActivity : AppCompatActivity(), CameraXController.Callback {
// class for FeedbackUI
private lateinit var feedbackUI: FeedbackUI
// The team's 5-step terminology, in order -- deliberately not one entry
// per BowlingPhase, since some phases span two of these (see
// PosePhaseDetector.phaseForStep).
private val stepLabels = listOf(
R.string.pose_phase_starting_stance,
R.string.step_term_half_step,
R.string.step_term_preparation_step,
R.string.pose_phase_approach,
R.string.pose_phase_pushaway,
R.string.pose_phase_back_swing,
R.string.pose_phase_power_step,
R.string.step_term_slide,
R.string.step_term_finishing_position,
R.string.pose_phase_slide_and_release,
)
private var currentPhaseToggleIndex = 0
@@ -141,7 +137,6 @@ class BowlingCameraActivity : AppCompatActivity(), CameraXController.Callback {
stepCounterUi = StepCounterUiController(
cardStepCounter = binding.cardStepCounter,
textStepCountBig = binding.textStepCountBig,
textStepCounterLabel = binding.textStepCounterLabel,
)
feedbackUI = FeedbackUI(binding.root)
audioFeedbackSettings = AudioFeedbackSettings(applicationContext)
@@ -264,7 +259,7 @@ class BowlingCameraActivity : AppCompatActivity(), CameraXController.Callback {
}
launch {
viewModel.stepEvents.collect { events ->
stepCounterUi.renderStepCount(events.size, events.lastOrNull()?.poseConfirmed ?: true)
stepCounterUi.renderStepCount(events.size)
if (events.isNotEmpty()) {
Log.d(TAG, "Step ${events.size}: ${events.last()}")
}
@@ -312,10 +307,17 @@ class BowlingCameraActivity : AppCompatActivity(), CameraXController.Callback {
// Combined with poseEnabled (rather than posePhase alone) so
// the label can tell "pose off" (hidden) apart from "pose on
// but not yet in the target posture" (amber prompt) -- both
// cases otherwise report a null phase.
// cases otherwise report a null phase. poseCorrection rides
// along the same collector (rather than its own, like
// poseMetrics below) since it directly changes what
// renderPosePhase puts in the label -- see that function.
launch {
combine(viewModel.poseEnabled, viewModel.posePhase) { enabled, phase -> enabled to phase }
.collect { (enabled, phase) -> renderPosePhase(enabled, phase) }
combine(
viewModel.poseEnabled,
viewModel.posePhase,
viewModel.poseCorrection,
) { enabled, phase, correction -> Triple(enabled, phase, correction) }
.collect { (enabled, phase, correction) -> renderPosePhase(enabled, phase, correction) }
}
// Raw angle readout backing the label above -- its own
// collector since it's driven by a separate StateFlow
@@ -400,15 +402,24 @@ class BowlingCameraActivity : AppCompatActivity(), CameraXController.Callback {
*
* @param poseEnabled Whether pose detection is currently on at all.
* @param phase The bowler's current delivery phase, or null if none currently validates.
* @param correction A specific "here's what to fix" instruction from
* [PosePhaseDetector] when [phase] is null and the bowler is
* being measured against one of the stationary phases (starting
* stance, pushaway, slide & release) -- shown in place of the
* generic "waiting" message when present, so the bowler knows
* exactly what to adjust instead of just that they're not there yet.
*/
private fun renderPosePhase(poseEnabled: Boolean, phase: BowlingPhase?) {
private fun renderPosePhase(poseEnabled: Boolean, phase: BowlingPhase?, correction: String?) {
if (!poseEnabled) {
binding.textPoseFeedback.visibility = View.GONE
return
}
binding.textPoseFeedback.visibility = View.VISIBLE
// Every other BowlingPhase falls back to the "waiting" message too --
// see PosePhaseDetector's class doc, only STARTING_STANCE is detected today.
// Every confirmed phase gets its own label/color below; an
// unconfirmed one falls back to a specific correction when
// PosePhaseDetector has one (see its class doc), otherwise the
// generic "waiting" message -- e.g. mid-approach, where per-frame
// correction isn't meaningful.
when (phase) {
BowlingPhase.STARTING_STANCE -> {
binding.textPoseFeedback.text = getString(R.string.pose_phase_starting_stance)
@@ -434,8 +445,12 @@ class BowlingCameraActivity : AppCompatActivity(), CameraXController.Callback {
binding.textPoseFeedback.text = getString(R.string.pose_phase_slide_and_release)
binding.textPoseFeedback.setBackgroundColor(ContextCompat.getColor(this, R.color.Slide_and_release_ready))
}
BowlingPhase.FOLLOW_THROUGH -> {
binding.textPoseFeedback.text = getString(R.string.pose_phase_follow_through)
binding.textPoseFeedback.setBackgroundColor(ContextCompat.getColor(this, R.color.Follow_through_ready))
}
else -> {
binding.textPoseFeedback.text = getString(R.string.pose_phase_waiting)
binding.textPoseFeedback.text = correction ?: getString(R.string.pose_phase_waiting)
binding.textPoseFeedback.setBackgroundColor(ContextCompat.getColor(this, R.color.Starting_stance_waiting))
}
}
@@ -88,6 +88,13 @@ class CameraViewModel(application: Application) : AndroidViewModel(application)
//This frame's torso/knee/elbow angle readings, or null if pose detection is off.
val poseMetrics: StateFlow<PosePhaseDetector.Metrics?> = _poseMetrics.asStateFlow()
// Specific "here's what to fix" instruction while holding one of the
// stationary phases (starting stance, pushaway, slide & release) -- see
// PosePhaseDetector.correctionFor's doc for why only those three.
private val _poseCorrection = MutableStateFlow<String?>(null)
/** @brief Live corrective instruction for the current stationary phase, or null if nothing to correct. */
val poseCorrection: StateFlow<String?> = _poseCorrection.asStateFlow()
/** @brief Steps detected so far in the current attempt, since the last reset. */
val stepEvents: StateFlow<List<StepEvent>> get() = stepCountingSession.stepEvents
@@ -132,6 +139,7 @@ class CameraViewModel(application: Application) : AndroidViewModel(application)
posePhaseDetector.reset()
_posePhase.value = null
_poseMetrics.value = null
_poseCorrection.value = null
}
}
@@ -153,6 +161,7 @@ class CameraViewModel(application: Application) : AndroidViewModel(application)
val phaseResult = posePhaseDetector.update(landmarks, angles)
_posePhase.value = phaseResult.phase
_poseMetrics.value = phaseResult.metrics
_poseCorrection.value = phaseResult.correction
// Check if the current pose matches the Starting Stance (pure posture query)
val isStartingStance = (phaseResult.phase == BowlingPhase.STARTING_STANCE)
@@ -164,7 +173,6 @@ class CameraViewModel(application: Application) : AndroidViewModel(application)
angles = angles,
timestampMs = System.currentTimeMillis(),
isStartingPosition = isStartingStance,
currentPhase = phaseResult.phase,
)
val currentStepCount = stepEvents.value.size
_poseStageFeedback.value = PoseStageAdvisor.feedback(
@@ -399,16 +399,15 @@ class CameraXController(
}
private fun drawPhaseBadgeOverlay(canvas: Canvas, phase: BowlingPhase, scale: Float) {
val label = appContext.getString(
when (phase) {
BowlingPhase.STARTING_STANCE -> R.string.pose_phase_starting_stance
BowlingPhase.APPROACH -> R.string.pose_phase_approach
BowlingPhase.PUSHAWAY -> R.string.pose_phase_pushaway
BowlingPhase.BACK_SWING -> R.string.pose_phase_back_swing
BowlingPhase.POWER_STEP -> R.string.pose_phase_power_step
BowlingPhase.SLIDE_AND_RELEASE -> R.string.pose_phase_slide_and_release
}
)
val label = when (phase) {
BowlingPhase.STARTING_STANCE -> "Starting Stance"
BowlingPhase.APPROACH -> "Approach"
BowlingPhase.PUSHAWAY -> "Pushaway"
BowlingPhase.BACK_SWING -> "Backswing"
BowlingPhase.POWER_STEP -> "Power Step"
BowlingPhase.SLIDE_AND_RELEASE -> "Slide & Release"
BowlingPhase.FOLLOW_THROUGH -> "Follow Through"
}
val colorRes = when (phase) {
BowlingPhase.STARTING_STANCE -> R.color.Starting_stance_ready
BowlingPhase.APPROACH -> R.color.Approach_ready
@@ -416,6 +415,7 @@ class CameraXController(
BowlingPhase.BACK_SWING -> R.color.Back_swing_ready
BowlingPhase.POWER_STEP -> R.color.Power_step_ready
BowlingPhase.SLIDE_AND_RELEASE -> R.color.Slide_and_release_ready
BowlingPhase.FOLLOW_THROUGH -> R.color.Follow_through_ready
}
val textPaint = overlayTextPaint.apply { textSize = 14f * scale }
val textWidth = textPaint.measureText(label)
@@ -124,8 +124,16 @@ class PosePhaseDetector(
* @brief One [update] call's outcome: the classified phase plus the raw angles it was based on.
* @param phase See [update]'s return doc.
* @param metrics This frame's raw angle readings, for display regardless of whether [phase] validated.
* @param correction A specific corrective instruction (e.g. "Straighten
* your legs") when the bowler is being measured against one of
* [STATIONARY_PHASES] but doesn't currently match it, or null
* when there's nothing to correct -- either because the current
* posture already validates, the target phase is one of the
* moving phases this detector doesn't give live corrections for
* (see [correctionFor]'s doc), or a needed angle wasn't
* confidently read this frame.
*/
data class Result(val phase: BowlingPhase?, val metrics: Metrics)
data class Result(val phase: BowlingPhase?, val metrics: Metrics, val correction: String? = null)
/**
* @brief Feeds one frame's landmarks/angles into the detector.
@@ -160,7 +168,11 @@ class PosePhaseDetector(
BowlingPhase.PUSHAWAY -> BowlingPhase.BACK_SWING
BowlingPhase.BACK_SWING -> BowlingPhase.POWER_STEP
BowlingPhase.POWER_STEP -> BowlingPhase.SLIDE_AND_RELEASE
else -> currentPhase
// The only BowlingPhase not already matched above -- every
// other case (including null) is explicit, so reaching here
// means currentPhase is SLIDE_AND_RELEASE, there's no phase
// after it to advance to.
else -> BowlingPhase.SLIDE_AND_RELEASE
}
}
@@ -172,7 +184,6 @@ class PosePhaseDetector(
BowlingPhase.BACK_SWING -> isBackSwingValid(metrics)
BowlingPhase.POWER_STEP -> isPowerStepValid(metrics)
BowlingPhase.SLIDE_AND_RELEASE -> isSlideAndReleaseValid(metrics)
else -> false
}
if (isTargetValid) {
@@ -210,7 +221,13 @@ class PosePhaseDetector(
consecutiveInvalidFrames = 0
}
return Result(currentPhase, metrics)
val correction = if (!isTargetValid && targetPhase in STATIONARY_PHASES) {
correctionFor(targetPhase, metrics)
} else {
null
}
return Result(currentPhase, metrics, correction)
}
/** @brief Clears all detection state. Call at the start of a new session/attempt. */
@@ -297,46 +314,145 @@ class PosePhaseDetector(
@Suppress("UNUSED_PARAMETER")
private fun isPowerStepValid(metrics: Metrics): Boolean = true
/** @brief Placeholder validation for Slide & Release phase (Step 5). */
@Suppress("UNUSED_PARAMETER")
private fun isSlideAndReleaseValid(metrics: Metrics): Boolean = true
/**
* @brief Checks whether this single frame's [Metrics] match the slide &
* release phase (finishing position, step 5).
*
* Slide & release is characterized by:
* - Torso Tilt: 15-45 degrees (the deepest forward lean of any phase --
* the bowler is bent into the slide)
* - Knee Angle: at least one knee bent to 90-150 degrees (the
* sliding/front leg lowers the body through the release; requiring
* only one, not both, since we don't know which leg is forward)
* - Elbow Angle: at least one elbow extended to 150-180 degrees (the
* swing arm straightens through the release -- same release cue
* [PoseStageAdvisor] already uses for its own final-step check)
*
* @param metrics This frame's raw angle readings.
* @return true if torso tilt, at least one bent knee, and at least one extended elbow all fall within range.
*/
private fun isSlideAndReleaseValid(metrics: Metrics): Boolean {
val torsoTilt = metrics.torsoTiltDegrees ?: return false
if (torsoTilt !in 15f..45f) return false
val kneeAngles = listOfNotNull(metrics.leftKneeAngleDegrees, metrics.rightKneeAngleDegrees)
if (kneeAngles.isEmpty() || kneeAngles.none { it in 90f..150f }) return false
val elbowAngles = listOfNotNull(metrics.leftElbowAngleDegrees, metrics.rightElbowAngleDegrees)
return elbowAngles.isNotEmpty() && elbowAngles.any { it in 150f..180f }
}
/**
* @brief Produces a specific corrective instruction for why [metrics]
* doesn't currently match [targetPhase].
*
* Only covers [STATIONARY_PHASES] -- the three phases a bowler actually
* holds still in long enough for frame-by-frame angle feedback to be
* meaningful. The remaining phases (approach, backswing, power step) are
* mid-motion by nature, so a per-frame "here's what's wrong" cue would
* either be stale by the time it's read or just describe normal
* transitional movement as an error; those are left to [update]'s
* existing pass/fail phase label instead.
*
* Checks each phase's conditions in the same order as its `isXValid`
* counterpart and returns on the first one that fails, so the bowler
* gets one actionable instruction at a time rather than a list.
*
* @param targetPhase Which stationary phase to check [metrics] against. Must be one of [STATIONARY_PHASES].
* @param metrics This frame's raw angle readings.
* @return A short corrective instruction, or null if [metrics] already
* validates for [targetPhase] (nothing to correct) or a needed
* angle wasn't confidently read this frame (nothing useful to say yet).
*/
private fun correctionFor(targetPhase: BowlingPhase, metrics: Metrics): String? = when (targetPhase) {
BowlingPhase.STARTING_STANCE -> {
val torsoTilt = metrics.torsoTiltDegrees
val kneeAngles = listOfNotNull(metrics.leftKneeAngleDegrees, metrics.rightKneeAngleDegrees)
val elbowAngles = listOfNotNull(metrics.leftElbowAngleDegrees, metrics.rightElbowAngleDegrees)
when {
torsoTilt == null -> null
torsoTilt > torsoTiltMaxDegrees -> "Stand up straighter"
torsoTilt < torsoTiltMinDegrees -> "Relax your stance slightly"
kneeAngles.isEmpty() -> null
kneeAngles.any { it < kneeAngleMinDegrees } -> "Straighten your legs"
elbowAngles.isEmpty() -> null
elbowAngles.any { it > elbowAngleMaxDegrees } -> "Bring the ball in closer to your body"
elbowAngles.any { it < elbowAngleMinDegrees } -> "Relax your arms a little"
else -> null
}
}
BowlingPhase.PUSHAWAY -> {
val torsoTilt = metrics.torsoTiltDegrees
val kneeAngles = listOfNotNull(metrics.leftKneeAngleDegrees, metrics.rightKneeAngleDegrees)
val elbowAngles = listOfNotNull(metrics.leftElbowAngleDegrees, metrics.rightElbowAngleDegrees)
when {
torsoTilt == null -> null
torsoTilt < 5f -> "Lean forward slightly as you push away"
torsoTilt > 25f -> "Don't lean too far forward yet"
kneeAngles.isEmpty() -> null
kneeAngles.any { it < 145f } -> "Keep your legs mostly straight here"
elbowAngles.isEmpty() -> null
elbowAngles.none { it in 130f..180f } -> "Push the ball further out"
else -> null
}
}
BowlingPhase.SLIDE_AND_RELEASE -> {
val torsoTilt = metrics.torsoTiltDegrees
val kneeAngles = listOfNotNull(metrics.leftKneeAngleDegrees, metrics.rightKneeAngleDegrees)
val elbowAngles = listOfNotNull(metrics.leftElbowAngleDegrees, metrics.rightElbowAngleDegrees)
when {
torsoTilt == null -> null
torsoTilt < 15f -> "Bend forward more into the slide"
torsoTilt > 45f -> "Don't lean in too far"
kneeAngles.isEmpty() -> null
kneeAngles.none { it in 90f..150f } -> "Bend your sliding knee more"
elbowAngles.isEmpty() -> null
elbowAngles.none { it in 150f..180f } -> "Extend your swing arm fully"
else -> null
}
}
else -> null
}
companion object {
/**
* @brief Maps a 5-step approach step count (0..5) to the [BowlingPhase]
* the bowler should be in once that step has landed.
* @brief The phases a bowler holds still in long enough for
* frame-by-frame angle correction to be meaningful -- see
* [correctionFor]'s doc for why the rest are excluded.
*/
private val STATIONARY_PHASES = setOf(
BowlingPhase.STARTING_STANCE,
BowlingPhase.PUSHAWAY,
BowlingPhase.SLIDE_AND_RELEASE,
)
/**
* @brief Maps a 5-step approach step count (0..5) to its corresponding [BowlingPhase].
*
* Follows the team's 5-step terminology: starting position -> 1/2 step
* -> preparation step -> push away & backswing -> power step -> slide
* -> finishing position. The 1/2 and preparation steps are both
* [BowlingPhase.APPROACH] (ball still held, CG moving forward); the
* ball is pushed away into the swing on step 3, reaches the peak of
* the backswing as the very short power step lands on step 4, and is
* released during the slide on step 5.
*
* step 0 -> STARTING_STANCE (starting position)
* steps 1-2 -> APPROACH (1/2 step, preparation step)
* step 3 -> PUSHAWAY (push away & backswing; BACK_SWING shares this step)
* step 4 -> POWER_STEP
* step 5+ -> SLIDE_AND_RELEASE (slide, finishing position)
* step 0 -> STARTING_STANCE
* step 1 -> APPROACH
* step 2 -> PUSHAWAY
* step 3 -> BACK_SWING
* step 4 -> POWER_STEP
* step 5 -> SLIDE_AND_RELEASE
*/
fun phaseForStep(stepCount: Int): BowlingPhase = when {
stepCount <= 0 -> BowlingPhase.STARTING_STANCE
stepCount <= 2 -> BowlingPhase.APPROACH
stepCount == 3 -> BowlingPhase.PUSHAWAY
stepCount == 1 -> BowlingPhase.APPROACH
stepCount == 2 -> BowlingPhase.PUSHAWAY
stepCount == 3 -> BowlingPhase.BACK_SWING
stepCount == 4 -> BowlingPhase.POWER_STEP
else -> BowlingPhase.SLIDE_AND_RELEASE
}
/**
* @brief The first step of a 5-step approach at which [phase] begins
* -- see [phaseForStep] for the full mapping. APPROACH spans
* steps 1-2, and PUSHAWAY/BACK_SWING both begin on step 3.
* @brief Maps a [BowlingPhase] to its corresponding 5-step approach step count.
*/
@Suppress("unused")
fun stepForPhase(phase: BowlingPhase): Int = when (phase) {
BowlingPhase.STARTING_STANCE -> 0
BowlingPhase.APPROACH -> 1
BowlingPhase.PUSHAWAY -> 3
BowlingPhase.PUSHAWAY -> 2
BowlingPhase.BACK_SWING -> 3
BowlingPhase.POWER_STEP -> 4
BowlingPhase.SLIDE_AND_RELEASE -> 5
@@ -31,24 +31,19 @@ package com.example.jnicpp.bowling
*/
object PoseStageAdvisor {
// Step names follow the team's 5-step terminology (see
// PosePhaseDetector.phaseForStep): 1/2 step, preparation step, push away
// & backswing, power step, slide -> finishing position.
// Step-2 cue: ball still close to the body just after push-away, so the
// swing-arm shoulder angle (elbow-shoulder-hip) should still be small.
private const val PUSH_AWAY_MAX_SHOULDER_DEG = 30f
// Preparation-step cue: the ball is still held close to the body before
// the push away, so the swing-arm shoulder angle (elbow-shoulder-hip)
// should still be small.
private const val PREPARATION_MAX_SHOULDER_DEG = 30f
// Step-3 cue: arm swinging down and back past the body.
private const val DOWNSWING_MIN_SHOULDER_DEG = 25f
private const val DOWNSWING_MAX_SHOULDER_DEG = 75f
// Push away & backswing cue: ball pushed out and swinging past the body.
private const val PUSH_AWAY_MIN_SHOULDER_DEG = 25f
private const val PUSH_AWAY_MAX_SHOULDER_DEG = 75f
// Step-4 cue: arm swinging well back behind the body.
private const val BACKSWING_MIN_SHOULDER_DEG = 60f
// Power-step cue: ball at the peak of the backswing, well behind the body.
private const val POWER_STEP_MIN_SHOULDER_DEG = 60f
// Slide / finishing-position cues: front knee bent to lower the slide,
// swing arm relatively straight through the release.
// Final-position cues: front knee bent to lower the slide, swing arm
// relatively straight through the release.
private const val RELEASE_MAX_KNEE_DEG = 140f
private const val RELEASE_MIN_ELBOW_DEG = 150f
@@ -66,35 +61,25 @@ object PoseStageAdvisor {
val frontKnee = smallerOf(angles.leftKnee, angles.rightKnee)
return when {
(stepNumber == null || stepNumber <= 0) -> "Starting position - stay relaxed"
stepNumber == 1 -> "½ step - small step, weight shifting forward"
(stepNumber == null || stepNumber <= 1) -> "Starting position - stay relaxed"
stepNumber == 2 -> swingShoulder?.let {
if (it <= PREPARATION_MAX_SHOULDER_DEG) {
"Good preparation step"
} else {
"Keep the ball close until the push away"
}
if (it <= PUSH_AWAY_MAX_SHOULDER_DEG) "Good push-away" else "Push the ball out first"
}
stepNumber == 3 -> swingShoulder?.let {
if (it in PUSH_AWAY_MIN_SHOULDER_DEG..PUSH_AWAY_MAX_SHOULDER_DEG) {
"Good push away & backswing"
if (it in DOWNSWING_MIN_SHOULDER_DEG..DOWNSWING_MAX_SHOULDER_DEG) {
"Good downswing"
} else {
"Push the ball up and let it swing"
"Let the arm swing naturally"
}
}
stepNumber == 4 -> swingShoulder?.let {
if (it >= POWER_STEP_MIN_SHOULDER_DEG) {
"Good power step - ball at the top"
} else {
"Let the ball swing higher on the power step"
}
if (it >= BACKSWING_MIN_SHOULDER_DEG) "Good backswing" else "Swing the arm further back"
}
else -> { // slide & finishing position (5+)
else -> { // final step (5+)
val kneeGood = frontKnee != null && frontKnee <= RELEASE_MAX_KNEE_DEG
val armGood = swingElbow != null && swingElbow >= RELEASE_MIN_ELBOW_DEG
when {
@@ -6,22 +6,16 @@ package com.example.jnicpp.bowling
import android.view.View
import android.widget.TextView
import androidx.core.content.ContextCompat
import com.example.jnicpp.R
/**
* @brief Owns rendering for [BowlingCameraActivity]'s step-counter card.
*
* @param cardStepCounter The step-counter card container view.
* @param textStepCountBig The large step-count number TextView.
* @param textStepCounterLabel The small "STEPS" label below the count,
* repurposed to flag when the latest step wasn't corroborated by
* [PosePhaseDetector] -- see [renderStepCount]'s `latestPoseConfirmed`.
*/
class StepCounterUiController(
private val cardStepCounter: View,
private val textStepCountBig: TextView,
private val textStepCounterLabel: TextView,
) {
// Last step count rendered, so pulse() in renderStepCount only plays
// when a new step actually pushed the count up.
@@ -42,34 +36,14 @@ class StepCounterUiController(
/**
* @brief Renders the current step count, pulsing the card if a new step was just confirmed.
*
* Never withholds or delays a step because of [latestPoseConfirmed] --
* the ankle-peak count is always trusted (see [StepEvent.poseConfirmed]'s
* doc for why); this only swaps the small label below the number to flag
* a disagreement for whoever's testing/tuning detection to notice.
*
* @param stepCount Total steps counted so far in the current attempt.
* @param latestPoseConfirmed Whether the most recently counted step (if
* any) was corroborated by [PosePhaseDetector] at the time it was
* counted; ignored when [stepCount] is 0.
*/
fun renderStepCount(stepCount: Int, latestPoseConfirmed: Boolean = true) {
fun renderStepCount(stepCount: Int) {
textStepCountBig.text = stepCount.toString()
if (stepCount > lastRenderedStepCount) {
pulse()
}
lastRenderedStepCount = stepCount
val flagged = stepCount > 0 && !latestPoseConfirmed
textStepCounterLabel.text = textStepCounterLabel.context.getString(
if (flagged) R.string.step_counter_label_unconfirmed else R.string.step_counter_label
)
textStepCounterLabel.setTextColor(
ContextCompat.getColor(
textStepCounterLabel.context,
if (flagged) R.color.recording_red else R.color.step_counter_accent,
)
)
}
/** @brief Briefly scales the step counter up and back down, drawing the eye to a newly confirmed step. */
@@ -57,17 +57,12 @@ class StepCountingSession {
* @param angles Joint angles computed for this same frame.
* @param timestampMs Wall-clock time this frame was analyzed, in milliseconds.
* @param isStartingPosition Whether the bowler is currently in the starting position.
* @param currentPhase [PosePhaseDetector]'s currently-classified delivery
* phase for this same frame, or null if none currently validates
* -- used only to set each new [StepEvent.poseConfirmed] below,
* never to gate step counting itself (see that field's doc).
*/
fun onFrame(
landmarks: Map<Int, SmoothedLandmark>,
angles: PoseAngles,
timestampMs: Long,
isStartingPosition: Boolean = false,
currentPhase: BowlingPhase? = null,
) {
val smoothedAnkleHip = ankleHipSmoother.smooth(landmarks)
val frame = buildPoseFrame(
@@ -83,15 +78,7 @@ class StepCountingSession {
_stepEvents.value = emptyList()
}
if (result.newSteps.isNotEmpty()) {
// Compared phase-to-phase (via each phase's starting step) rather
// than against the raw step number, since one phase can span
// several steps -- e.g. APPROACH covers steps 1-2, so a pose still
// in APPROACH is correct when step 2 lands.
val posePhaseStart = PosePhaseDetector.stepForPhase(currentPhase ?: BowlingPhase.STARTING_STANCE)
_stepEvents.value += result.newSteps.map { step ->
val expectedPhaseStart = PosePhaseDetector.stepForPhase(PosePhaseDetector.phaseForStep(step.stepIndex))
step.copy(poseConfirmed = posePhaseStart >= expectedPhaseStart)
}
_stepEvents.value += result.newSteps
}
}
@@ -19,21 +19,11 @@ enum class Foot { LEFT, RIGHT }
* @param timestampMs Time this foot-plant was detected, in milliseconds.
* @param foot Which foot planted.
* @param stepIndex 1-based position of this step in the overall approach sequence.
* @param poseConfirmed Whether [PosePhaseDetector]'s independently-classified
* posture had already reached the delivery phase this step index
* expects (see [PosePhaseDetector.phaseForStep]) at the moment this
* step was counted -- see [StepCountingSession.onFrame]. The ankle-peak
* count is trusted either way (this never blocks a step from being
* counted); false just flags that the two signals disagreed, e.g. a
* step 1 landing before pose ever confirmed the bowler had actually
* left the starting stance. Always true from [StepDetector.detect]'s
* batch pass, which has no phase information to compare against.
*/
data class StepEvent(
val timestampMs: Long,
val foot: Foot,
val stepIndex: Int,
val poseConfirmed: Boolean = true,
)
/**
@@ -102,7 +102,6 @@
android:textStyle="bold" />
<TextView
android:id="@+id/text_step_counter_label"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:text="@string/step_counter_label"
@@ -142,7 +142,6 @@
android:textStyle="bold" />
<TextView
android:id="@+id/text_step_counter_label"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:text="@string/step_counter_label"
+1
View File
@@ -21,5 +21,6 @@
<color name="Back_swing_ready">#CC9C27B0</color> <!-- purple -->
<color name="Power_step_ready">#CCFF9800</color> <!-- orange/amber -->
<color name="Slide_and_release_ready">#CCE91E63</color> <!-- pink/red -->
<color name="Follow_through_ready">#CC00BCD4</color> <!-- cyan -->
<color name="final_position_highlight">#FFFFD600</color>
</resources>
+7 -11
View File
@@ -15,7 +15,6 @@
<string name="recording_timer_placeholder">00:00</string>
<string name="step_count_big_placeholder">0</string>
<string name="step_counter_label">STEPS</string>
<string name="step_counter_label_unconfirmed">STEPS · UNCONFIRMED BY POSE</string>
<string name="reset_counter">Reset Steps</string>
<string name="reset_hint_idle">✋ Raise a hand, hold 5s to reset</string>
<string name="reset_hint_holding">Keep holding… %1$d%%</string>
@@ -51,16 +50,13 @@
<string name="editor_save">Save</string>
<string name="editor_saved_toast">Settings saved</string>
<string name="editor_invalid_value_toast">Enter a valid number for every field</string>
<string name="pose_phase_starting_stance">Starting Position</string>
<string name="pose_phase_approach">½ Step / Preparation Step</string>
<string name="pose_phase_pushaway">Push Away &amp; Backswing</string>
<string name="pose_phase_back_swing">@string/pose_phase_pushaway</string>
<string name="pose_phase_starting_stance">Starting Stance</string>
<string name="pose_phase_approach">Approach</string>
<string name="pose_phase_pushaway">Pushaway</string>
<string name="pose_phase_back_swing">Backswing</string>
<string name="pose_phase_power_step">Power Step</string>
<string name="pose_phase_slide_and_release">Slide &amp; Finishing Position</string>
<string name="pose_phase_waiting">Waiting for starting position…</string>
<string name="step_term_half_step">½ Step</string>
<string name="step_term_preparation_step">Preparation Step</string>
<string name="step_term_slide">Slide</string>
<string name="step_term_finishing_position">Finishing Position</string>
<string name="pose_phase_slide_and_release">Slide and Release</string>
<string name="pose_phase_follow_through">Follow Through</string>
<string name="pose_phase_waiting">Waiting for stance…</string>
<string name="pose_metrics_format">Torso: %1$s · Knee L: %2$s R: %3$s\nElbow L: %4$s R: %5$s</string>
</resources>
+19
View File
@@ -5,11 +5,30 @@
## Suggested sections to include
- Problem statement / motivation
Provide an app for users to improve their bowling by learning hwo to get into the correct positions.
- Target users
Anyone who is able to bowl
- Proposed solution and key features
Able to see their mistakes
Able to see instructions on how to correct their mistakes
Able to hear instructions on how to correct their mistakes
Able to see the ideal form
- Scope (in-scope vs. out-of-scope for this project)
Five step bowling only
Right handed bowling only
Starting position correction and finishing position only
- Success criteria
User is able to get into the correct position with help from the app
- Team members and roles
Lu Yong Wei (Product Owner)
Gabriel Low (Programmer/ Version control manager)
Harine S/O Sumen (Tech Lead)
Liu Jingwen (Programmer/ Serialization manager)
Auvik Kumar Biswas (Programmer/ Feedback manager)
Chan Qi Ying (Programmer/ UI lead)
Low Yu Sheng Javier (Product Manager)
Khalil Belabadia (Programmer/ Audio Lead)
## Working summary (placeholder, derived from the current codebase — replace with the real proposal)
+1 -1
View File
@@ -1,5 +1,5 @@
[versions]
agp = "9.4.0"
agp = "9.3.0"
coreKtx = "1.18.0"
junit = "4.13.2"
junitVersion = "1.3.0"
+1 -1
View File
@@ -1,7 +1,7 @@
#Fri Apr 10 12:25:49 SGT 2026
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionSha256Sum=b266d5ff6b90eada6dc3b20cb090e3731302e553a27c5d3e4df1f0d76beaff06
distributionSha256Sum=bbaeb2fef8710818cf0e261201dab964c572f92b942812df0c3620d62a529a01
distributionUrl=https\://services.gradle.org/distributions/gradle-9.6.0-bin.zip
networkTimeout=10000
validateDistributionUrl=true