Antonio Colon
  • HomePage
  • ProjectsPage
  • AboutPage
  • ResumePage
  • ContactPage
  • RockyiOS / Computer Vision
  • CartWiseFull Stack
  • Modus AIAI / Machine Learning
  • NOMADMobile App
  • Learn++Full Stack
  • Paco's Puppy ProblemMachine Learning
  • Insta485Full Stack
  • Distributed MapReduceDistributed Systems
  • Scalable Search EngineSearch Systems
  • LC-2K Cache SimulatorComputer Architecture
  • SillyQL Database EngineDatabase Systems
  • Mine EscapeAlgorithms
  • Letterman Word MorphAlgorithms
  • Copy email addressantoniopcolon@gmail.com
  • Open resume PDFAction
  • GitHubLink
  • LinkedInLink
  • Switch light / dark themeAction

↑↓ to move, Enter to open, Esc to close

← All projects
Rocky app icon

Rocky

projecting · Builder · September 2026 - Present

An iPhone app that watches you boulder and tells you how to climb better. It finds the holds on a wall, groups them by route, and tracks your body through a recorded climb.

In the app

Rocky showing a gym wall with 94 detected holds boxed and grouped into 6 route coloursRocky with the pink route selected: its 17 holds highlighted and ready to correct by tappingRocky's Analyze screen, with buttons to photograph a wall or choose a photo

What it is

Rocky is a climbing coach I'm building for my own sessions, and a way to learn computer vision, backend design, and iOS on one real problem. Today it does two things. Photograph a wall and it boxes every hold and groups them by route colour, with a tap to fix anything it gets wrong. Record a climb and it returns the clip with a skeleton tracked over your body. The part I'm working on now joins the two: working out which hold each hand and foot is on, so a clip becomes a sequence of moves that technique feedback can be built from.

How a climb moves through it

  1. iPhone appTakes the wall photo and records the climb.
  2. FastAPICreates the climb and hands back a short-lived upload URL.
  3. Cloudflare R2The phone uploads the video directly, so the API never touches the bytes.
  4. Redis queueThe API enqueues the analysis and answers right away.
  5. WorkerRuns hold detection and pose tracking on the clip.
  6. Back on the phoneThe app polls for the result and draws holds and skeleton over the footage.

Build log

  1. 0

    Plumbing

    Record on the phone, upload to storage, get a response from a deployed server.

    Done
  2. 1

    Hold detection

    Photograph a wall and see every hold boxed and grouped by route colour.

    Done
  3. 2

    Pose tracking

    Upload a clip and play it back with a skeleton over the climber.

    Done
  4. 3

    Move segmentation

    Line up holds with joint positions to turn a clip into a list of moves.

    In progress
  5. 4

    Technique feedback

    Hip distance from the wall, hesitation before a move, foot precision, as notes timed to the clip.

    Planned
  6. 5

    Accounts and history

    Sign in, keep every climb, and see how technique changes over time.

    Planned
  7. 6

    Grading

    Relative difficulty first: harder or easier than your last few climbs.

    Planned

What I learned building it

Colour is not RGB

The first version grouped holds by raw RGB and produced muddy browns and greys, because gym lighting changes how bright each hold looks far more than it changes its hue. Dropping brightness and clustering on hue and saturation alone fixed it: on my test wall it picked six groups, matching the six routes actually set.

A detector on a free tier

The server has roughly 512 MB of memory and PyTorch alone uses more than that. I exported the trained model to ONNX and wrote letterboxing, box decoding, and non-max suppression by hand, so the deployed API runs on ONNX Runtime and NumPy with no PyTorch installed.