In this activity, we will be learning to use BlazePose, a software library that finds joints and other body landmarks in an image or video. We will use the p5.js library for creative coding to control it.
This is a programming-intensive activity, but don’t worry if you don’t have coding experience. We have done it on the first day of class with many students with a range of prior coding experience, from none to lots. Just remember to ask for help when you are stuck, so that this is a exercise in learning and not in frustration.
BlazePose is a software library that estimates the pose of a person in an image or video. It employs machine learning to compute the probable positions of 33 body landmarks, or skeleton keypoints. These keypoints are located at major skeletal joins (e.g. wrists, elbows, shoulders, knees) and facial features (eyes, ears, and mouth).
BlazePose was developed by Google. It can be used with programs written in JavaScript, or Python. In this activity, we will use the JavaScript version.
More details on how BlazePose works, including a list of keypoints, are in this blog post.
p5.js is a JavaScript library with the goal of making coding accessible to artists, designers, educators, and beginners.
p5.js is inspired by — and very similar to — Processing. If you know Processing, many of the concepts in p5.js will look familiar (and vice versa). Unlike Processing, p5.js uses JavaScript instead of Java, and runs in a web browser (including on mobile devices) instead of in a dedicated IDE.
In this activity, you will use the browser version of p5.js, which is maintained by the Processing Foundation. We recommend running it in Chrome, (we can try other browsers in class).
You will run our class “starter kit” project that uses BlazePose (it’s based on, and similar to, public demo code for BlazePose). You will become familiar with the browser interface to p5, and you will modify the program.
You will create an account on the P5 Editor site, so that you can create and save versions of the program to make your own projects using BlazePose.
Run BlazePose in your browser by opening the following URL. Chrome works better (runs at a higher frame rate) than Safari.
Note: The browser may request permission to use your camera. Grant it.
Note: If your computer has multiple cameras, Chrome may choose the wrong one. See the Troubleshooting section at the end of this document.