React Websockets example

To get access to the Fitty Camera URL and endpoint for the sockets please contact our team.

The overal architecture of the SDK can be seen bellow:

The architecture is composed of two main components:

1. The Fitty Camera is a iframe application that captures person’s joint coordinates, extracted via pose estimation. The pose-estimation computation is fully done on the client’s device thus the video feed never leaves the client’s device.
2. The Fitty Websockets are used to receive feedback, repetition count and other metrics computed from person’s joint coordinates by the Fitty Trainer running in our servers. Fitty Websockets are also used to communicate to the server which exercise the person should currently perform.

Setting-up the project

Setup the project

The project can be cloned from our private SDKs repository it also has an example of Web application that is based on this tutorial: Please email our team at darius.butkevicius@fittyai.com to get access to the repository.

Rendering iframe with Fitty Camera

Fitty Camera is accessed through a web application. To integrate it into your application we recommend using a iframe component.

For iframe component you will need random generated sessionId.

    import { v4 as uuidv4 } from "uuid";

    // Create random guid for sessionId
    const sessionId = uuidv4();

Now we need to connect the iframe to a workout session that will process user joint data from the users camera:

    const environment = "prod";
    const clientId = "YOUR_CLIENT_NAME";

    const FittyCameraEmbed = ({ sessionId }) => (
        <iframe
            src={`https://${environment}.fittyai.com/web/${clientId}/fitty-camera/${sessionId}`}
            allow="camera"
        />
    );

After rendering this component you should see your camera feed in your iframe.

Connecting to the Fitty Websockets

To receive repetition count, and feedback and communicate which exercises to track, the direct websocket client connection is required with the same session GUID.The CLIENT-NAME field needs to be replaced with the one we provided to you during onboarding. Here is a simple client example:

   import { useWebSocket } from "react-use-websocket/dist/lib/use-websocket";
   export const SOCKET_URL = "wss://backend.fittyai.com:443";
   // Initialize socket connection
   const { sendMessage, lastMessage, readyState } = useWebSocket(SOCKET_URL);

To initialise the session send a session_start event from the client:

    const startSession = () => {
        const parameters = {
            company_name: clientId,
            event_type: "session_start",
            jwt: "GENERATED FROM OUR API"
            user_bio_information: {
                "height": 184,
                "weight": 77,
                "gender": 0
            },
            timestamp: "2022-03-01 12:32:12"
        };

        const payload = createPayload(parameters, sessionId);
        sendMessage(payload);
    };

    useEffect(() => {
        // If socket connection is open
        if (readyState === ReadyState.OPEN) {
            console.log("Socket OPENED");
            // Start FittyAI workout session
            startSession();
        }
    }, [readyState]);

To generate a JWT please follow the instructions here.

At this point, you should have successfully connected to your iframe session and can start communicating with the virtual trainer.

If you want to see full solution with libraries, utils and all other bits and pieces please check our react application example on GitHub .

Example React code on GitHub

Communication with web sockets

Event types

Start Session

    {
        "company_name": "YOUR_CLIENT_NAME",
        "session_id": "11bc6524-7ae4-4e13-a994-a6593bc6297d",
        "event_type": "session_start",
        "jwt": "GENERATED FROM OUR API",
        "user_bio_information": {
            "height": 184,
            "weight": 77,
            "gender": 0
        },
        "timestamp": "2022-03-01 12:32:12"
    }

Start New Exercise

    {
        "event_type": "start_exercise",
        "session_id": "11bc6524-7ae4-4e13-a994-a6593bc6297d",
        "timestamp": "2022-03-01 12:32:12",
        "exercise_id": "squat"
    }

Metric Response

    {
        "event_type": "state_update",
        "timestamp": "2022-03-01 12:32:12",
        "data": {
            "metrics": [
                {
                    "name": "rep_count",
                    "value": 3
                },
                {
                    "name": "calories",
                    "value": 12
                }
            ],
            "message": [
                {
                    "name": "feedback",
                    "value": "Spread your legs wider!"
                }
            ]
        },
        "version": "0.1v"
    }

Exercises Names for testing

Exercise Id	Exercise Name
squat	Ordinary Squat
push_up	Ordinary Push Up
sit_up	Sit up

The extended list of exercises can be provided on email request at modestas.jurcius@fittyai.com.