Integrate Haut.AI Image Analysis with Your App

This page describes the necessary steps needed to integrate Haut.AI powered image analysis into your Custom App

In the previous step, you already configured your backend and SaaS platform, have a valid Dataset, and attached an image analysis Algorithm Application.

Now, let's continue with image upload and processing with SaaS.

About Image Analysis

Algorithms is an AI core functionality of Haut.AI products. Algorithms enable AI image analysis. You can read more about each application here: Algorithms

Create Subject in SaaS (repeat this for every unique end-user)

To upload Images, you need to create a Subject. SaaS Subject is a representation of a unique end-user (person) that uses your application, powered by Haut.AI technology, e.g., one of your customers or one of your study participants.

Check the Subjects feature description to know more about Subjects.

Check the Subjects API Overview for details about the relevant methods.

  • The only required key for subject creation is name.

  • The Subject is always stored in the Dataset.

Critical Information

New Subject creation action must be performed every time a new customer (end-user) uses your app and does the image analysis.

We highly encourage you to implement the logic of end-user authentification/detection on the side of your app to ensure that you can identify different real customers (end-users).

Avoid storing images of different people in one Subject!

This implementation significantly limits your app in:

  • GDPR compliance: with 1 Subject representing only 1 real person, you can always perform GDPR-compliant actions like full personal data deletion easily.

  • Historical data and dynamics: with 1 Subject representing only 1 real person, your app can easily leverage such features as "showing the dynamics in skin health change" or smooth the results of algorithms to avoid too high variability (see #results-smoothing)

Upload Image to SaaS

When the Subject is created successfully, you can upload Images to it.

Check the Images API overview for details about the relevant methods and uploaded Image limitations.

  • Image is always linked to a Subject.

  • Image is always a part of a Dataset.

  • Images are always grouped into Batches (this is useful for uploading side images along with the frontal image. However, every single uploaded Image is automatically linked to a Batch as well).

FAQ about images

When will Algorithms process my Image?

Image is sent to Algorithms only if the Subject's Dataset has attached at least 1 Algorithms Application. If no Algorithms are attached to Dataset of the Subject, the Image will be simply stored in the Dataset without being processed.

If I attach the Algorithm Application after the image upload, will it process all previously uploaded images?

Unfortunately, no. Haut.AI services currently do not support the function for retrospective image analysis. Only images uploaded after the Algorithm Application is attached are processed with this Application.

Receive Metrics and Submetrics from SaaS to Your App Server

Metrics and Submetrics are the results of image analysis performed by one or more of Algorithms on the uploaded image.

Check the description of the Algorithm Results for more details.

Check the Metrics and Submetrics API Overview for details about the relevant methods.

After you've uploaded your Image and called the /compute/ method, the processing of algorithms on your Image begins. Once the calculation is complete, a Webhook will be sent to you with the Metrics and Submetrics of the processed Image.

Each webhook payload contains:

  • Details about the Subject and the Image processed.

  • Metrics and Submetrics for specific Areas of the Image.

  • Product Categories recommended by Recommendation Engine

  • Optional Algorithms Application information

Receive Masks

Masks are additional results of image analysis, representing the spatial information about detected skin/hair health returned by the selected set of Algorithms.

Masks are not included in the Webhook data due to their weight. To receive Masks, you need to make an additional API call.

Check the description of the Algorithm Results for more details.

Check the Metrics and Submetrics API Overview for details about the relevant methods.

Receive Smoothed Metrics

Metrics of some Algorithms may vary from image to image naturally due to skin variability or image quality variability. Haut.AI labels all Algorithms that demonstrate higher variability by a disclaimer; read more, for example, for Face Metrics 2.0 Algorithms.

If your application has a returning user experience (make photo daily/weekly/monthly), you might benefir from having a historical-based Metrics Smoothing for every new image uploaded to the same Subject. Using this method adjusts the numerical values of every new Image Metrics result based on previous analyses of the same Subject.

Historical Smoothing of Metrics values works ONLY if your application correctly stores images of a single real person in one Subject. See #create-subject for more details.

Check the Metrics and Submetrics API Overview for details about the relevant methods.

Design a Fallback in Your App for Algorithm Processing Errors

It is important to always prepare your app to be able to navigate your end-users correctly in case of any issue occurs with Haut.AI services.

In the case of (low probability) image processing errors with Algrorithms, the image might be not processed by the Algorithm, and its result will not include any Metrics or Submetrics at all. Instead, it might have an object with just one key, like the example below:

{
  "id": "bde7dc81-44ad-4bba-84c2-3202c166b367",
  "result": {
    "error": "No face detected"
  },
  "is_ok": false,
  "image_id": "72251ab1-7374-4a2a-a32f-f917c8e907ba",
  "algorithm_version_id": 69,
  "creation_time": "2024-10-10 08:46:54",
  "application_id": "8b5b3acc-480b-4412-8d2c-ebe6ab4384d7",
  "application_name": "Face Skin Metrics 2.0",
  "application_description": "Use this application to analyze the skin’s overall condition.<br>This app will check the selfie’s image quality and assess visual skin features, such as:<ol>    <li>perceived age</li>    <li>eye age</li>    <li>skin tone</li>    <li>eye bags</li>    <li>uniformness</li>    <li>redness</li>    <li>wrinkles (deprecated)</li>    <li>pores</li>    <li>sagging (deprecated)</li>    <li>acne</li>    <li>pigmentation</li>    <li>translucency</li>    <li>hydration</li>    <li>facial landmarks</li>    <li>lines</li></ol>For the calculated metrics, the application will produce analytics, which is available in the interface. You can use this application to compare subjects, track skin dynamics over time, and get skin metric reports.",
  "application_run_id": "c7f522cb-8bf0-4afd-b96c-ed81cfc2f803",
  "image_type": "selfie",
  "algorithm_family_tech_name": "selfie_v2.lines"
}

The top object of the result has a property "is_ok": false which means that it is an unsuccessful calculation. Also, you can see that the key result contains another object - "error", containing a short explanation of the issue that lead to the failure of analysis:

{
  "result": {
    "error": "No face detected"
  }
}

How to deal with errors

If this is a quality detection error (detection errors are usually the same across all algorithms), you should kindly ask the user to upload another photo.

If this is an individual algorithm error, you should indicate to the user that this algorithm can't be calculated by this algorithm (you can optionally show the error message to the user as well).

Next Steps

Follow our next guide Create the Best End-User Experience for Image Collection in Your App to learn how to:

  • Integrate smart camera experience for high-quality image collection

  • Handle the image collection for laptop/desktop users to ensure the high quality of the analysis

PreviousConfigure Your App Backend and SaaS for IntegrationNextCreate the Best End-User Experience for Image Collection in Your App

Last updated