# Image analysis

> Extract an image's color palette, perceptual hash, face positions, content tags, aesthetic score, and feature vector.

With the following `fmt` values, CoreIX returns analysis results for the image instead of the image itself.

| fmt value      | Returns                                                  | Format | Typical use                                              |
| :------------- | -------------------------------------------------------- | :----: | -------------------------------------------------------- |
| `palette`      | **Color palette**: dominant colors and their proportions |  JSON  | Theme colors, placeholder backgrounds                    |
| `phash`        | **Perceptual hash**                                      |  Text  | Duplicate image detection                                |
| `faces`        | **Face positions**                                       |  JSON  | Face location, custom cropping                           |
| `tags`         | **Content tags**: scenes and objects                     |  JSON  | Automatic classification, search                         |
| `aesthetics`   | **Aesthetic score**                                      |  JSON  | Filtering for high-quality images, choosing cover images |
| `featureprint` | **Feature vector**                                       |  JSON  | Similar image search                                     |

- Analysis runs on the **processed image**, so parameters such as `w` and `rect` take effect first.
- For animated images, the first frame is used; for video, a frame is extracted first; for Live Photos, the still image is used.

## Color palette

<https://demo.bitiful.com/girl.jpeg?fmt=palette&colors=5>

| Parameter | Description                 | Default | Values  |
| :-------- | --------------------------- | :-----: | ------- |
| `colors`  | Number of colors to extract |    6    | 1 \~ 16 |

```json
    {
      "colors": [
        { "hex": "#c8a27e", "rgb": [200, 162, 126], "ratio": 0.3124 },
        { "hex": "#2f3b4a", "rgb": [47, 59, 74], "ratio": 0.2218 }
      ],
      "dominant": "#c8a27e",
      "average": "#8f8479"
    }
```

`colors` is sorted by proportion in descending order. `dominant` is the color with the highest proportion, and `average` is the average color of the whole image. If you only need a single dominant color, `fmt=color` is lighter weight.

## Perceptual hash

<https://demo.bitiful.com/girl.jpeg?fmt=phash>

Returns a 16-character hexadecimal string (a 64-bit hash), such as `c3a59e1b7d2460f8`.

Images with the same content produce similar hashes, even after resizing, compression, or format conversion. Compare the Hamming distance (the number of differing bits) between two hashes: **a distance of 10 or less usually means the same image**.

## Face positions

<https://demo.bitiful.com/girl.jpeg?fmt=faces>

```json
    {
      "width": 1200,
      "height": 800,
      "faces": [
        { "x": 412, "y": 156, "w": 180, "h": 180, "conf": 0.987 }
      ]
    }
```

Coordinates are pixel values on the processed image. `x` and `y` are the top-left corner of the face box, and `conf` is the confidence. Results are sorted by face area, from largest to smallest.

## Content tags

<https://demo.bitiful.com/girl.jpeg?fmt=tags>

| Parameter  | Description                     | Default | Values   |
| :--------- | ------------------------------- | :-----: | -------- |
| `tags-min` | Minimum confidence              |   0.2   | 0 \~ 1   |
| `tags-max` | Maximum number of tags returned |    20   | 1 \~ 100 |

```json
    {
      "labels": [
        { "name": "outdoor", "conf": 0.9312 },
        { "name": "bridge", "conf": 0.8745 }
      ]
    }
```

Tags are in English and cover more than 1,300 common scenes and objects.

## Aesthetic score

<https://demo.bitiful.com/girl.jpeg?fmt=aesthetics>

| Field     | Description                                                                          |
| :-------- | ------------------------------------------------------------------------------------ |
| `overall` | Overall aesthetic score, from -1 to 1; higher is better                              |
| `utility` | Whether the image is a utilitarian image, such as a screenshot, document, or receipt |

## Feature vector

<https://demo.bitiful.com/girl.jpeg?fmt=featureprint>

```json
    {
      "n": 768,
      "type": "float32",
      "norm": "l2",
      "input": 224,
      "b64": "…"
    }
```

`b64` is the Base64 encoding of 768 32-bit floats (little-endian), and the vector is already normalized. To compare two images, compute the Euclidean distance between their vectors:

| Distance         | Meaning           |
| :--------------- | ----------------- |
| Less than 0.4    | Near-duplicate    |
| 0.4 \~ 0.9       | Related content   |
| Greater than 0.9 | Largely unrelated |

> **Tip**
>
> Perceptual hashes are best for finding "different versions of the same image"; feature vectors are best for finding "different images with similar content."
