BLOCKS Reference

Machine Learning

AutoML Vision prediction

This BLOCK is currently only available as a limited-release alpha version. This version may become unusable after beta and official version release.

Overview

This BLOCK sends images to a model deployed in AutoML Vision to predict what objects are contained within the images.

Input image format

You must store the input images for the prediction into a variable before executing this BLOCK.

The following examples show how to prepare your data using JSON or the Construct object BLOCK.

There are several ways to designate your data:

  • Designate the GCS URL (gs://BUCKETNAME/FILENAME) for the image file as a string.

    • Construct object BLOCK example:

      Using the Construct object BLOCK for a GCS URL string

      Set the value’s type to String

      If the Construct object BLOCK’s Results variable property is set to _, you would enter _.data into the AutoML Vision prediction BLOCK’s Input variable property.

    • JSON text example:

      "gs://my-bucket/sample01.jpg"

      Use a string ("...") to designate the image file’s GCS URL.

  • For multiple images, you can designate an array of GCS URLs:

    • Construct object BLOCK example:

      Using the Construct object BLOCK for an array of GCS URL strings

      Set the value’s type to Array. Click the + button next to Array to add elements to the array. Enter a GCS URL for an image into each element as a String.

      If the Construct object BLOCK’s Results variable property is set to _, you would enter _.data into the AutoML Vision prediction BLOCK’s Input variable property.

    • JSON text example:

      ["gs://my-bucket/sample01.jpg", "gs://my-bucket/sample02.jpg"]

      • Designate an array ([...]).
      • For each element of the array, designate the GCS URL of an image as a string.
  • You can also designate a key for each image file. This can make it easier to manage the image files and corresponding results for multiple images.

    • Construct object BLOCK example:

      Designating a key

      Set the value’s type to Object. Click the + button next to Object to add a key and the GCS URL for an image as Strings to the object. Set the Key for the key as key, and set the Key for the image as image.

    • JSON text example:

      {"key": "sample01", "image": "gs://my-bucket/sample01.jpg"}

      • Use an object ({...}) to designate the data.
      • You must designate a member as the key, and this member’s name must be "key". The value for this member should be string that specifies which image file is being sent in this object.
      • The second member contains the image file information. This member’s name must be "image". Set this member’s value as a string for GCS URL of an image.

    You can use an array to designate multiple images:

    • Construct object BLOCK example:

      Designating multiple objects with keys

      Set the value's type to Array. Click the + button next to Array to create elements in the array for each object. Set each object’s contents in the same manner explained above.

      If the Construct object BLOCK’s Results variable property is set to _, you would enter _.data into the AutoML Vision prediction BLOCK’s Input variable property.

    • JSON text example:
      [
        {
          "key": "sample01",
          "image": "gs://my-bucket/sample01.jpg"
        },
        {
          "key": "sample02",
          "image": "gs://my-bucket/sample02.jpg"
        }
      ]
      
      • Use an array ([...] to designate the data.
      • Designate each element in the array as an object with information for an image file. Format the members within the object in the same manner explained above.
  • If you are designating keys for your image files, you can send your image file as the following, rather than as a GCS URL:

    • The contents of the image as Base64 encoded data
    • The contents of the image as binary data

Format of the prediction results

The results of the prediction are output as a variable. The following example shows the results of using the Output to log BLOCK to view the prediction results as JSON format text.

{
  "predictions": [
    {
      "key": "gs://MyBucket/sample01.jpg",
      "labels": [
        {
          "label": "navy",
          "score": 0.28951025009155273
        },
        {
          "label": "black",
          "score": 0.11685572564601898
        },
        {
          "label": "red",
          "score": 0.10765111446380615
        },
        {
          "label": "wine",
          "score": 0.10530507564544678
        },
        {
          "label": "blue",
          "score": 0.060610901564359665
        }
      ]
    }
  ]
}
  • The results are output as a single JSON object.
  • "predictions": An array containing the results of the prediction. It The prediction results are an object with the following members:
    Name Explanation
    "key" If you designated a key when making the prediction, it is returned here. Otherwise, the GCS URL of the image will be returned.
    "labels"

    A list of the object labels ordered according to the "score" value (descending).

    "label" A label for something that is detected in the image.
    "score"

    The confidence level for detecting the label’s object.

Properties

Property Explanation
BLOCK name

Configure the name displayed on this BLOCK.

GCP service account

Select the GCP service account to use with this BLOCK.

Dataset

Designate the dataset of the deployed AutoML Vision model that will be used for the prediction.

Model

Designate the deployed AutoML Vision model that will be used to make the prediction.

Input variable

Designate the variable that contains the images/information for images to be used for the prediction.

Output variable

Designate the variable that will store the results of the prediction.

Number of annotation labels

Designate the maximum number of labels to be returned in the results.

BLOCK memos

Make notes about this BLOCK.