BLOCKS Reference

Machine Learning

Model Generator prediction (online)

Trainings made prior to 2017/4/12 became unusable after 2017/6/1. Please use trainings made after 2017/4/12 when making predictions.

Overview

This BLOCK uses a trained model from a Model Generator with input data to make predictions.

It supports the following types of Model Generator:

  • Classification
  • Regression
  • Feature vector extractor (alpha)
  • Image classification
  • Object detection

It reads in input data from a variable and outputs the prediction results as another variable.

Model Generator prediction BLOCK overview

We recommend using the Model Generator prediction (batch) BLOCK for predictions with a large amount of prediction input data.

  • There is a 1,000 row limit when making predictions from the results of a BigQuery query.
  • This BLOCK may not be able to make predictions if there is a very large amount of prediction input data.

You must first click Apply for a training on a Model Generator before you can make predictions with this BLOCK.

Formatting of prediction input data

You must store your prediction input data into a variable before executing this BLOCK.

The formatting of the contents of this variable differ depending on whether you are using the classification, regression, or feature vector extractor (alpha) types or the image classification or object detection types of Model Generator.

This page uses the Construct object BLOCK with JSON data notation to explain how to format this data.

What is JSON?

First, we will briefly explain the basics of JSON. For a more detailed explanation, please refer to Introducing JSON and RFC 7159 .

  • JSON is a notation format for expressing data.
    {
      "name": "John Smith",
      "age": 35,
      "languages": ["English", "Japanese"]
    }
    
  • Data is referred to as values in JSON. Values can be of the following types:
    • String: These are strings of characters enclosed within double quotes ("), such as "abc" or "xyz".
    • Numerical values: Directly entered numbers such as 1 or 1.23. These are expressed in base 10. Scientific notation, such as 1.234e-5 can also be used.
    • Boolean values: Shows if the value is true or false. Expressed as true or false.
    • null: Shows that the value is empty. Expressed as null.
    • Arrays: Ordered series of values, such as [1, 2, 3] that are separated by commas (,). Arrays are enclosed by square brackets ([ and ]). Each value in an array is called an element.
    • Object: Sets of name and value pairs.
      • Objects are enclosed by curly brackets ({ and }).
      • Name/value pairs are separated by commas (,), as in {"a": 1, "b": 2, "c": 3}.
      • A name and it’s accompanying value are separated by a : (name: value.
      • Names are expressed as strings (enclosed in ").
      • Name/value pairs are called members.
  • Data expressed with JSON is called JSON text.
Formatting for classification, regression, and feature vector extractor (alpha) models

Store a set of the input data and a key into a variable.

  • Example method using the Construct object BLOCK:

    Example with one set of input data

    As shown in the image above, select Object for the value type, and set its contents (click the + button next to Object) with a key and your input data. Follow these rules when entering the data:

    • You must set a key for the data called key. The value of this key acts as an identifier for this set of input data. The value type must be set to String.
    • For the input data, set each key as the item names from the Model Generator’s training data. Set the values as the input data that will be used to make the prediction. Set the value types according to each type of data.

    This BLOCK can accept numbers that have been converted into strings for numerical values, month, and days. For example, numerical values that have been converted into strings, like “99” or “1.5”, will be treated as 99 and 1.5. Similarly, string data like “0” and “6” signifying days of the week are treated as 0 and 6.

  • info_outline For sequence data, set an array.

    Sequence data, like the following image of weather data over the past 7 days, has meaning in its ordering. This section explains how to set an array for this example sequence.

    Past 7 days weather sequence example

    When using this sequence data as part of an Model Generator’s training data, you would simple write it out with commas (,) separating each number, as shown below.

    Example sequence data for ML Board training

    In the same manner, you list sequence data as an array when using it for a prediction.

    The following image shows how to list sequence data as an array using the Construct object BLOCK.

    Sequence array

    (For the data in the image above, the Model Generator predict (online) BLOCK’s Input variable property would be set to _.data, and it assumes that the item name for sequence data in the Model Generator is weather.)

    The following shows how to write this array using JSON:

    "weather": [21, 35, 0, 20, 34, 0, . . . , 18, 32, 20]
    

    (This assumes that the sequence data’s item name is weather)

  • JSON text example:
    {
      "key": "1",
      "sepal_length": 5.9,
      "sepal_width": 3.0,
      "petal_length": 4.2,
      "petal.width": 1.5
    }
    
    • Specify an object ({...}).
    • You must set a member to act as a key for the object. This member must be named "key". Set its value as a string that will act as an identifier for this set of input data.
    • Set other members with names as found in the item names for the Model Generator’s training data, and values in accordance with the data for each item.

Do the following for making predictions with multiple sets of input data:

  • Example method using the Construct object BLOCK:

    Predictions with multiple sets of prediction input data

    As shown in the image above, select Array for the value type. Add an object for each set of prediction input data to the array by clicking the + button next to Array. Enter the data into each object as explained earlier.

  • JSON text example:
    [
      {
        "key": "1",
        "sepal_length": 5.9,
        "sepal_width": 3.0,
        "petal_length": 4.2,
        "petal_width": 1.5
      },
      {
        "key": "2",
        "sepal_length": 6.9,
        "sepal_width": 3.1,
        "petal_length": 5.4,
        "petal_width": 2.1
      },
      {
        "key": "3",
        "sepal_length": 5.1,
        "sepal_width": 3.3,
        "petal_length": 1.7,
        "petal_width": 0.5
      }
    ]
    
    • Set as an array ([...]).
    • Set each element of the array as objects for the prediction input data. The format for each object is the same as explained previously.
Formatting for image classification and object detection models

The variable’s contents should point to the JPEG, PNG, or GIF image files that will be used for the prediction. There are various ways to accomplish this.

Keep in mind the following in regards to image files:

  • File size: Images must be less than approximately 1.125 MB.
  • For object detection predictions: When making predictions for multiple images at once, all of the prediction images must be the same size.
  • Specify the image file’s GCS URL (gs://BUCKETNAME/FILENAME) as a string.

    • Example method using the Construct object BLOCK:

      Image classification: prediction input data as strings

      As shown in the image above, select String for the value type.

      Assuming that the above Construct object BLOCK’s Results variable property is set to _, _.data would refer to the prediction input data.

    • Example method using JSON text:

      "gs://my-bucket/sample01.jpg"

      Specify the image file’s GCS URL using a string value ("...").

  • Use an array to specify multiple image files.

    • Example method using the Construct object BLOCK:

      Image classification: prediction input data as an array of strings

      As shown in the image above, select Array as the value type. Click the + button to the left of Array to add elements to the array. Set each element as a string for an image file’s GCS URL.

      Assuming that the above Construct object BLOCK’s Results variable property is set to _, _.data would refer to the prediction input data.

    • Example method using JSON text:

      ["gs://my-bucket/sample01.jpg", "gs://my-bucket/sample02.jpg"]

      • Set as an array ([...]).
      • Set each element of the array as a string for each image file’s GCS URL.
  • As with the regression and classification type models, you can also assign keys to image files. By using these keys, you can make it easier to manage prediction input images and prediction results for predictions involving multiple images.

    • Example method using the Construct object BLOCK:

      Image classification: prediction input data as an object

      As shown in the image above, select Object as the value type. For the contents of an object, click the + button next to Object to enter a key, and again to enter a string for a GCS URL. Set the Key fields for each as key and image, respectively.

    • Example method using JSON text:

      {"key": "sample01", "image": "gs://my-bucket/sample01.jpg"}

      • Set as an object ({...}).
      • You must set a member to act as a key for the object. This member must be named "key". Set its value as a string that will be the identifier for an image file.
      • Set another member with information for the image file. This member must be named "image". Set its value as a string for the image file’s GCS URL.

    Use an array to specify multiple image files.

    • Example using the Construct object BLOCK:

      Image classification: prediction input data as multiple objects

      As shown in the image above, select Array for the type. Enter objects with information for each image file into the array by clicking the + button next to Array. Follow the same instructions as earlier for entering each file’s information into each object.

      Assuming that the above Construct object BLOCK’s Results variable property is set to _, _.data would refer to the prediction input data.

    • Example as JSON text:
      [
        {
          "key": "sample01",
          "image": "gs://my-bucket/sample01.jpg"
        },
        {
          "key": "sample02",
          "image": "gs://my-bucket/sample02.jpg"
        }
      ]
      
      • Set an array ([...]).
      • Set each element of the array as an object containing an image file’s information. Format each object the same as explained previously.
  • When setting objects with keys, it is also possible to pass image files using the following formats (aside from using a GCS URL):

    • The contents of the image file as Base64 encoded data.
    • The contents of the image file directly as binary data.

Formatting of the prediction results

Results from a prediction are output as a variable. We will explain the formatting of the prediction results by using the Output to log BLOCK to see the contents (JSON format) of this variable.

Classification prediction results

The following example shows prediction results for a classification model:

{
  "predictions": [
    {
      "label_index": 1,
      "score": [
        8.715780131751671e-05,
        0.9995228052139282,
        0.00039013021159917116
      ],
      "key": "1",
      "label": "Iris-versicolor"
    },
    {
      "label_index": 2,
      "score": [
        9.230815578575857e-08,
        0.007054927293211222,
        0.9929450154304504
      ],
      "key": "2",
      "label": "Iris-virginica"
    },
    {
      "label_index": 0,
      "score": [
        0.9998869895935059,
        0.00011299729521851987,
        1.5803254260760013e-09
      ],
      "key": "3",
      "label": "Iris-setosa"
    }
  ]
}
  • The results are output as a single JSON object.
  • "predictions": An array containing the results of the prediction. The elements of this array are objects with the following members:
    • "label_index": Shows which element of the "score" array the value of "label" represents. The elements in the array are ordered starting from 0.
    • "score": The level of certainty for predicting each class.
      • Shows the certainty level for each class in order, i.e. certainty for class 0, certainty for class 1, certainty for class 2, and so on.
      • For example, results of [8.715780131751671e-05, 0.9995228052139282, 0.00039013021159917116] show a 0.00871578% certainty for class 0, a 99.952280521% certainty for class 1, and a 0.039013021% certainty for class 2.
      • The class with the highest certainty level is the predicted class.
    • "key": The value of the "key" used in the prediction input data.
    • "label": The predicted class. The class with the highest value from "score".
Regression prediction results

The following is an example of prediction results for a regression model:

{
  "predictions": [
    {
      "output": 10261.9072265625,
      "key": "20170101"
    },
    {
      "output": 12506.2861328125,
      "key": "20170102"
    },
    {
      "output": 10304.1962890625,
      "key": "20170103"
    },
    {
      "output": 10350.099609375,
      "key": "20170104"
    }
  ]
}
  • The results are output as a single JSON object.
  • "predictions": An array containing the results of the prediction. The elements of this array are objects with the following members:
    • "output": The predicted value.
    • "key": The value of the "key" used in the prediction input data.
Feature vector extractor (alpha) prediction results

The following is an example of prediction results for a feature vector extractor (alpha) model:

{
  "predictions": [
    {
      "vector": [
        -1.3181713819503784,
        0.09076656401157379
      ],
      "key": "1",
    },
    {
      "vector": [
        1.53475821018219,
        -2.447021245956421
      ],
      "key": "2"
    },
    {
      "vector": [
        0.29401323199272156,
        0.5646417737007141
      ],
      "key": "3"
    }
  ]
}
  • The results are output as a single JSON object.
  • "predictions": An array containing the results of the prediction. The elements of this array are objects with the following members:
    • "vector": Vectors (arrays) that express the prediction data’s features. The number of elements within the vector array is determined by the feature vector dimensions setting when creating the Model Generator.
    • "key": The value of the "key" used in the prediction input data.
Image classification prediction results

The following example shows results from an image classification model:

{
  "predictions": [
    {
      "labels": [
        "cat",
        "dog"
      ],
      "score": [
        1.0,
        3.764146683238323e-08
      ],
      "key": "gs://my-bucket/blocks_ml_image_example/prediction/sample_01.jpg",
      "label": "cat"
    },
    {
      "labels": [
        "cat",
        "dog"
      ],
      "score": [
        7.805689392625936e-07,
        0.9999991655349731
      ],
      "key": "gs://my-bucket/blocks_ml_image_example/prediction/sample_02.jpg",
      "label": "dog"
    }
  ]
}
  • The results are contained in one JSON object.
  • "predictions": An array containing the results of the prediction. The elements of this array are objects with the following members:
    Name Explanation
    "labels"

    A list of the classification types. In this example, there are two types: "cat" and "dog".

    The order of "labels" corresponds with the order of "score".

    "score"

    The certainty for predicting each type.

    In this example, the first "score" corresponds with "cat" and the second "score" corresponds with "dog".

    "key" If you designate a key when making the prediction, it will be returned here. If you do not set a key, the GCS URL of the image will be returned.
    "label" The predicted type. This is the type that corresponds with the highest "score".
Object detection prediction results

The following is an example of prediction results for an object detection model:

{
  "predictions": [
    {
      "detection_box_xmin": [
        0.465076744556427,
        0.006737172603607178
      ],
      "detection_classes": [
        "cat",
        "dog"
      ],
      "detection_box_ymax": [
        0.7059540748596191,
        0.932410717010498
      ],
      "detection_scores": [
        0.991367757320404,
        0.7089109420776367
      ],
      "key": "gs://blocks-doc-samples/object-detection/prediction/bernese-mountain-dog-111878_1280.jpg",
      "num_detections": 2,
      "detection_box_xmax": [
        0.8645327687263489,
        0.39152249693870544
      ],
      "detection_box_ymin": [
        0.0,
        0.2869449853897095
      ]
    }
  ]
}
  • The results are output as a single JSON object.
  • "predictions": An array containing the results of the prediction. The elements of this array are objects with the following members:
Name Explanation
"key"

If you designated a key when making the prediction, it is returned here. Otherwise, the GCS URL of the image will be returned.

"num_detections"

The number of objects detected.

"detection_classes"

A list of the detected classes. In this example, "cat" and "dog" are the two classes.

The order in this list corresponds with the order of the lists in the following members:

  • "detection_scores"
  • "detection_box_xmin"
  • "detection_box_ymin"
  • "detection_box_xmax"
  • "detection_box_ymax"
"detection_scores"

The certainty for detecting each object.

The values in this list are ordered in correspondence with the list in "detection_classes". In this example, the first value is for "cat" and the second value is for "dog".

"detection_box_xmin"

The X coordinate for the upper-left corner of the boundary around each detected object.

The values are a ratio based on the image having a width of 1. You can multiply this value times the pixel-width of the image to convert into a pixel value.

The values in this list are ordered in correspondence with the list in "detection_classes". In this example, the first value is for "cat" and the second value is for "dog".

"detection_box_ymin"

The Y coordinate for the upper-left corner of the boundary around each detected object.

The values are a ratio based on the image having a height of 1. You can multiply this value times the pixel-height of the image to convert into a pixel value.

The values in this list are ordered in correspondence with the list in "detection_classes". In this example, the first value is for "cat" and the second value is for "dog".

"detection_box_xmax"

The X coordinate for the lower-right corner of the boundary around each detected object.

The values are a ratio based on the image having a width of 1. You can multiply this value times the pixel-width of the image to convert into a pixel value.

The values in this list are ordered in correspondence with the list in "detection_classes". In this example, the first value is for "cat" and the second value is for "dog".

"detection_box_ymax"

The Y coordinate for the lower-right corner of the boundary around each detected object.

The values are a ratio based on the image having a height of 1. You can multiply this value times the pixel-height of the image to convert into a pixel value.

The values in this list are ordered in correspondence with the list in "detection_classes". In this example, the first value is for "cat" and the second value is for "dog".

Properties

Property Explanation
BLOCK name Configure the name displayed on this BLOCK.
GCP service account Select the GCP service account to use with this BLOCK.
Model Generator Select the Model Generator whose applied training will be used to make the predictions.
Input variable Designate the variable that contains the input data that will be used to make a prediction.
Output variable Designate the variable that will store the prediction results.
BLOCK memos Make notes about this BLOCK.
Version used for predictions

Select whether to use the Production (current) or Testing (preview) version.

Threshold score for Object Detection Model

Designate the minimum value to be used when adopting object detection results ("detection_scores"). Only results greater (not including) than the threshold score will be adopted.

この情報は役に立ちましたか?