BLOCKS Reference

Output specifications

Video analysis

Overview

This page explains the results returned by the Video analysis BLOCK.

Video analysis results

If successful, the BLOCK outputs the results of the video analysis into the variable designated in the Results variable property.

The following is an example of the results as expressed in JSON format. The "..." and {...} parts indicate where contents have been omitted.

{
  "@type": "...",
  "annotationResults": [{
    "inputUri": "...",
    "segmentLabelAnnotations": [{...}],
    "shotLabelAnnotations": [{...}],
    "frameLabelAnnotations": [{...}],
    "explicitAnnotation": {...}
  }]
}
Name Format Explanation
"@type" String

A string indicating that these are video analysis results.

"annotationResults" Array

An array made up of objects ({...}) for each element.

Each object contains data for one video.

"inputUri" String

The GCS location of the video file.

"segmentLabelAnnotations" Array

Detected labels for the entire video or on a segment level.

The array is made up of objects ({...}) for each element.

Each object contains information for a detected label.

"shotLabelAnnotations" Array

Detected labels at the shot-level.

The array is made up of objects ({...}) for each element.

Each object contains information for a detected label.

"frameLabelAnnotations" Array

Detected labels at the frame-level.

The array is made up of objects ({...}) for each element.

Each object contains information for a detected label.

"explicitAnnotation" Object

Information for whether adult content was detected in any frames.

LabelAnnotation

Information for the labels detected in the video.

{
  "entity": {...},
  "categoryEntities": [{...}],
  "segments": [{...}],
  "frames": [{...}]
}
Name Format Explanation
"entity" Object

The detected entity (dog, flower, car, etc.).

"categoryEntities" Array

The category of the detected entity. For example, if the detected entity is labeled as "Terrier", this category is probably "dog". A single entity can have more than one category. For example, "Terrier" could also have the category "pet".

The array is made up of objects ({...}) for each element.

Each object contains the category information for the detected entity. Data formatting is the same as in "entity".

"segments" Array

Information for the segments in which the entity was detected.

The array is made up of objects ({...}) for each element.

Each object contains information for a segment in which the label was detected.

"frames" Array

Information for the frames in which the label was detected.

The array is made up of objects ({...}) for each element.

Each object contains information for a frame in which the label was detected.

Entity

Information for the detected entities (including their category information).

{
  "entityId": "...",
  "description": "...",
  "languageCode": "..."
}
Name Format Explanation
"entityId" String

The entity ID. Many ID can be used with the Google Knowledge Graph Search API open_in_new.

"description" String

A textual description of the entity. For example, "desk" or "furniture".

"languageCode" String

The language code in IETF BCP-47 format of the "description".

LabelSegment

Label detection results for the video at the segment level.

{
  "segment": {...},
  "confidence": 0.123456
}
Name Format Explanation
"segment" Object

Information for the segment in which the label was detected.

"confidence" Number

The confidence level (range 0 to 1) that the label is accurate. A higher number shows more confidence.

VideoSegment

Information for the video segment.

{
  "startTimeOffset": "12.345678s",
  "endTimeOffset": "34.567890s"
}
Name Format Explanation
"startTimeOffset" String

Shows the time-offset (in seconds) to the start of the segment relative to the beginning of the video. The string ends with an s.

"endTimeOffset" String

Shows the time-offset (in seconds) to the end of the segment relative to the beginning of the video. The string ends with an s.

LabelFrame

Information for labels detected on the frame level.

{
  "timeOffset": "12.345678s",
  "confidence": 0.123456
}
Name Format Explanation
"timeOffset" String

Shows the time-offset (in seconds) to the frame relative to the beginning of the video. The string ends with an s.

"confidence" Number

The confidence level (range 0 to 1) that the label is accurate. A higher number shows more confidence.

ExplicitContentAnnotation

Information for the likelihood of adult content existing in frames. Results are not returned for frames that don’t have possible adult content detected in them.

{
  "frames": [{...}]
}
Name Format Explanation
"frames" Array

Contains information for all frames that were detected to possibly contain adult content.

The array is made up of objects ({...}) for each element.

Each object contains information for a frame that may contain adult content.

ExplicitContentFrame

Frame-level information for adult content detection.

{
  "timeOffset": "12.345678s",
  "pornographyLikelihood": "..."
}
Name Format Explanation
"timeOffset" String

Shows the time-offset (in seconds) to the frame relative to the beginning of the video. The string ends with an s.

"pornographyLikelihood" String

The likelihood that there is adult content in the frame.

Likelihood
Likelihood Explanation
"LIKELIHOOD_UNSPECIFIED"

Unspecified likelihood.

"VERY_UNLIKELY"

Very low possibility of adult content.

"UNLIKELY"

Low possibility of adult content.

"POSSIBLE"

There may be adult content.

"LIKELY"

There is a high possibility of adult content.

"VERY_LIKELY"

There is a very high possibility of adult content.