BLOCKS Reference

Block Reference

BigQuery

Load to table from variable

This BLOCK enters data from variables into BigQuery tables.

Property name Explanation
BLOCK name Designate a name for BLOCKS. The names will be displayed on the BLOCKS.
GCP service account Select the GCP service account for use with this BLOCK.
Source data variables Specify the variables that will reference the data being written.
Destination data set

ID of data set to be written

[% format character string addressable] [variable expansion addressable]

Destination table Designate an ID for the destination table.
[% format character string addressable] [variable expansion addressable]
Schema settings

Designate schema for the destination table. When importing in from a JSON format file, it is possible to skip this step.

*When you click the "Edit using JSON" link, you can specify a JSON format character string directly.

In cases of non-empty tables

Select an action to perform when tables are available for data.

  • Additions: Load additional data.
  • Overwrite: Empty tables and write.
  • Error: If tables are not empty, operations will fail.
BLOCK memos Make any comments regarding BLOCKS.
Repetition attempts Designate the number of repetitions to occur in the case of an error or time out in BigQuery.
Minimum duration for time out Indicate how many seconds to wait for a response (time out) in BigQuery. It will double with every repetition. The maximum value for the time increase between repetitions is set under 'Maximum duration for time out" in the properties.
Maximum duration for time out As time outs increase from the "Minimum duration for time out", they reach a maximum value. It is defined in units of seconds.
File format

Choose a file format from those in GCS. Permissible formats are as follow.

  • CSV
  • NEWLINE_DELIMITED_JSON
  • DATASTORE_BACKUP
CSV delimitation characters

When a file format is CSV, choose, or specify, delimitation characters. Characters that can be used for delimitation are as follow.

  • Comma
  • Tab
  • Pipe
  • Other

If you choose "Other", you can specify the character you wish to use for delimitation in the field next to "Other".

Number of skipped rows When files are in CSV format, you can define the number of lead rows to be skipped.
Permit rows with insufficient fields When files are in CSV format, you can select whether or not to permit rows with insufficient fields.
Designate quotation marks Define the characters to be used for quotation when using CSV file format.
Include quoted character strings in new rows When files are in CSV format, you can select whether or not to have rows with character strings enclosed in quotes be included in subsequent rows.
Number of permissible bad records Define the number of bad records to be permitted. When this limit is reached, the next bad record will result in failure to read.
Ignore extra fields Define whether or not to ignore excess fields.