BLOCKS Reference

Block Reference

BigQuery

Convert to partitioned table

This BLOCK copies data from multiple tables separated by date (date-sharded tables) into one partitioned table

Convert to partitioned table BLOCK chart

The date-sharded tables must have their names formatted as shown below:

tablename_YYYYMMDD
  • YYYY: 4-digit year. E.g. 2016
  • MM: 2-digit month (01–12). E.g. 04
  • DD: 2-digit date (01–31). E.g. 09

For example, file names like sharded_20160401 and mytable_20161231 are properly formatted with the required year, month, and date in their suffixes.

Property name Explanation
BLOCK name Designate a name to be displayed on this BLOCK.
GCP service account Select the GCP service account to use with this BLOCK.
Source dataset

Designate the dataset containing the tables to be converted into a partitioned table.

Source table prefix

Designate the prefix of the source tables. Tables with this prefix followed by an 8-digit date (YYYYMMDD) suffix will be copied and converted into the partitioned table.

For example, the prefix for tables named sharded_20160401, sharded_20161231, etc., would be sharded_.

Destination dataset

Designate the ID of the dataset that will contain the partitioned table.

Destination table

Designate an ID for the partitioned table.

Overwrite Specify whether or not to erase pre-existing views and tables with identical names when saving newly added ones.
Data storage period Designate the number of days each partition of data will be saved.
BLOCK memos Make any comments regarding this BLOCK.