BLOCKS Reference

SNS

Search tweets

This BLOCK is currently in beta and will become unavailable post official release. Please use the official version once released.

As a beta release, there is the possibility that some functions may not execute properly. We appreciate contact from users regarding bugs or ways to improve BLOCKS.

The beta version of this BLOCK is only available to a limited number of users. If you would like to use this BLOCK, please request access by contacting us.

Overview

This BLOCK searches Twitter for keywords and outputs the results as a new-line delimited JSON text file to Google Cloud Storage (GCS).

The following is an example of the results:

{"created_at":"Thu Jan 03 14:39:30 +0000 2019","id":123,"id_str":"123","text":"***",...}
{"created_at":"Thu Jan 03 14:36:45 +0000 2019","id":456,"id_str":"456","text":"***",...}
{"created_at":"Thu Jan 03 14:33:26 +0000 2019","id":789,"id_str":"789","text":"***",...}

Each row of the results is formatted as a JSON object and can be stored into BigQuery as-is. Each JSON object represents one tweet returned by your search.

To store your search results into BigQuery, please download and use this BigQuery schema file (search-tweets-bigquery-schema.json).

For details about the contents of the tweet JSON objects, please refer to Twitter’s documentation page: Tweet objects .

This BLOCK does not output information for details of attached images or links, user profile pictures, sources of retweets, or sources of quotations.

Depending on the search results, this BLOCK may consume a large amount of memory. Repeated executions resulting in large amounts of search results may put you over the Flow Designer memory usage limit and make you unable to edit your Flow Designer or cause Flow execution to stop.

To use this BLOCK, you must visit theTwitter Developer page and apply for using the Twitter API and applications.

Properties

Property Explanation
BLOCK name

Configure the name displayed on this BLOCK.

GCP service account

Select the GCP service account to use with this BLOCK.

Consumer Key

Enter your API Key for using the Twitter API (found under Keys and tokens, Consumer API keys).

Consumer Secret

Enter your API secret key for using the Twitter API (found under Keys and tokens, Consumer API keys).

Access Token

Enter your Access token for using the Twitter API (found under Keys and tokens, Access token & access token secret).

Access Token Secret

Enter your Access token secret for using the Twitter API (found under Keys and tokens, Access token & access token secret).

Destination GCS URL

Designate the GCS URL that will contain the outputted new-line delimited JSON file of the search results.

Search terms

Enter the search terms you want to search for in Twitter. Separate keywords with spaces.

You can also use various operators to narrow down your search results. For details, refer to Twitter’s official Rules and filtering documentation page.

The following is an example that used search operators.

#machinelearning since:2019-01-01_00:00:00_JST until:2019-01-03_23:59:59_JST
Result type

Select the type of tweets you want included in your search results from the following:

  • Both recent and most popular tweets
  • Recent tweets only
  • Most popular tweets only
BLOCK memos

Make notes about this BLOCK.

Display Twitter API rate limit status

Select whether or not to output information about your Twitter API rate limit status in the logs for this BLOCK.

The following is an example of these logs (line 3):

[2019-01-10 16:52:28.504]  info : Job-5 start
[2019-01-10 16:52:28.533]  info : Search tweets step start
[2019-01-10 16:52:29.022]  info : Twitter API rate limit. reset: 2019-01-10 17:03:23, limit: 180, remaining:178.
[2019-01-10 16:52:30.685]  info : Search tweets step finished
[2019-01-10 16:52:30.713]  info : Job-5 finished