Text Search Engine How-To

The Text Search Engine is currently optimized for use with Japanese text.

Introduction

This page explains how to create and test a Text Search Engine. You can use the sample data provided to do the following:

  • Learn how to create a Text Search Engine.
  • Learn how to use the included Simple Search App.

There is also an appendix titled Web API intro that explains how to use the Text Search Engine through a web API. Refer to this section if you will develop an app that uses the Text Search Engine.

What is the Text Search Engine?

With the Text Search Engine, you can easily create your own dedicated text search engine. The search engine itself is constructed within your Google Cloud Platform (GCP) environment.

Text Search Engine overview

You can submit questions to the search engine using natural language, and it will return the answers most likely to be relevant from among those you’ve registered into it.

With the Text Search Engine, you can quickly and easily build a search service for public or private use.

Features

The Text Search Engine features the following:

  • Simple to build a text search engine

    When you create a Text Search Engine in BLOCKS, it automatically creates the environment for your text search engine without the need for any technical expertise. All you need to do is specify the GCP project where the engine will be constructed and configure settings like the number/type of virtual machines that will run it.

  • Strong support for Q&A format text

    You can get highly accurate search results for Q&A format text.

    The Text Search Engine is currently optimized for use with Japanese text.

  • JSON format RESTful API

    You can access your search engine using a JSON format RESTful API. By using this API, you can develop search apps or bots that use your search engine.

  • Simple data registration

    To register the text data for your Text Search Engine, all you need to do is prepare it as a CSV text file.

  • Customizable

    You can customize your Text Search Engine with three optional dictionary files: the user-defined dictionary, synonyms dictionary, and stop-words dictionary.

    The user-defined dictionary can only be used with Japanese text.

  • Simple Search App included

    The Text Search Engine also includes the Simple Search App, which you can use to quickly test your search engine.

Testing the Text Search Engine

Let’s try actually creating a Text Search Engine and testing its results using the Simple Search App.

The basic steps will be as follows:

Overview of steps
  1. Preparing data

    We’s prepared sample text data and dictionary files for you to download and use.

  2. Creating the Text Search Engine

    Create a Text Search Engine with just a view mouse clicks.

  3. Uploading the data

    Before you can add text data to your Text Search Engine, you’ll need to upload it into Google Cloud Service (GCS). In this step, you’ll upload your text and dictionary files to GCS.

  4. Creating an index

    A single Text Search Engine can support multiple indices, each of which can support a search service. In this step, you will create an index and add your dictionary files to it.

    You cannot perform searches without creating at least one index.

  5. Importing data

    In this step, you’ll import your text data into the index you created in the last step. Once finished, you’ll be able to start using your search engine.

  6. Test searching with the Simple Search App

    You can use the included Simple Search App to test out your Text Search Engine.

Preparing data

We’ve prepared sample text data and dictionary files for you to download and use with this guide.

The data comes from World Health Organization Q&A archives: How can I avoid getting the flu? and has been formatted for use with the Text Search Engine.

Click the following link to download the sample data. It’s a ZIP file, so you’ll need to extract it after it finishes downloading.

https://storage.googleapis.com/blocks-docs/search-board-howto/english-text-search-sample.zip

Once you’ve extracted the ZIP file, you should have the following folder:

Overview of sample data folder

We’ll explain each file briefly below:

The dictionaries folder (“analysis”)

Dictionary files for the Text Search Engine need to be placed into a folder named analysis. You can name the actual dictionary files whatever you like.

The Text Search Engine supports three types of dictionary files: user-defined, synonyms, and stopwords. However, the user-defined dictionary is only for use with Japanese text and is not necessary for this guide.

  • Synonyms dictionary

    You can use this dictionary to define synonymous terms. Refer to the Text Search Engine Help document for more details.

    Contents of synonyms.txt:
    flu => influenza
    outbreaks,pandemic => epidemic
    

  • Stopwords dictionary

    With the stopwords dictionary, you can specify terms that the search engine will ignore when searching the target text. Refer to the Text Search Engine Help page for more details.

  • Contents of stopwords.txt:
    and
    or
    the
    

The sample dictionary files in this guide are for showing how to format and use these files, and not necessarily for improving the accuracy of the search results.

The target text

Text for the Text Search Engine should be divided into questions and answers with defined associations. You can also prepare the text data without dividing it as just an “answers” file.

Standard case with linked answer and question data
Example of unseparated data
Standard case Unseparated data

The associations between answer and question data can be categorized into the following types:

Association type
(answer-to-question)
Image Explanation
one-to-none
Association type one-to-none
There is no association because there is only answer data.
one-to-one
Association type one-to-one
One element of the answer data corresponds to one element in the question data and vice versa.
one-to-many
Association type one-to-many
One element in the answer data can correspond to multiple elements in the question data, but one element in the question data only corresponds to one element in the answer data.
many-to-one
Association type many-to-one
One element in the answer data only corresponds to one element in the question data, but one element in the question data can correspond to multiple elements in the answer data.
many-to-many
Association type many-to-many
One element in the answer data can correspond to multiple elements in the question data and vice versa.

The sample data for this guide has a one-to-one association between the answer and question data. The following are excerpts from the answer and question files:

Answer data excerpt (flu-answer.csv):
id,body
1,"Seasonal influenza (or “flu”) is most often caused by type A or B influenza viruses. Symptoms include sudden onset of fever, cough (usually dry), headache, muscle and joint pain, sore throat and a runny nose. The cough can be severe and can last 2 or more weeks. Most people recover from fever and other symptoms within a week without requiring medical attention. However, influenza can cause severe illness or death in high-risk groups (see Who is most at risk? below). Seasonal epidemics occur mainly during winter, from October to March in the northern hemisphere and April to September in the southern hemisphere. In tropical and subtropical countries, seasonal influenza can happen all year round."
2,"The best way to avoid getting the flu is to get the flu vaccine every year. Influenza viruses evolve constantly, and twice a year WHO makes recommendations to update the vaccine compositions. For the 2016-2017 northern hemisphere influenza season, the vaccine formulation was updated in February 2016 to contain two type A viruses (H1N1 and H3N2), and a type B virus. WHO recommends annual vaccination for high-risk groups including health care workers. People should ideally get vaccinated just before the influenza season begins for the most effective coverage, although getting vaccinated at any time during the influenza season can still help prevent flu infections."

Question data excerpt (flu-question.csv):
id,body,target_id
1,What is seasonal influenza?,1
2,How can I avoid getting the flu?,2

Refer to the following Text Search Engine Help page links for more information about how to format the data in each file:

Once you’ve finished downloading and extracting the sample data, move on to the next section for creating the Text Search Engine.

Creating the Text Search Engine

Starting from the BLOCKS dashboard, open the menu () in the global navigation bar and select Text Search Engine.

Opening the Text Search Engine from the BLOCKS dashboard

If your project doesn’t contain any Text Search Engines yet, click Start on the What is the Text Search Engine? page.

Creating the first Text Search Engine

If your project already contains at least one Text Search Engine, click Add at the top of the Text Search Engine list.

Creating an additional Text Search Engine

info_outline A message will appear if your organization does not have sufficient licenses to create the Text Search Engine. If you are an admin for your organization, you will be prompted with instructions on how to to purchase an additional license. If you are not an admin, you will be prompted to contact your admins.

Select the Text Search Type.

Selecting the type of Text Search Engine

Enter a name for your Text Search Engine and click Next.

Selecting the type of Search Engine

Users on the BLOCKS free trial or the Self-Service payment plan will see the GCP service account settings page next. You won’t see this step if you are using the Full Service Plan.

Select the GCP service account you will use with the Text Search Engine.

Selecting a GCP service account
   

For Self-Service Plan and Trial Version Users:

  • Your GCP service account must be given the Owner role.

    To give your GCP service account the owner rold, open the GCP console and go to the IAM page and change the role for the relevant service account to owner (steps 1, 2, 3 below).

    The GCP console IAM menu

Once you’ve selected your GCP service account, the Enable APIs step will appear.:

GCP service account settings

Follow the steps below if there are any API without an accompanying checkmark:

  1. Click the icon next to the API’s name to open its GCP page.

    The link to the API page

  2. Click Enable.

    Enabling the API

  3. Once the API is enabled and the button changes to Disable, close GCP and return to BLOCKS.

    Confirming that the API is enabled

Repeat these steps for any other API that are not enabled.

Click Check for the APIs you’ve enabled and wait for their checkmarks to appear.

If a checkmark does not appear, try waiting for a bit and clicking Check again. The checkmark may not appear right away in some cases.

If the checkmark still doesn’t appear, the cause may be one of the following:

  • You GCP service account has not been given the Owner role.
  • You have not enabled billing for your GCP project.

Once all of the API have been enabled, click Next.

This will bring up the optional settings page.

Optional settings (Text Search Engine)

On this screen, you can customize the specs of the virtual machine that will run the search engine and enable or disable the optional Simple Search App.

We’ll leave the machine settings as their defaults and only change the Simple Search App settings.

Enabling the Simple Search App

Click Enable GAE and Activate the Simple Search App. Enter a password for the Simple Search App and click Next.

You can click the eye () icon to toggle visibility of the password text.

This will bring up the confirmation screen where you can check all of your settings before creating the Text Search Engine.

Confirmation screen

Click Finish.

You will see the following final confirmation about creating the Text Search Engine:

Final confirmation

Click OK to create the Text Search Engine. This takes a bit of time, and you will not be able to use BLOCKS until it finishes. You will see the Text Search Engine details screen when setup completes.

After the Text Search Engine has been created

Uploading data to GCS

Before you can import data into the Text Search Engine, you’ll need to upload it into GCS first. In this section, we’ll upload the text data and dictionary files we downloaded earlier into GCS.

Upload the data into the GCS bucket shown in your Text Search Engine details screen under Resources used by this Text Search EngineCloud StorageBucket.

Confirming the bucket in GCS

This bucket is unique for each Text Search Engine, so yours will not be the same as the one shown in the above image.

You can use the GCS Explorer in BLOCKS to quickly upload the data. Open the menu () in the global navigation bar and select GCS Explorer.

Opening the GCS Explorer

Select the GCP service account associated with your Text Search Engine, then select the bucket listed on your Text Search Engine’s details screen.

Confirming the bucket in GCS

Click Upload Folder to upload the folder containing the dictionary files.

Confirming the bucket in GCS

The window for selecting your folder will appear (macOS version shown below).

Select the folder analysis from the sample data and click Upload.

Selecting the folder

Upload the question data and answer data by clicking Upload File.

Uploading files

The window for selecting your files will appear (macOS version shown below).

Select flu-answer.csv and flu-question.csv, then click Upload.

Selecting the folder

You should see the files in the list once they finish uploading.

Finished uploading

Creating an index

You can create multiple search services within one Text Search Engine by creating an index for each. In this section, you’ll create an index and import the sample text data and dictionary files into it.

The index screen

Find the Index list section of your Text Search Engine details screen and click Create Index.

Creating an index

Enter a name for your index using only lowercase letters, numbers, or hyphens (-). Names can only start with a letter or number. For example, we used the name flu-faq for this guide.

For the dictionary files, click its folder icon to open the GCS File Selector.

Select the appropriate dictionary file and click Select.

Selecting the dictionary files

Enter the following files for each dictionary:

User-defined None
Synonyms synonyms.txt
Stopwords stopwords.txt

Your screen should look similar to the following once you have finished selecting your files.

After dictionary files have been selected

Click Create to create the index.

Your index should appear in the index list with the status Creating index.

Creating the index

It takes a bit of time to create the index. The status will change to Index created once it finishes.

Index created

Importing data

In this section, you will import the sample text data for your search engine into your index.

Click Import data.

Importing data

Click the folder icon for each data file. Click the bucket you uploaded your data into, then select the appropriate file and click Select.

Importing data

Select the following files from the sample data:

Answer data flu-answer.csv
Question data flu-question.csv

Once you’ve selected your files, your screen should look similar to the following:

After selecting the data to import

Click Import to start importing the data into your index.

Your index list should look like the following:

Importing

It takes a bit of time for the data to be imported into the index. The status will change from Importing to Imported once it finishes, and your index list should look like the following:

Imported

Test searching with the Simple Search App

Once you’ve finished importing your data, we’ll test the Text Search Engine’s searching capabilities using the Simple Search App that comes with it.

Opening the Simple Search App

Click Open for the Simple Search App and an authentication page will appear.

Opening the Simple Search App

Enter the following username and password and click Log In.

Username demo
Password The password you configured when creating the Text Search Engine.

The Simple Search App will open.

The Simple Search App main page

To use the Simple Search App, just enter your search text into the box and click Search. Search results will appear below the search button.

You can adjust the limit to the left of the search button to set a maximum number of results to be returned. You can select 5, 10, 20, or 30 for the limit.

Selecting a results limit

The following example shows results for searching with the term "who is most at risk for the flu?".

Simple Search App search results

Tips for using the Text Search Engine

As long as you have data, it’s simple to create a search engine service with the Text Search Engine.

However, just creating a Text Search Engine isn’t all it takes to build an effective text search service.

Here are three tips to help you build a more effective text search service with your Text Search Engine:

  • Tip #1: Keep each text entry focused a single topic.

    Your search engine's accuracy decreases if the text entries in the data contain info for multiple topics.

    If your entries contain more than one theme, try separating them into different entries focusing on a single theme each.

  • Tip #2: Reexamine any short text entries.

    Short text entries have a tendency to be missed by the search engine or to not show up in the top results.

    Short text entries in the question or answer data often lack the words that would distinguish them, and as a result fail to show up as search results when you might expect them to.

    In these cases, you may be able to improve your results by editing or adding in distinguishing words into such entries.

  • Tip #3: Integrate evaluations of search results

    Try adding a question like "Was this result helpful?" to the app you develop. If an answer is chosen as helpful, add a link between the question and answer in your data.

    By doing this, the answer you linked will show up higher in the search results the next time a similar question is asked.

Web API intro

The example API calls used in this section use Japanese text for the sample data files found in the Japanese version of this guide.

You can access the Text Search Engine through a JSON format RESTful API. By using this API, you can develop applications or bots that use your search engine.

This appendix will briefly explain the 14 types of API calls that are currently publically available. You can refer to this section when developing an app that uses the Text Search Engine.

The following chart explains how to read <> enclosed parts of the API descriptions:

<IP address:Port number> The IP address and port number that is shown in your Text Search Engine details screen under Connection information.
<Index name> The target index name for the API call.
<Answer ID> The ID of the answer text.
<Question ID> The ID of the question text.

The "curl command examples" from the API explanations are issued from virtual machines in the same network that the search engine is constructed in.

1. Search for answers from questions (answer data only)

Overview

Send a question. Responds with the answer data closest to the question text in order of relevance.

Target: A:Q association type is one-to-none (answer data only)

Endpoint <IP address:Port number>/<Index name>/target,hint/_search
Method GET
Example request
api_search_1_0
Example response
{
  "took": 12,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "failed": 0
  },
  "hits": {
    "total": 32,
    "max_score": 4.0507264,
    "hits": [
      {
        "_index": "influenza-faq",
        "_type": "target",
        "_id": "7",
        "_score": 4.0507264,
        "_source": {
          "body": " インフルエンザは、地域によって時期は異なりますが、世界中で流行が見られます。一般的には、温帯地方では冬季(南半球では6~9月)に流行が見られます。熱帯・亜熱帯地方では国や地域により様々で、年間を通じて低レベルの発生が見られる地域や、複数回流行する地域もあります。流行するウイルスの種類は地域によって差はありますが、大きく違いません。世界における流行状況は、WHOのホームページ等で知ることができます。\n\n世界保健機関(WHO):Influenza updates(インフルエンザ最新情報)→http://www.who.int/influenza/surveillance_monitoring/updates/en/",
          "target_id": "7"
        }
      },
      {
        "_index": "influenza-faq",
        "_type": "target",
        "_id": "5",
        "_score": 3.4223063,
        "_source": {
          "body": " 平成21(2009)年4月に新型インフルエンザA(H1N1)2009ウイルスがメキシコで確認され、世界的大流行となり、我が国でも多くの人々が免疫を持っていなかったため、同年秋季を中心に大規模な流行となりました。発生後、一年余で約2千万人が罹患したと推計されましたが、入院患者数は約1.8万人、死亡者は203人であり、死亡率は0.16(人口10万対)と、諸外国と比較して低い水準にとどまりました。翌年には、新型インフルエンザA(H1N1)2009ウイルスに加え、A香港型やB型のインフルエンザウイルスも流行しており、季節性インフルエンザとは異なる時期に大きな流行が発生する等の特別な状況は確認されませんでした。\n\n このような状況を踏まえ、厚生労働省は、平成23(2011)年3月31日の時点において「新型インフルエンザ」と呼ばれていたインフルエンザA(H1N1)2009ウイルスについて、季節性インフルエンザとして取り扱うこととし、対応も季節性インフルエンザの対策に移行しました。",
          "target_id": "5"
        }
      },
      {
        "_index": "influenza-faq",
        "_type": "target",
        "_id": "13",
        "_score": 3.0334578,
        "_source": {
          "body": " 毎年、日本では、国立感染症研究所と全国の地方衛生研究所が中心となってタミフルやリレンザなどの抗インフルエンザウイルス薬に耐性をもつウイルスの調査を行っています。詳しくは国立感染症研究所のホームページを御覧下さい\n\n国立感染症研究所:抗インフルエンザ薬剤耐性株サーベイランス→http://www.nih.go.jp/niid/ja/influ-resist.html\n\n 抗インフルエンザウイルス薬に耐性化したウイルスが検出される割合は、1~4%程度です。これらのウイルスのほとんどは、抗インフルエンザウイルス薬にて治療を行った後、採取されたウイルスです。\n\n 2013/2014年インフルエンザ流行シーズン当初、札幌で相次いで確認されたタミフルに耐性を持つウイルスは、タミフルでの治療を行っていない患者から検出されました。患者間での接触はなかったと判断されていますが、ウイルスの遺伝子が非常に似ているため、タミフルに耐性を持つウイルスが札幌市内で同時期に流行していた可能性が高いと考えられています。\n\n 一般的に抗インフルエンザウイルス薬に耐性を持ったウイルスは、伝播するスピードが遅いため広く流行することなく、自然に消失します(詳しくは国立感染症研究所にて掲載しています\n\n国立感染症研究所:IASR<速報>2013/14シーズンに札幌市で検出された抗インフルエンザ薬耐性A(H1N1)pdm09ウイルス→http://www.nih.go.jp/niid/ja/flu-m/flu-iasrs/4232-pr4081.html\n\n しかし、2008/2009年インフルエンザ流行シーズンにヨーロッパで出現した、タミフルに耐性化したソ連型A(H1N1)ウイルスが世界的に流行したことから、今後も注意が必要です。",
Example curl command
curl -X GET 10.128.0.3:9200/influenza-faq/target,hint/_search -H 'Content-Type:application/json' --data '{ "size": 5, "query": { "match": { "body": "インフルエンザが流行する時期はいつか" } } }'
Notes

2. Search for answer IDs from questions (A:Q is one-to-many)

Overview

Send a question. Returns the IDs ("target_id") of the answers most closely related to the question in order of relevance.

Target: A:Q association type is one-to-many

Endpoint <IP address:Port number>/<Index name>/target,hint/_search
Method GET
Example request
api_search_1_m
Example response
{
  "took": 94,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "failed": 0
  },
  "hits": {
    "total": 61,
    "max_score": null,
    "hits": [
      {
        "_index": "influenza-faq",
        "_type": "target",
        "_id": "7",
        "_score": 5.2207584,
        "_source": {
          "target_id": "7"
        },
        "fields": {
          "target_id": [
            "7"
          ]
        }
      },
      {
        "_index": "influenza-faq",
        "_type": "target",
        "_id": "5",
        "_score": 4.3226757,
        "_source": {
          "target_id": "5"
        },
        "fields": {
          "target_id": [
            "5"
          ]
        }
      },
      {
        "_index": "influenza-faq",
        "_type": "target",
        "_id": "13",
        "_score": 3.787116,
        "_source": {
          "target_id": "13"
        },
        "fields": {
          "target_id": [
            "13"
          ]
        }
      }
    ]
  }
}
Example curl command
curl -X GET 10.128.0.3:9200/influenza-faq/target,hint/_search -H 'Content-Type:application/json' --data '{ "_source": {"excludes": ["body"] }, "size": 3, "query": { "match": { "body": "インフルエンザが流行する時期はいつか" } }, "collapse": { "field": "target_id" } }'
Notes

3. Search for answer IDs from questions (A:Q is one-to-many)

Overview

Send a question. Returns the IDs ("target_id") of the answers most closely related to the question in order of relevance.

Target: A:Q association type is many-to-many

Endpoint <IP address:Port number>/<Index name>/target,hint/_search
Method GET
Example request
api_search_m_m
Example response

{
  "took": 11,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "failed": 0
  },
  "hits": {
    "total": 63,
    "max_score": 13.286041,
    "hits": [
      {
        "_index": "influenza-faq",
        "_type": "hint",
        "_id": "10",
        "_score": 13.286041,
        "_source": {
          "body": "インフルエンザにかかったらどうすればよいのですか?",
          "target_ids": [
            "10",
            "14",
            "15"
          ]
        }
      },
      {
        "_index": "influenza-faq",
        "_type": "hint",
        "_id": "9",
        "_score": 10.997316,
        "_source": {
          "body": "インフルエンザにかからないためにはどうすればよいですか?",
          "target_ids": [
            "9"
          ]
        }
      },
      {
        "_index": "influenza-faq",
        "_type": "hint",
        "_id": "17",
        "_score": 8.525941,
        "_source": {
          "body": "インフルエンザにかかったら、どのくらいの期間外出を控えればよいのでしょうか?",
          "target_ids": [
            "17"
          ]
        }
      }
    ]
  }
}
Example curl command
curl -X GET 10.128.0.3:9200/influenza-faq/target,hint/_search -H 'Content-Type:application/json' --data '{ "size": 3, "query": { "match": { "body": "インフルエンザにかかったらどうすればよいのですか?" } } }'
Notes
  • This is the same as “1. Search for answers from questions (answer data only)”.
  • The same answer ID ("target_id") may be returned multiple times. You will need to manage processing them uniquely on the application side.

4. Get a specific answer

Overview Send an answer ID. Returns the text for that answer.
Endpoint <IP address:Port number>/<Index name>/target/
Method GET
Example request (There’s no request body)
Example response
{
  "_index": "influenza-faq",
  "_type": "target",
  "_id": "10",
  "_version": 1,
  "found": true,
  "_source": {
    "body": "(1) 具合が悪ければ早めに医療機関を受診しましょう。\n\n(2) 安静にして、休養をとりましょう。特に、睡眠を十分にとることが大切です。\n\n(3) 水分を十分に補給しましょう。お茶でもスープでも飲みたいもので結構です。\n\n(4) 咳やくしゃみ等の症状のある時は、周りの方へうつさないように、不織布製 マスクを着用しましょう。\n\n(5) 人混みや繁華街への外出を控え、無理をして学校や職場等に行かないようにしましょう。\n\n また、小児、未成年者では、インフルエンザの罹患により、急に走り出す、部屋から飛び出そうとする、ウロウロと歩き回る等の異常行動を起こすおそれがあるので、自宅において療養を行う場合、少なくとも発症から2日間、小児・未成年者が一人にならないよう配慮しましょう(Q14、15を参照)。",
    "target_id": "10"
  }
}
Example curl command
curl -X GET 10.128.0.3:9200/influenza-faq/target/10
Notes

5. Get questions related to answers (A:Q is one-to-many)

Overview

Send an answer ID. Returns the questions related to that answer.

Target: A:Q association is one-to-many

Endpoint <IP address:Port number>/<Index name>/hint/_search
Method GET
Example request
api_hint_search_1_m
Example response
{
  "took": 44,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "failed": 0
  },
  "hits": {
    "total": 1,
    "max_score": 3.2580965,
    "hits": [
      {
        "_index": "influenza-faq",
        "_type": "hint",
        "_id": "10",
        "_score": 3.2580965,
        "_source": {
          "body": "インフルエンザにかかったらどうすればよいのですか?",
          "target_id": "10"
        }
      }
    ]
  }
}
Example curl command
curl -X GET 10.128.0.3:9200/influenza-faq/hint/_search -H 'Content-Type:application/json' --data '{ "query": { "term": { "target_id": "10" } } }'
Notes

6. Get questions related to answers (A:Q is many-to-many)

Overview

Send answer ID. Returns the data for the related questions.

Target: A:Q association is many-to-many

Endpoint <IP address:Port number>/<Index name>/hint/_search
Method GET
Example request
api_hint_search_1_m
Example response
{
  "took": 3,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "failed": 0
  },
  "hits": {
    "total": 1,
    "max_score": 1.4505018,
    "hits": [
      {
        "_index": "influenza-faq",
        "_type": "hint",
        "_id": "10",
        "_score": 1.4505018,
        "_source": {
          "body": "インフルエンザにかかったらどうすればよいのですか?",
          "target_ids": [
            "10",
            "14",
            "15"
          ]
        }
      }
    ]
  }
}
Example curl command
curl -X GET 10.128.0.3:9200/influenza-faq/hint/_search -H 'Content-Type:application/json' --data '{ "query": { "term": { "target_ids": "10" } } }'
Notes

7. Add or update answers

Overview Adds or updates answer data.
Endpoint <IP address:Port number>/<Index name>/target/<Answer ID>
Method POST
Example request
api_target_add
Example response
  • When adding data:
    {
      "_index": "influenza-faq",
      "_type": "target",
      "_id": "33",
      "_version": 1,
      "result": "created",
      "_shards": {
        "total": 1,
        "successful": 1,
        "failed": 0
      },
      "created": true
    }
    
  • When updating data:
    {
      "_index": "influenza-faq",
      "_type": "target",
      "_id": "33",
      "_version": 3,
      "result": "updated",
      "_shards": {
        "total": 1,
        "successful": 1,
        "failed": 0
      },
      "created": false
    }
    
Example curl command
curl -X POST 10.128.0.3:9200/influenza-faq/target/33 -H 'Content-Type:application/json' --data '{ "body": "回答データを追加するサンプル文章です。", "target_id": 33 }'
Notes

8. Add or update questions (A:Q is one-to-many)

Overview

Adds or updates question data.

Target: A:Q association is one-to-many

Endpoint <IP address:Port number>/<Index name>/hint/<Answer ID>
Method POST
Example request
api_target_add
Example response
  • When adding data:
    {
      "_index": "influenza-faq",
      "_type": "hint",
      "_id": "33",
      "_version": 1,
      "result": "created",
      "_shards": {
        "total": 1,
        "successful": 1,
        "failed": 0
      },
      "created": true
    }
    
  • When updating data:
    {
      "_index": "influenza-faq",
      "_type": "hint",
      "_id": "33",
      "_version": 2,
      "result": "updated",
      "_shards": {
        "total": 1,
        "successful": 1,
        "failed": 0
      },
      "created": false
    }
    
Example curl command
curl -X POST 10.128.0.3:9200/influenza-faq/hint/33 -H 'Content-Type:application/json' --data '{ "body": "質問データを追加するサンプル文章です。", "target_id": 33 }'
Notes

9. Add or update questions (A:Q is many-to-many)

Overview

Adds of updates question data.

Target: A:Q ratio is many-to-many

Endpoint <IP address:Port number>/<Index name>/hint/<Answer ID>
Method POST
Example request
api_hint_add_m_m
Example response
  • When adding data:
    {
      "_index": "influenza-faq",
      "_type": "hint",
      "_id": "33",
      "_version": 1,
      "result": "created",
      "_shards": {
        "total": 1,
        "successful": 1,
        "failed": 0
      },
      "created": true
    }
    
  • When updating data:
    {
      "_index": "influenza-faq",
      "_type": "hint",
      "_id": "33",
      "_version": 1,
      "result": "updated",
      "_shards": {
        "total": 1,
        "successful": 1,
        "failed": 0
      },
      "created": false
    }
    
Example curl command
curl -X POST 10.128.0.3:9200/influenza-faq/hint/33 -H 'Content-Type:application/json' --data '{ "body": "質問データを追加するサンプル文章です。", "target_ids": [1, 16, 32] }'
Notes

10. Add answers related to questions

Overview

Add an association between an answer ID and question data.

Endpoint <IP address:Port number>/<Index name>/hint/<Question ID>/_update
Method POST
Example request
api_hint_id_add
Example response
{
  "_index": "influenza-faq",
  "_type": "hint",
  "_id": "5",
  "_version": 2,
  "result": "updated",
  "_shards": {
    "total": 1,
    "successful": 1,
    "failed": 0
  }
}
Example curl command
curl -X POST 10.128.0.3:9200/influenza-faq/hint/5/_update -H 'Content-Type:application/json' --data '{ "script": "ctx._source.target_ids.add(\"20\")" }'
Notes

11. Update question or answer body text

Overview

Update question or answer text.

Endpoint
  • When updating question text:

    <IP address:Port number>/<Index name>/hint/<Question ID>/_update

  • When updating answer text:

    <IP address:Port number>/<Index name>/target/<Answer ID>/_update

Method POST
Example request
api_hint_update
Example response
{
  "_index": "influenza-faq",
  "_type": "hint",
  "_id": "5",
  "_version": 3,
  "result": "updated",
  "_shards": {
    "total": 1,
    "successful": 1,
    "failed": 0
  }
}
Example curl command
curl -X POST 10.128.0.3:9200/influenza-faq/hint/5/_update -H 'Content-Type:application/json' --data '{ "doc": { "body": "これは質問文章を更新するサンプル文章です。" } }'
Notes

12. Delete specific question or answer data

Overview

Designate question or answer IDs. It deletes the data for the specified IDs.

Method DELETE
Endpoint
  • When deleting question data:

    <IP address:Port number>/<Index name>/hint/<Question ID>

  • When deleting answer data:

    <IP address:Port number>/<Index name>/target/<Answer ID>

Method DELETE
Example request (There is no request body)
Example response
{
  "found": true,
  "_index": "influenza-faq",
  "_type": "hint",
  "_id": "33",
  "_version": 2,
  "result": "deleted",
  "_shards": {
    "total": 1,
    "successful": 1,
    "failed": 0
  }
}
Example curl command
curl -X DELETE 10.128.0.3:9200/influenza-faq/hint/33
Notes

13. Delete all question or answer data

Overview

Deletes all question or answer data.

Endpoint
  • When deleting all question data:

    <IP address:Port number>/<Index name>/hint/_delete_by_query

  • When deleting all answer data:

    <IP address:Port number>/<Index name>/target/_delete_by_query

  • Method POST
    Example request
    {
      "query": {
        "match_all": {}
      }
    }
    
    Example response
    {
      "took": 72,
      "timed_out": false,
      "total": 32,
      "deleted": 32,
      "batches": 1,
      "version_conflicts": 0,
      "noops": 0,
      "retries": {
        "bulk": 0,
        "search": 0
      },
      "throttled_millis": 0,
      "requests_per_second": -1,
      "throttled_until_millis": 0,
      "failures": []
    }
    
    Example curl command
    curl -X POST 10.128.0.3:9200/influenza-faq/hint/_delete_by_query -H 'Content-Type:application/json' --data '{ "query": { "match_all": {} } }'
    
    Notes

    14. Add question or answer data in bulk

    Overview

    Adds question or answer data in bulk.

    Endpoint
    • When adding question data:

      <IP address:Port number>/<Index name>/hint/_bulk

    • When adding answer data:

      <IP address:Port number>/<Index name>/target/_bulk

    Method POST
    Example request
    • Question data:
      api_hint_request

      To add associations to multiple answers, change "target_id" to"target_ids"

    • Answer data:

      Formatting is the same as the question data above, but:
      Read Question ID as Answer ID and question text as answer text.

    Example response
    {
      "took": 37,
      "errors": false,
      "items": [
        {
          "index": {
            "_index": "influenza-faq",
            "_type": "hint",
            "_id": "1",
            "_version": 1,
            "result": "created",
            "_shards": {
              "total": 1,
              "successful": 1,
              "failed": 0
            },
            "created": true,
            "status": 201
          }
        },
        {
          "index": {
            "_index": "influenza-faq",
            "_type": "hint",
            "_id": "2",
            "_version": 1,
            "result": "created",
            "_shards": {
              "total": 1,
              "successful": 1,
              "failed": 0
            },
            "created": true,
            "status": 201
          }
        }
      ]
    }
    
    Example curl command
    curl -X POST 10.128.0.3:9200/influenza-faq/hint/_bulk -H 'Content-Type:application/x-ndjson' --data-binary @sample.json
    
    Notes