Treasure Data Bulk Import API

The samples shown here are used to access your data through the Treasure Data REST API.

Description	API call (or REST call)
enables bulk importing of data to the targeted database and table and stores it in the default resource pool	bulk_import/create/:name/:database_name/:table_name
returns the list of available bulk imports	bulk_import/list
upload a file to the bulk import session. Files must be in msgpack.gz format.	bulk_import/upload_part
returns the list of the available parts uploaded through the bulk import under the specified name	bulk_import/list_parts/:name
executes a job to perform bulk import with the indicated priority	bulk_import/perform/:name
commits the bulk import information having the specified name	bulk_import/commit/:name
deletes the imported information with the specified name	bulk_import/delete/:name
shows the details of the bulk import with the specified name	bulk_import/show/:name
freezes the bulk import with the specified name	bulk_import/freeze/:name
unfreezes bulk_import with the specified name	bulk_import/unfreeze/:name

v3/bulk_import/create/:name/:database_name/:table_name

This operation enables bulk importing of data to the targeted database and table and stores it in the default resource pool.
Default expiration for bulk import is 30 days.

Verb	URI
POST	`/v3/bulk_import/create/:name/:database_name/:table_name(.:format)`

URI Parameters

Parameter Name	Description
`name`	Name of the bulk import
`database_name`	Name of the target database
`table_name`	Name of the target table

Sample Request Status 200 OK

Sample Request

curl -X POST \
  https://api.treasuredata.com/v3/bulk_import/create/test_bulk_import/test_db/table_a1 \
  -H 'Authorization: TD1 <API Key>' \
  -H 'cache-control: no-cache,no-cache'

Sample Response


{
    "name": "test_bulk_import",
    "bulk_import": "test_bulk_import"
}

Sample Request Status 404 Not Found

Sample Request

curl -X POST \
  https://api.treasuredata.com/v3/bulk_import/create/test_bulk_import/test_db_abc/table_xyz \
  -H 'Authorization: TD1 <API Key>' \
  -H 'cache-control: no-cache,no-cache'

Sample Response

{ 
    "error": "Resource not found",
    "message": "Resource not found",
    "text": "Resource not found",
    "severity": "error",
    "status_code": 404
}

v3/bulk_import/upload_part/table_name/:name

This operation add files to be imported to the bulk import session having the specified name. Name can be identifier you use for that part of the upload.

Verb	URI
PUT	`/v3/bulk_import/upload_part/table_name/:name`

Sample Request

curl -X PUT \
  https://api.treasuredata.com/v3/bulk_import/upload_part/test_bulk_import/:name \
  -H 'Authorization: TD1 <API Key>' \
  -H 'cache-control: no-cache,no-cache'

Sample Response

{
    "bulk_import": "test_bulk_import",
    "name": "test_bulk_import"
}

v3/bulk_import/commit/:name

This operation commits the bulk import information having the specified name.

Verb	URI
POST	`/v3/bulk_import/commit/:name`

URI Parameters

Parameter Name	Description
`name`	name of the bulk import

Sample Request Status 200 OK

Sample Request

curl -X POST \
  https://api.treasuredata.com/v3/bulk_import/commit/test_bulk_import \
  -H 'Authorization: TD1 <API Key>' \
  -H 'cache-control: no-cache,no-cache'

Sample Response

{
    "name": "thyra_test_bulk_import",
    "bulk_import": "thyra_test_bulk_import"
}

Sample Request Status 404 Not Found

Sample Request

curl -X POST \
  https://api.treasuredata.com/v3/bulk_import/commit/xyz \
  -H 'Authorization: TD1 <API Key>' \
  -H 'cache-control: no-cache,no-cache'

Sample Response

{
    "error": "Resource not found",
    "message": "Resource not found",
    "text": "Resource not found",
    "severity": "error",
    "status_code": 404
}

v3/bulk_import/delete/:name

This operation deletes the imported information with the specified name.

Verb	URI
POST	`/v3/bulk_import/delete/:name(.:format)`

URI Parameters

Parameter Name	Description
`name`	name of the bulk import

Sample Request Status 200 OK

Sample Request

curl -X POST \
  https://api.treasuredata.com/v3/bulk_import/delete/thyra_test_bulk_import \
  -H 'Authorization: TD1 <API Key>' \
  -H 'cache-control: no-cache,no-cache'

Sample Response

{
    "name": "thyra_test_bulk_import",
    "bulk_import": "thyra_test_bulk_import"
}

Sample Request Status 404 Not Found

Sample Request

curl -X POST \
  https://api.treasuredata.com/v3/bulk_import/delete/import_abc \
  -H 'Authorization: TD1 <API Key>' \
  -H 'cache-control: no-cache,no-cache'

Sample Response

{
    "error": "Resource not found",
    "message": "Resource not found",
    "text": "Resource not found",
    "severity": "error",
    "status_code": 404
}

v3/bulk_import/freeze/:name

This operation freezes the bulk import with the specified name.

Verb	URI
POST	`/v3/bulk_import/freeze/:name(.:format)`

URI Parameters

Parameter Name	Description
`name`	name of the bulk import

Sample Request Status 200 OK

Sample Request

curl -X POST \
  https://api.treasuredata.com/v3/bulk_import/freeze/test_bulk_import \
  -H 'Authorization: TD1 <API Key>' \
  -H 'cache-control: no-cache,no-cache'

Sample Response

{
    "name": "test_bulk_import",
    "bulk_import": "test_bulk_import"
}

Sample Request Status 404 Not Found

Sample Request

curl -X POST \
  https://api.treasuredata.com/v3/bulk_import/freeze/import_abc \
  -H 'Authorization: TD1 <API Key>' \
  -H 'cache-control: no-cache,no-cache'

Sample Response

{
    "error": "Resource not found",
    "message": "Resource not found",
    "text": "Resource not found",
    "severity": "error",
    "status_code": 404
}

v3/bulk_import/list

This operation returns the list of available bulk imports.

Verb	URI
GET	`/v3/bulk_import/list(.:format)`

Sample Request Status 200 OK

Sample Request

curl -X POST \
  https://api.treasuredata.com/v3/bulk_import/list
  -H 'Authorization: TD1 <API Key>' \
  -H 'cache-control: no-cache,no-cache'

Sample Response

{
    "bulk_imports": [
        {
            "name": "bulk_import_test_session_0_2_1_0",
            "valid_records": 5,
            "error_records": 0,
            "valid_parts": 1,
            "error_parts": 0,
            "status": "committed",
            "upload_frozen": true,
            "database": "tdtests_0_2_1_0",
            "table": "bulk_import_test",
            "job_id": "50696647"
        },
        {
            "name": "bulk_import_test_session_0_2_1_1",
            "valid_records": 5,
            "error_records": 0,
            "valid_parts": 1,
            "error_parts": 0,
            "status": "committed",
            "upload_frozen": true,
            "database": "tdtests_0_2_1_1",
            "table": "bulk_import_test",
            "job_id": "50611306"
        },
           :
           :
           :
    ]
}

v3/bulk_import/list_parts/:name

This operation returns the list of the available parts uploaded through /v3/bulk_import/upload_part/:name/:part under the specified name.

Verb	URI
GET	`/v3/bulk_import/list_parts/:name(.:format)`

URI Parameters

Parameter Name	Description
`name`	name of the bulk import

Sample Request Status 200 OK

Sample Request

curl -X GET \
  https://api.treasuredata.com/v3/bulk_import/list_parts/test_bulk_import \
  -H 'Authorization: TD1 <API Key>' \
  -H 'cache-control: no-cache,no-cache'

Sample Response

{
    "parts": [],
    "name": "test_bulk_import",
    "bulk_import": "test_bulk_import"
}

Sample Request Status 404 Not Found

Sample Request

curl -X GET \
  https://api.treasuredata.com/v3/bulk_import/list_parts/upload_abc \
  -H 'Authorization: TD1 <API Key>' \
  -H 'cache-control: no-cache,no-cache'

Sample Response

{
    "error": "Resource not found",
    "message": "Resource not found",
    "text": "Resource not found",
    "severity": "error",
    "status_code": 404
}

v3/bulk_import/perform/:name

This operation executes a job to perform bulk import with the indicated priority using the resource pool if indicated, else it will use the account's default.

Verb	URI
POST	`/v3/bulk_import/perform/:name(.:format)`

URI Parameters

Parameter Name	Description
`name`	name of the bulk import

Sample Request Status 200 OK

Sample Request

curl -X POST \
  https://api.treasuredata.com/v3/bulk_import/perform/test_bulk_import \
  -H 'Authorization: TD1 <API Key>' \
  -H 'cache-control: no-cache,no-cache'

Sample Response

{
    "name": "thyra_test_bulk_import",
    "bulk_import": "thyra_test_bulk_import",
    "job_id": 50707565
}

Sample Request Status 404 not found

Sample Request

curl -X POST \
  https://api.treasuredata.com/v3/bulk_import/perform/upload_abc \
  -H 'Authorization: TD1 <API Key>' \
  -H 'cache-control: no-cache,no-cache'

Sample Response

{
    "error": "Resource not found",
    "message": "Resource not found",
    "text": "Resource not found",
    "severity": "error",
    "status_code": 404
}

v3/bulk_import/show/:name

This operation shows the details of the bulk import with the specified name

Verb	URI
GET	`/v3/bulk_import/show/:name(.:format)`

URI Parameters

Parameter Name	Description
`name`	name of the bulk import

Sample Request Status 200 OK

Sample Request

curl -X GET \
  https://api.treasuredata.com/v3/bulk_import/show/test_bulk_import \
  -H 'Authorization: TD1 <API Key>' \
  -H 'cache-control: no-cache,no-cache'

Sample Response

{
    "name": "test_bulk_import",
    "status": "ready",
    "job_id": 50707565,
    "valid_records": 0,
    "error_records": 0,
    "valid_parts": 0,
    "error_parts": 0,
    "upload_frozen": false,
    "database": "test_db",
    "table": "table_a1"
}

Sample Request Status 404 not found

Sample Request

curl -X GET \
  https://api.treasuredata.com/v3/bulk_import/show/upload_abc \
  -H 'Authorization: TD1 <API Key>' \
  -H 'cache-control: no-cache,no-cache'

Sample Response

{
    "error": "Resource not found",
    "message": "Resource not found",
    "text": "Resource not found",
    "severity": "error",
    "status_code": 404
}

v3/bulk_import/unfreeze/:name

This operation unfreezes bulk_import with the specified name.

Verb	URI
POST	`/v3/bulk_import/unfreeze/:name(.:format)`

URI Parameters

Parameter Name	Description
`name`	name of the bulk import

Sample Request Status 200 OK

Sample Request

curl -X POST \
  https://api.treasuredata.com/v3/bulk_import/unfreeze/test_bulk_import \
  -H 'Authorization: TD1 <API Key>' \
  -H 'cache-control: no-cache,no-cache'

Sample Response

{
    "name": "test_bulk_import",
    "bulk_import": "thyra_test_bulk_import"
}

Sample Request Status 404 not found

Sample Request

curl -X POST \
  https://api.treasuredata.com/v3/bulk_import/unfreeze/upload_abc \
  -H 'Authorization: TD1 <API Key>' \
  -H 'cache-control: no-cache,no-cache'

Sample Response

{
    "error": "Resource not found",
    "message": "Resource not found",
    "text": "Resource not found",
    "severity": "error",
    "status_code": 404
}

Treasure Data Bulk Import API

v3/bulk_import/create/:name/:database_name/:table_name

Sample Request Status 200 OK

Sample Request Status 404 Not Found

v3/bulk_import/upload_part/table_name/:name

v3/bulk_import/commit/:name

Sample Request Status 200 OK

Sample Request Status 404 Not Found

v3/bulk_import/delete/:name

Sample Request Status 200 OK

Sample Request Status 404 Not Found

v3/bulk_import/freeze/:name

Sample Request Status 200 OK

Sample Request Status 404 Not Found

v3/bulk_import/list

Sample Request Status 200 OK

v3/bulk_import/list_parts/:name

Sample Request Status 200 OK

Sample Request Status 404 Not Found

v3/bulk_import/perform/:name

Sample Request Status 200 OK

Sample Request Status 404 not found

v3/bulk_import/show/:name

Sample Request Status 200 OK

Sample Request Status 404 not found

v3/bulk_import/unfreeze/:name

Sample Request Status 200 OK

Sample Request Status 404 not found

Was this helpful?