This article will explain how to use Python bindings for REST API.
- Basic knowledge of Arm Treasure Data, including the Toolbelt.
- A table with some data. An example is provided in the Getting Started guide.
- Basic knowledge of our query language.
- Python 3.5+
The Python bindings are released on PyPI as
td-client (stands for ‘T’reasure ‘D’ata). You can install the package from
pip install td-client
The source code is available at github.
List Databases and Tables
The example below lists the databases and tables. The API key is your authentication key. Please refer here to retrieve your API key.
import os import tdclient apikey = os.getenv("TD_API_KEY") with tdclient.Client(apikey) as client: for db in client.databases(): for table in db.tables(): print(table.db_name) print(table.table_name) print(table.count)
The example below issues a query from a Python program. The query API is asynchronous — you can check for query completion by polling the job periodically (e.g. by issuing job.finished? calls).
import os import tdclient apikey = os.getenv("TD_API_KEY") with tdclient.Client(apikey) as client: job = client.query("sample_datasets", "SELECT COUNT(1) FROM www_access") # sleep until job's finish job.wait() for row in job.result(): print(row)
|job.result() does not put the job result into memory. It iterates through the rows in a streaming fashion.|
If you would like to get result’s schema, you need to call
job.result_schema after job finished.
List and Get the Status of Jobs
The example below lists and gets the status of jobs.
import os import tdclient apikey = os.getenv("TD_API_KEY") with tdclient.Client(apikey) as client: # recent 20 jobs len(client.jobs()) # recent 127 jobs of specific status client.jobs(0, 127, "running") client.jobs(0, 127, "success") client.jobs(0, 127, "error") client.jobs(0, 127, "killed") # get job status client.job(job_id) # get job result client.job_result(job_id)