You use input transfers to transfer data in bulk from a data source into Arm Treasure Data. You can transfer data from a number of sources.
Managing Input Transfer
After you create a data connector, you configure the data connection, Then you can create input transfers that specify the specific source and also where in Treasure Data to put the coming data.
Connections Console Overview
You can create and manage your input transfers from the
Connections section of the TD Console – at https://console.treasuredata.com/app/connections/
|Source Catalog||You can explore many of the bulk and streaming connection methods from the source catalog in the TD Console. Go to this page to setup a new authentication method for a particular source.|
|My Connections||A list of all the authenticated Connections you’ve created in your Treasure Data account. You can also create and schedule new Input Transfers from Connections on here.|
|My Input Transfers||A list of all the input transfers. You can edit and delete input transfers from here. The input transfer page is empty until you have create a New Transfer on the My Connections page.|
Create an Input Transfer
After creating a connection, you are automatically taken to the My Connections tab. Look for the connection you created and click New Transfer.
Complete the fields for your connector, processing through each tab of the
Create Transfer window by clicking Next. The required field vary based on the type of data connector and the scheduling that you want to implement.
On the last tab of the
Create Transfer window, click Start Transfer to save and start the data transfer.
You will see the new data transfer in progress listed under the
My Input Transfers tab.
You also can see a corresponding job listed in the
Jobs section of the console.
Running an Input Transfer
Jobs are kicked off when you click Start Transfer which is on the last pane of the
Create Transfer window.
You can either:
- Click New Transfer for a data connector from My Connections
- Click on the more-ellipse (“…”), and under the dropdown click Edit in the
My Input Transferpane for a data connector
Either way, you must complete the fields in each tab of the
Create Transfer window, and click Start Transfer in order to start the run of a data transfer job.
Viewing an Input Transfer as a Job
After completing the configuration of an input transfer by clicking Start Transfer on the last pane of the
Create Transfer window, go to the
Jobs section of the console.
You see a list of jobs that you have run most recently. The most recent job run appears at the top.
A job status can be one of the following:
- Success – The job was successful and the data transfer was completed
- Error – There was an error with the job and the data transfer was not successful
- Running – The job is still running
- Killed – The job was stopped while running and did not complete
- JobId – The unique identifier for the job. If you contact Support with an issue, you will be asked to provide the job ID.
Click on your job to see additional information.
- Query – Shows the configuration used to complete the data connector job
- Output Logs – A detailed logs that captures the events of the data connector job
- Details – A summary details about the job completion
Canceling an Input Transfer
You can cancel an input transfer by:
- Editing the schedule in the data connector listed in the Connections >My Input Transfer file page. You can remove or change the time that the input transfer is scheduled to execute.
- Killing the job that is executing the input transfer. See Viewing an Input Transfer as a Job.
Examples of Configuring and Running Input Transfers
You might want to see how to run an input transfer from the console by reviewing the information for specific data connectors. For example:
Other Ways to Transfer Data