You use Sources to transfer data in bulk from a data source into Arm Treasure Data. You can transfer data from a number of sources. 

After you create a data connector, which includes authenticating your connection so that you data access is secure, you configure the data connection. You can create Sources that specify the source type and also where in Treasure Data to put the data.

You can create and manage your Sources from the Sources section of the TD Console.

Outside of the TD Console, other data transfer options include REST API for Postback, Mobile SDK, JS SDK.

Creating a Source

1. Open TD Console.
2. Navigate to Integrations > Catalog.
3. Optionally, use the filters to refine the catalog type.
   
4. Select one of the options listed. For example, Salesforce.
5. Complete the fields for your Integration type. The required fields vary based on the type of data connector. This first part of creating a data connection is authentication. Authentication is specifying a secure method for connecting to your data source. 
   
6. Click Continue and Create Connection to complete the dialog. You now have an authentication object. If you would like to share this connection with other users in your organization, check the Share with others checkbox. If this box is unchecked this connection is visible only to you.

Editing an Existing Connection to Specify a Specific Source

1. Navigate to Integrations Hub > Authentications
2. Click on the connection you want to edit.
3. The Edit Authentications dialog opens. Edit the relevant information. This is the second part of creating a data connection. You are specifying the source and schedule for data transfer into Treasure Data.
4. Click Continue and Done to complete the dialog. You now have a source object.

Transferring Data from a Source to Treasure Data

Data is transferred to Treasure Data using a job. Jobs are started when a transfer runs.

1. Navigate to Integrations > Sources.
2. Locate and select the Source for which you want to transfer data. Or, select the more-ellipse (“…”) from the Sources section, and select Edit.
   
3. Complete the fields in each page of the popup window. If necessary, supply the transfer details such as aggregation period, review the preview data, add the Transfer To details, and indicate the frequency with which you would like the transfer to run.
4. Click Start Transfer to run the data transfer job (now or according to the schedule you defined).

You can used the Jobs or the Sources section to monitor the progress of your data transfer.

Viewing the Data Transfer Status and Output Logs

The Jobs page of TD Console allows you to view and monitor all your TD jobs.  A job status can be one of the following:

• Success – The job was successful and the data transfer was completed
• Error – There was an error with the job and the data transfer was not successful
• Running – The job is still running
• Killed – The job was stopped while running and did not complete
• JobId – The unique identifier for the job. If you contact Support with an issue, you are asked to provide the job ID.

1. Navigate to Jobs.
   
2. Optionally, filter the jobs page.
3. Review your jobs and their status.
4. Select your job to see additional information. For example:
   
5. Review the information on the following job tabs:
   Query – Shows the configuration used to complete the data connector job
   Output Logs – A detailed logs that captures the events of the data connector job
   Details – A summary details about the job completion

Canceling the Data Transfer

You can cancel data transfers using the schedule definition of the data connector or by using the jobs page.

Editing the schedule in the data connector

You can remove or change the time that the input transfer is scheduled to execute.

1. Navigate to Integrations > Sources.
2. Select the ellipses.
3. Select Edit.
   
4. Edit the Schedule.

Using the jobs page

1. Navigate to Jobs.
2. Filter for the job that you want to stop.
3. Select the job.
   
4. Select Kill.  

Defining Specific Data Sources

Each of the many TD data connectors is described in the Importing and Exporting Data category. For example:

• Salesforce Source
• Salesforce Marketing Cloud
• Facebook Page Insights