This article explains how to import data from MySQL to Arm Treasure Data.
Install Legacy Bulk Import
First, install the toolbelt, which includes bulk loader program, on your computer.
Downloads
After the installation, the td
command is installed on your computer. Open the terminal and type td
to execute the command. Make sure you have java
as well. Execute td import:jar_update
to download the up-to-date version of our bulk loader:
$ td usage: td [options] COMMAND [args] $ java Usage: java [-options] class [args...] $ td import:jar_update Installed td-import.jar 0.x.xx into /path/to/.td/java
Importing data from MySQL
Run the following commands to upload data from MySQL.
$ td db:create my_db $ td table:create my_db my_tbl $ td import:auto \ --auto-create my_db.my_tbl \ --format mysql \ --db-url jdbc:mysql://my_sql_host/my_sql_db \ --db-user my_user \ --db-password my_pass \ --time-column date_time \ my_sql_tbl
Because `td import:auto` executes MapReduce jobs to check the invalid rows, it'll take at least 1-2 minutes. |
In this example, we assumed that:
- the bulk import reads the input data from the my_sql_tbl on the MySQL database specified by the ‘--db-’ prefixed options
- the bulk import session is created automatically by our request using the --auto-create option.
When using the --auto-create option you must provide the database and table name in the format indicated. The session name will be ‘my_db.my_tbl’.
Alternatively you can create the bulk import session with a name of choice, using thetd import:create
command (for details see Bulk Import Internals, Import Step-by-Step). - the time field is called “date_time” and is provided with the --time-column option
You can use the following options to specify the access to the MySQL database:
MySQL specific options: --db-url URL JDBC connection URL --db-user NAME user name for MySQL account --db-password PASSWORD password for MySQL account
For further details, check the following pages:
Comments
0 comments
Please sign in to leave a comment.