The hash filter plugin for Arm Treasure Data's Data Connectors allows you to specify the hashing algorithm on certain columns.

Options

This plugin contains the following option:

Option	Description
column_options	a key-value pairs. Key is a column name and Value is for column options.

Child Elements	Description
algorithm	The algorithm to digest. Can be: MD5, SHA-1, or SHA-256 (string, required)
salt	The salt value. When present, "salt value + original string" is digested (string, optional)
salt_column	The column that stores the salt value. When present, "salt column's value + original string" is digested (string, optional)
rename_to	Used to change the column name (string, optional)
charset_to_encode_strings	The character set encoding name. Used to encode strings in the column that your Java Runtime supports (string, default: UTF-8).

Configuration

Add the filter section, including the type, to your load.yml. In the following example, the type is digest:

in:
...
filters:
- type: digest
  column_options:
    id:
      algorithm: MD5
    name:
      algorithm: SHA-1
      rename_to: name_digest
    email:
      algorithm: SHA-256
      salt: salt_value
    phone:
      algorithm: SHA-266
      salt_column: col1
out:
...

Hash Filter Plugin for Data Connectors AWS US AWS Tokyo IDCF

Options

Configuration

Comments

Options

Configuration

Related articles