The remove_columns
filter plugin for Arm Treasure Data's Data Connector expands columns having json into multiple columns. This filter takes five options as explained below:
Option | type | description | required? |
---|---|---|---|
remove | array | An array of names of columns that it removes from schema. | [] by default |
keep | array | An array of names of columns that it keeps in schema. | [] by default |
accept_unmatched_columns | boolean | If true, skip columns that aren’t included in schemas. | false by default |
Configuration
Add the filter
section with type remove_columns
to your load.yml
, for example:
in: ... # This configuration removes "_c0" and "_c1" named columns from schema. filters: ... - type: remove_columns remove: ["_c0", "_c1"] out: ...
Comments
0 comments
Please sign in to leave a comment.