The remove_columns filter plugin for Arm Treasure Data's Data Connector expands columns having json into multiple columns. This filter takes five options as explained below:
| Option | type | description | required? |
|---|---|---|---|
| remove | array | An array of names of columns that it removes from schema. | [] by default |
| keep | array | An array of names of columns that it keeps in schema. | [] by default |
| accept_unmatched_columns | boolean | If true, skip columns that aren’t included in schemas. | false by default |
Configuration
Add the filter section with type remove_columns to your load.yml, for example:
in: ... # This configuration removes "_c0" and "_c1" named columns from schema. filters: ... - type: remove_columns remove: ["_c0", "_c1"] out: ...
Comments
0 comments
Please sign in to leave a comment.