This is a summary of new Arm Treasure Data features and improvements introduced in the June 1st, 2019 release. If you have any product feature requests, submit them to feedback.treasuredata.com.
New Feature Available: Policies
Increase your level of control and security by using Treasure Data's Policies feature. Policies make it easier to manage user access to features and data. Policies combine permissions together. You can configure multiple policies and assign one or more policies to one or more users.
Contact your Customer Success representative at firstname.lastname@example.org for more information.
Improvements in Defining Segment Rules
The operator labels in the segment rule editor have been updated to clarify the logic and reordered for quicker access. Also you can click on the '+' button to select multiple values when defining a segment rule.
Segments: Expanded Time Window
The Time Window range for your segment rules is greater. The upper limit for each time value -Day, Hour, Minute and Second - is 86400. For example. you can define a segment rule that collects data within a time window of 4380 Hours (which is equivalent to .5 year). Or, you might want to define a rule with a time window of 150 days.
Beta Import and Export: Salesforce DMP (Krux) - Improvement
You can import dissent list data from the Salesforce DMP (Krux) platform, or export segments from Treasure Data to Salesforce DMP (Krux). For more information, see Salesforce DMP (Krux) - Import or Salesforce DMP (Krux) - Export.
Contact your Customer Success representative at email@example.com to a participate in the beta.
Beta: Salesforce V2
Salesforce data connector, V2 is available as a Beta. The V2 connector is more efficient, capable of asynchronous data transfer, with minimal impact to current users.
Current Salesforce data connector users must complete migration steps to move from the legacy Salesforce data connector to Salesforce V2. New users of the Salesforce data connector can start using Salesforce data connector, V2 immediately.
After the beta period, all TD users of the Salesforce data connector will be encouraged to migrate to the improved data connector.
Contact your Customer Success representative at firstname.lastname@example.org to learn more.
Export: Google BigQuery - More export modes
When exporting results into Google BigQuery, you have more modes to choose from:
- Append: append your target table with new data. This option existed in the previous version of the connector.
- Replace: replace your existing target table with new data and new structure
- Replace Backup: backup your existing target table before replacing with the new one
- Delete: delete the existing target table and recreate a new one
For more information, see Writing Job Results into Google BigQuery.
Query Engines: Select from Different Releases
For more convenience when upgrading your Hive or Presto releases, Treasure Data provides a drop -down list of supported query engine versions. You can see new TD Hive and TD Presto releases along with earlier releases, and can choose when choose a newer release as your preferred default engine.
The engine release numbering is based on release year and sequence number: for example:
- 2019 features Hive releases 2019.1, 2019.2, etc. and Presto releases 2019.1, 2019.2 etc.
Previous releases, such as Hive 0.13 and Presto 0.205, follow the previous release number convention.
You select a query engine and version on the query editor page. You can see available versions of each engine, with the current default for the customer's account in bold:
You can also select engine versions on a per-query basis in Console, Workflow, command line and TD API.
You can view both new and older releases in one list so that you can test and validate new engine releases locally or on your own workloads before committing to migrate most or all of your production jobs to the new engine release.
Control Panel: TD Console Navigation Change to Benefit Administrators
The control panel groups administrator-level account maintenance features together in the Treasure Data console user interface.
The improved navigation is available now or you can wait until late July when the navigation change is implemented on all Treasure Data systems. Contact your Customer Success representative at email@example.com to activate the improved navigation now.
Some task steps in the administrative area and user profile area are impacted by this user interface change. You can review a summary of changes.
You can view a video walkthru of control panel: .
Query Engines: TD Hive 2019.1 Ready to Release
TD Hive 2019.1 is a major update to Hive-on-TD functionality. It brings compatibility with Apache Hive 2.3 to the Arm Treasure Data platform. Production release of TD Hive 2019.1 is expected this month, June 2019.
As with our current production release, Hive 0.13, TD Hive 2019.1 will continue to generate MapReduce jobs. These jobs will execute on the same cluster and using the same execution resources as current Hive 0.13 jobs. Performance is similar, and Hive 2019.1 fixes all known outstanding for greater reliability.
Future TD Hive releases will build on TD Hive 2019.1 to add a number of performance-oriented capabilities, such as:
- Capacity Scheduler, which ensures that no job is ultimately starved for resources
- the Tez runtime, which supersedes MapReduce and promises substantial performance improvement
- Vectorized implementations of many complex functions, to allow for vectorized execution across a query
- Full user-defined partitioning support in Hive, comparable to Presto
Most queries from Hive 0.13 will run in Hive 2019.1 unchanged. Some queries will need minor modifications to use ANSI SQL syntax because Hive 2019.1 is stricter.
NOTE: Hive 0.13 queries will be supported for an extended period (>12 months). While we recommend the new Hive releases for future workloads, you have time to migrate your existing workloads, as needed by your business.
Arm Treasure Data Console - Deprecations of Previous Versions
We improved the look of the TD Console to make it more intuitive and ensure that the platform features are easier for you to access. The new TD Console is now generally available. You can no longer access old TD Consoles, including the previous, separately accessed console-cdp.treasuredata.com.