TD Workflow is designed to provide a scalable and flexible solution for you to manage your data pipelines in a cloud hosted environment.
Limits of Use
To ensure optimal and fair processing, the following limits exist:
- Maximum of 10 tasks running concurrently, per account. All other tasks get queued, and are issued on a first come first served basis.
- 1000 tasks (Saved Workflows). If tasks exceed 1000, not all workflows will queue for processing. They are still accessible via CLI.
- Maximum non-finished tasks in an attempt is 1000.
- Maximum attempts per account is 100.
- Period of time until a running task is killed is 1 day.
- Period of time until a running attempt is killed is 7 days.
Advised Limits of Use
For better performance of the TD Console Workflow, try to stay below the following limits:
- 200 Saved Workflows
- 400 Tasks in a Workflow
These limits are subject to change if there are changes to the configuration of Treasure Data capacity.