Transwarp is committed to building enterprise-level big data infrastructure software, providing enterprises with infrastructure software and supporting around the whole data lifecycle to build a data world of the future.
Distributed and scalable, providing millions of tasks per day
Transwarp WorkfIow is a scheduling platform for data tasks that helps users design complete business processes,
set execution times, define interdependence between tasks, and automatically schedule the set processes through a graphical interface. And the cancellation of time-out tasks and retry of erroneous tasks are approved. It can support a distributed scheduling system of millions of tasks per day.
8 Reasons to Choose Workflow
The distributed scheduling engine with scalable performance can support stable scheduling work at the level of 100,000 task streams per day.
With the canvas drag-and-drop design, the entire task flow scheduling design and the configuration of each task can be completed.
The Workflow supports multiple types of task scheduling,including data development tasks (SQL tasks, scripting tasks), data integration tasks (data synchronization, data loading) and data quality checking tasks, etc.
Flexible scheduling cycles
The daily scheduling can be simply configured, and the complex and flexible scheduling cycles can also be configured through corn expressions.Furthermore, users can schedule and execute tasks through external triggering to adapt to more real-world scenarios.
Fair priority scheduling strategies
Through multi-scheduling queue design with proportional resource allocation, the problem of starvation of low priority tasks caused by high priority tasks occupying all resources for a long time is avoided. Thus scheduling tasks according to priority, fair priority scheduling is realized.
Decentralized distributed architecture
With the distributed task scheduling and execution realized by multi-scheduling + multi-execution nodes, it supports online expansion and contraction of scheduling and execution nodeswith zero interruption to existing business, meeting the requirements of high availability, load balancing and horizontal expansion.
High concurrency and low latency
Through lock-free design, dynamic concurrency control, and resource pooling control, it reduces the task’s overhead in the scheduling process, decreases scheduling delays, and improves system throughput.It can support the scheduling capacity of millions of tasks per day by combining the scheduling capacity that supports horizontal scaling.
Highly stable and fault-tolerant design
It adds monitoring of database, network and other resources that depends on service, and automatically retries when the database or network is jittery or unavailable for a short period of time, under the premise of guaranteeing business idem-potency, so as to guarantee the stable operation of the system for a long time under a high-load environment.