1. Home
  2. Cloud
  3. Guides
  4. Tours
  5. Concepts
  6. Tools
    1. Alert Editor
    2. Analyzer
    3. Avro Extractor Tool
    4. Business Schema Designer
    5. Business Schema Manager
    6. CMC Logs Manager
    7. CMC Monitoring
    8. CMC Scheduler
    9. CMC Tenant Manager
    10. Content Manager
    11. Dashboard Filters Manager
    12. Dashboard Manager
    13. Data Agent
    14. Data Lineage Viewer
    15. Data Lineage Viewer v2
    16. Data Manager
    17. Formula Builder
    18. Inspector Tool
    19. Join Editor
    20. Load Job Details Viewer
    21. Load Job Viewer
    22. Load Plan DAG Viewer
    23. Model Update Viewer
    24. Notebook Editor
    25. Profile Manager
    26. Public API
    27. Query Plan Viewer
    28. Scheduler
    29. Schema Designer
    30. Schema Diagram Viewer
    31. Schema Manager
    32. Schema Wizard
    33. Security Manager
    34. Table Editor
    35. Upload Service Command Line Interface (CLI)
  7. References
  8. Integrations
  9. Data applications
  10. Releases

Tools → Load Plan DAG Viewer

About the Load Plan DAG Viewer

The Load Plan DAG Viewer allows schema managers to view a directed acyclic graph (DAG) of the load plan execution. The graph displays objects in the load plan, their dependencies, and order of execution. This tool provides a visual representation of the data loading process per load group.

The Load Plan DAG Viewer is available as a preview feature starting with the 2023.7.0 release.

Accessing the Load Plan DAG Viewer

The Super User (tenant administrator) or a user that belongs to a group with the Schema Manager or the SuperRole roles can access the Load Plan DAG Viewer of load plans that they can view in the Scheduler > Load Plans.

To open the Load Plan DAG Viewer:

  1. On the navigation bar, select Scheduler > Load Plans.
  2. For the load plan you want, select Show DAG.

Anatomy of the Load Plan DAG Viewer

The Load Plan DAG Viewer consists of the following:

  • Load Groups panel
  • Action bar
  • Search and Zoom bar
  • Overview window
  • Diagram canvas

Load Groups panel

In the case of a load plan with multiple load groups, the Viewer shows a panel that includes all the load groups of the plan. By default, the Viewer shows the DAG of the first load group. Select another load group to view its DAG. You can show or hide this panel.

Starting with 2024.1.0, this part is available for all load plans: single-group and multi-group. The panel shows the included schemas and their objects and you can search and filter the group panel by the schema or object name. Selecting a schema or object from this panel filters the DAG diagram to show only the tasks and nodes for the selected schema or object and also filters the diagram legend to show the related task or node types only.

Action bar

The action bar shows:

  • The name of the load plan.
  • An X icon that you can select to close the Viewer.

Search and Zoom bar

The search and zoom bar contains the following:

  • Load group name
  • Search box
  • Hide Grouping (toggle to hide or show task groupings)
  • Zoom In (magnifying glass with a plus sign icon)
  • Zoom Out (magnifying glass with a minus sign icon)
  • Fit Diagram (four-corner icon)

The Search box and the Hide Grouping toggle are available starting with 2024.1.0. You can use the Search box to search the diagram by the object name. The Search box shows matching objects categorized by task type. The search is limited to the current diagram; therefore, it is affected by the filters applied from the group panel. Selecting an object from the search list highlights the object in the respective node and also highlights it along with its downstream tasks in all expanded nodes across the diagram, showing the full path of the selected object.

Overview window

The Overview window provides a high-level view of the entire graph. Use the Overview window to focus on a specific part of the DAG. Drag the light-blue rectangle or change its width and height to set or change the focus.

You can change the size of the Overview window, or you can collapse it.

  • To increase the size of the Overview window, select maximize (diverging arrows icon).
  • To decrease the size of the Overview window, select minimize (converging arrows icon).
  • To collapse the Overview window, select v.
  • To expand the Overview window, select <.

Diagram canvas

The diagram canvas shows the load plan execution DAG. In the case of a multi-group load plan, it shows the execution plan of one group at a time. In the DAG, nodes represent the tasks or stages of the load plan, and these tasks are grouped by their type and order of execution. The lines between groups or nodes indicate the order of execution or the dependency between nodes. You can expand each group to view the tasks included and their dependencies. There is a legend key that indicates the stage or task type by color.

Turning on the Hide Grouping toggle expands all nodes and hides their groupings. Turning it off shows the groupings without collapsing the nodes. Grouped nodes show the number of tasks included.

The DAG shows nodes for the following stages or tasks:

  • Extraction: Tables that the Loader Service reads from source.
  • Enrichment: Materialized views that Spark processes.
  • PK-Index: Objects that have key columns defined and require PK-Index calculation.
  • Compaction: Objects that require deduplication or compaction.
  • Loading: Objects that the Loader Service loads into the Engine memory. A separate node is available for objects with load filters.
  • Calculations: Objects and processes that require calculations, such as formulas, joins, and derived tables. Each of these processes has its own nodes.
  • Locking: The transition between the running and the Load/ Post-load calculations.