crp-order-qty-opt-service - Details for Query 34168

Submitted Time: 2026/07/21 20:47:17
Duration: 0.4 s
Succeeded Jobs: 46514

Plan Visualization

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { subgraph cluster0 { isCluster="true"; id="cluster0"; label="WholeStageCodegen (2)\n \nduration: total (min, med, max (stageId: taskId))\n72 ms (0 ms, 0 ms, 14 ms (stage 1312502.0: task 2447098))"; tooltip="WholeStageCodegen (2)"; 1 [id="node1" labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build total (min, med, max (stageId: taskId)) 10 ms (0 ms, 0 ms, 4 ms (stage 1312502.0: task 2447107)) peak memory total (min, med, max (stageId: taskId)) 7.5 MiB (256.0 KiB, 256.0 KiB, 256.0 KiB (stage 1312502.0: task 2447105)) number of output rows: 1 number of sort fallback tasks: 0 avg hash probes per key: 0" tooltip="HashAggregate(keys=[shardId#5509166], functions=[])"]; } 2 [id="node2" labelType="html" label="Exchange shuffle records written: 1 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 1 local bytes read: 92.0 B merged fetch fallback count: 0 local blocks read: 1 remote merged chunks fetched: 0 remote blocks read: 0 data size: 32.0 B local merged bytes read: 0.0 B local merged chunks fetched: 0 shuffle write time: 0 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 fetch wait time: 0 ms remote bytes read: 0.0 B number of partitions: 30 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 92.0 B" tooltip="Exchange hashpartitioning(shardId#5509166, 30), ENSURE_REQUIREMENTS, [plan_id=2855155]"]; subgraph cluster3 { isCluster="true"; id="cluster3"; label="WholeStageCodegen (1)\n \nduration: total (min, med, max (stageId: taskId))\n173 ms (0 ms, 4 ms, 56 ms (stage 1312501.0: task 2447082))"; tooltip="WholeStageCodegen (1)"; 4 [id="node4" labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build total (min, med, max (stageId: taskId)) 72 ms (0 ms, 0 ms, 55 ms (stage 1312501.0: task 2447082)) peak memory total (min, med, max (stageId: taskId)) 7.5 MiB (256.0 KiB, 256.0 KiB, 256.0 KiB (stage 1312501.0: task 2447072)) number of output rows: 1 number of sort fallback tasks: 0 avg hash probes per key: 0" tooltip="HashAggregate(keys=[shardId#5509166], functions=[])"]; 5 [id="node5" labelType="html" label=" Project " tooltip="Project [shardId#5509166]"]; 6 [id="node6" labelType="html" label="Filter number of output rows: 2" tooltip="Filter (isnotnull(shardId#5509166) AND (shardId#5509166 = 0_1_10000001445))"]; 7 [id="node7" labelType="html" label="Scan ExistingRDD number of output rows: 2" tooltip="Scan ExistingRDD[shardId#5509166,orderProposalId#5509167L,items#5509168,optimizationGroups#5509169,isOptimal#5509170,lastModified#5509171,optimizationInsights#5509172]"]; } 2->1; 4->2; 5->4; 6->5; 7->6; }

Details

== Physical Plan ==
* HashAggregate (6)
+- Exchange (5)
   +- * HashAggregate (4)
      +- * Project (3)
         +- * Filter (2)
            +- * Scan ExistingRDD (1)


(1) Scan ExistingRDD [codegen id : 1]
Output [7]: [shardId#5509166, orderProposalId#5509167L, items#5509168, optimizationGroups#5509169, isOptimal#5509170, lastModified#5509171, optimizationInsights#5509172]
Arguments: [shardId#5509166, orderProposalId#5509167L, items#5509168, optimizationGroups#5509169, isOptimal#5509170, lastModified#5509171, optimizationInsights#5509172], MapPartitionsRDD[387754] at start at FileStorageAdapterImpl.java:598, ExistingRDD, UnknownPartitioning(0)

(2) Filter [codegen id : 1]
Input [7]: [shardId#5509166, orderProposalId#5509167L, items#5509168, optimizationGroups#5509169, isOptimal#5509170, lastModified#5509171, optimizationInsights#5509172]
Condition : (isnotnull(shardId#5509166) AND (shardId#5509166 = 0_1_10000001445))

(3) Project [codegen id : 1]
Output [1]: [shardId#5509166]
Input [7]: [shardId#5509166, orderProposalId#5509167L, items#5509168, optimizationGroups#5509169, isOptimal#5509170, lastModified#5509171, optimizationInsights#5509172]

(4) HashAggregate [codegen id : 1]
Input [1]: [shardId#5509166]
Keys [1]: [shardId#5509166]
Functions: []
Aggregate Attributes: []
Results [1]: [shardId#5509166]

(5) Exchange
Input [1]: [shardId#5509166]
Arguments: hashpartitioning(shardId#5509166, 30), ENSURE_REQUIREMENTS, [plan_id=2855155]

(6) HashAggregate [codegen id : 2]
Input [1]: [shardId#5509166]
Keys [1]: [shardId#5509166]
Functions: []
Aggregate Attributes: []
Results [1]: [shardId#5509166]

SQL / DataFrame Properties

Name	Value
spark.activity.hdlfsEnabled	true
spark.activity.isolateSession	true
spark.databricks.delta.lastCommitVersionInSession	7
spark.sql.adaptive.enabled	false
spark.sql.cbo.enabled	false
spark.sql.parquet.fieldId.read.enabled	true
spark.sql.parquet.fieldId.write.enabled	true
spark.sql.requireAllClusterKeysForDistribution	false
spark.sql.shuffle.partitions	30
temporal.activity.id	ad458d20-d9c7-36d3-8134-433c400366bc
temporal.activity.type	RunStreamWithId