Bvh&1dZddlZddlZedZd dZdZddZddZdZ d Z e d k(rMeZ e d e dd lmZee j e j"Ze ee ZeD] Ze e yy)zThis profiler result processor print out the kernel time spent on each Node of the model. Example of importing profile result file from onnxruntime_perf_test: python profile_result_processor.py --input profile_2021-10-25_12-02-41.json N)ScanLoopIfctj}|jdddtd|jddtdd |jd dtd d |jd ddd|j d|jdddd|j d|j |S)Nz-iz--inputFz2Set the input file for reading the profile results)requiredtypehelpz --thresholdg{Gz?zfThreshold of run time ratio among all nodes. Nodes with larger ratio will show in top expensive nodes.)rrdefaultr z --providercudazExecution provider to usez--kernel_time_only store_truez.Only include the kernel time and no fence time)ractionr )kernel_time_onlyz-vz --verbose)rr )verbose)argparseArgumentParser add_argumentstrfloat set_defaults parse_args)argvparsers [/RAG/venv/lib/python3.12/site-packages/onnxruntime/transformers/profile_result_processor.pyparse_argumentsrs  $ $ &F    A    u    (   =  / kE,O &   T ""ctd|dt|5}tj|}dddt t sJ|S#1swYxYw)Nzloading profile output z ...)printopenjsonload isinstancelist) profile_file opened_file sess_times rload_profile_jsonr&;sR #L> 67 l {IIk*   i && &   s A  Ac i}i}i}d}d}|D]}|ddk(r |ddk(rd}|s|ddk(s!d |vs&d |vs+d |d vs3|d}|d d } | tvrI| sd |d } ||vr||xx|d z cc<||xxdz cc<n|d ||<d||<| ||<||d z }|sdgSg} | jd|dzdd| jd| jdt|jddD]I\}} | |z } | |kr||} | t | z }| j| dd| dzdd| dd|dd| Ki}|jD]!\}} ||} | |vr|| xx| z cc<| || <#| jd| jd| jdt|jd dD](\} } | |z } | j| dd| dzdd| *| S)!a<Parse profile data and output nodes in two sections - nodes in the original order, and top expensive nodes. Args: sess_time (List[Dict]): profile data threshold (int, optional): Minimum ratio of duration among all. Defaults to 0. Returns: List[str]: lines of string for output. rFcatSessionnamesession_initializationTKerneldurargsop_name()zNo kernel record found!z% Top expensive kernels with Time% >= d.2f:@----------------------------------------------------------------u&Total(μs) Time% Calls Avg(μs) Kernelc |dSNr2xs rz&parse_kernel_results..xs1Q4rkeyreverse10d Y@5.2f5d8.1fz Group kernel time by operator:uTotal(μs) Time% Operatorc |dSr8r9r:s rr<z&parse_kernel_results..s1Q4r)_NODES_TYPE_CONTAINING_SUBGRAPHappendsorteditemsr)r% thresholdkernel_name_to_op_name kernel_time kernel_freqtotal session_inititem kernel_namer/linesdurationratiocallsavg_timeop_times rparse_kernel_resultsrYEs KK EL ;) #V 8P(PL  ;( "u}4IY]^dYeLev,K6l9-G99k]!,k)K(DK7(K(A-(+/; K(+, K(6=&{3 T%[ E58 )** E LL9)c/#9NaPQ LL LL=>!' (9(9(;Y]!^ X5  9  K(eEl* ~R d';2eBZr(SWXZ[fZghi"_G 6 < < > W{+ g  G  ( 'GG  !? LL34 LL LL./#GMMOQUV5  ~R d';2gYGHW Lrcdg}i}i}i}d}|D]}|ddk(s d|vsd|vsd|dvs|djdd jd d jd d } d |dvr=|dd d k(rd} n|dd dk(rd} n |dd dk(rd} | |vr || <n ||  k(sJ|r|dd} | tvr| |vr|| xx|dz cc<|| xxdz cc<n|d|| <d|| <|j| ||dz }gd} d} |D]d} || }|| }|t|z }||z dz}|j | d }| |z } | j|dd|dd| dd|dd|dd|dd| f| jd|dzdd | jd!| jd"t |j d#d$%D]d\} }||z }||kr|| }|t|z }||z dz}|j | d }| j|dd|dd|dd|dd|dd| f| S)&aParse profile data and output nodes in two sections - nodes in the original order, and top expensive nodes. Args: sess_time (List[Dict]): profile data kernel_time_only (bool, optional): Only include items for kernel time. Defaults to False. threshold (int, optional): Minimum ratio of duration among all. Defaults to 0. Returns: List[str]: lines of string for output. rr(Noder-r.r/r* _kernel_time _fence_before _fence_afterproviderCPUExecutionProviderCPUCUDAExecutionProviderCUDADmlExecutionProviderDMLr2)z Nodes in the original order:r6u3Total(μs) Time% Acc % Avg(μs) Calls Provider NodegrBr@rArCrErD8sz# Top expensive nodes with Time% >= r3r4r5r6u-Total(μs) Time% Avg(μs) Calls Provider Nodec |dSr8r9r:s rr<z$parse_node_results..sqtrTr=)replacerGrHrgetrIrJ)r%rrKnode_name_list node_time node_freq node_providerrOrQ node_namedevicer/rSbefore_percentagerTrVrW percentager`rUs rparse_node_resultsrss2NIIM E ;& Ud]v~)W[\bWcJcV $$^R8@@RTU]]^lnpq T&\)< +/EE"F&\*-1HH#F&\*-1GG"FM1/5M),(3v===!6l9-G99I%)$U 3$)$)$'+E{ )$'( )$%%i0 T%[ ECH E # Y')$eEl*&%/  $$Y3Z' nBz$/r2CD1IHUY?Z\]bce\ffhiqrthuuwyBxC D $ LL7 C7LANO LL LLEF%ioo&7^UYZ 85  9  )$eEl*&%/  $$Y3 ~R 4'88D/ERT:UWX`acWddfgpfqrs [ Lrc2i}i}d}i}i}i}i}d}i} |D]} | ddk(s d| vsd| vsd| dvs| dd} | tvr0d| dvr-d| d vr%| |vr|| xx| dz cc<n| d|| <|| dz }d| djdd } | | vr| | xxd z cc<nd | | <| d | } | |vr|| xx| dz cc<|| xxd z cc<n | d|| <d || <| |vr|| xx| dz cc<n| d|| <| |vr|| xx| dz cc<|| xxd z cc<n | d|| <d || <|| dz }d d g}|jd|jdt|j ddD]h\} }|j| d}||z }||z}|||zz }|| }||z }|j|dd|dzdd|dd|dzdd|dd|dd|dd| j|d dgz }|jd|jdt|j ddD]n\} }| j d }|d} |d } | j dd }|| }||z }||| z }|j|dd|dzdd|dd|dd|dd| p|S) zGroup results by operator name. Args: sess_time (List[Dict]): profile data Returns: List[str]: lines of string for output. rr(r[r-r.r/r`fencer*r]r2r5zGrouped by operatorr6uMTotal(μs) Time% Kernel(μs) Kernel% Calls AvgKernel(μs) Fence(μs) Operatorc |dSr8r9r:s rr<z$group_node_results..0sQqTrTr=r@rArBrC11drDz14.1fzGrouped by provider + operatoru<Kernel(μs) Provider% Calls AvgKernel(μs) Provider Operatorc |dSr8r9r:s rr<z$group_node_results..>s RSTURVrExecutionProviderz9.2frg)rGrjrHrIrJsplitri)r%op_kernel_timeop_kernel_recordstotal_kernel_timeprovider_op_kernel_timeprovider_op_kernel_recordsprovider_kernel_time op_fence_timetotal_fence_timeprovider_counterrQr/r`r>rSrM fence_timekernel_time_ratio total_time time_ratio kernel_callsavg_kernel_timepartsshort_eprVprovider_time_ratios rgroup_node_resultsrs*N !#M ;& Ud]v~)W[\bWcJc6l9-G99f-d6l*-/%g.$u+=.15e g.$U 3$F|'' B7H++ *a/*-. *Jay)C--',U ;,*3/14//3E{',23*3///$X.$u+=.15e$X..(w'4;6'!'*a/**.u+w'-.!'* e , WZ& 'E LL LLgh &~';';'=>[_ `"&&w2 '*;; :- #47G#GH (1 % 4 #be!3D 9K;LBO`chOhimNnnpq}AqBBDETUZD[[]^hil]mmopwox y !a b2 33E LL LLTU"#:#@#@#B`de[ #8(##$7<*3/%-),@,JJ 3r"5"=d!C2eBZrRabgQhhjkstvjwwy{BzC D f Lrct|}t||j}|t||j|jz }|t |z }|SN)r&rYrKrsrr)r#r.profile_recordsrSs rprocess_resultsrMsO' 5O $.. AE 1F1F WWE  00E Lr__main__ Arguments) setup_loggerr)r)Fr)__doc__rr frozensetrGrr&rYrsrr__name__ argumentsrbenchmark_helperrrinputr#resultsliner9rrrs   "+,B"C'#TL^Tn_D  z!I +y!-""#??LlI6G d r