BvhY8ddlZddlZddlZddlmZddlZddlmZm Z m Z ddl m Z de de de de d ejf d Zd e de de d ejfd Zd e de fdZ d,de de de de dede d ejfdZdedeeejffdZde de de de dede de d e de de dede fdZ d-de de de de dede d e de de dede de fdZdZ d.de dedzd edzd!edzd eejdzejdzejdzff d"Z d.d#ededzd edzd!edzd eejdzejdzejdzff d$Zd%Zd&ed'ede de de de dededzd edzd!edzd(ede dede fd)Zd*Ze d+k(reyy)/N)Path) ModelProto TensorProto numpy_helper) OnnxModel input_ids batch_sizesequence_lengthdictionary_sizereturnc(|jjjtjtj tj fvsJtjj|||ftj}|jjjtjk(rtj|}|S|jjjtj k(rtj|}|S)a`Create input tensor based on the graph input of input_ids Args: input_ids (TensorProto): graph input of the input_ids input tensor batch_size (int): batch size sequence_length (int): sequence length dictionary_size (int): vocabulary size of dictionary Returns: np.ndarray: the input tensor created )sizedtype) type tensor_type elem_typerFLOATINT32INT64nprandomrandintint32float32int64)rr r r datas Q/RAG/venv/lib/python3.12/site-packages/onnxruntime/transformers/bert_test_data.pyfake_input_ids_datars >> % % / /4   99  _J3PXZX`X`  aD~~!!++{/@/@@zz$ K  # # - -1B1B Bxx~ K segment_idsc|jjjtjtj tj fvsJtj||ftj}|jjjtjk(rtj|}|S|jjjtj k(rtj|}|S)a,Create input tensor based on the graph input of segment_ids Args: segment_ids (TensorProto): graph input of the token_type_ids input tensor batch_size (int): batch size sequence_length (int): sequence length Returns: np.ndarray: the input tensor created r) rrrrrrrrzerosrrr)r r r rs rfake_segment_ids_datar$1s    ' ' 1 16   88Z1 BD##--1B1BBzz$ K    % % / /;3D3D Dxx~ Krmax_sequence_lengthaverage_sequence_lengthc|dk\r||ksJd|z|kDrtjd|z|z |Stjdd|zdz S)N)rr)r%r&s rget_random_lengthr*Lsd "a ',CGZ,ZZ Z ""%88~~a"99*QRUVQVtJNQ,>'?$q1u+'M\]Z!^$YZ]^Y^tJNQ4F/G$qSTu+/UdeQ^a'!+,+:&1Q':>*'(+B'BZ!^$/03J/JQ^a'!+,+"",, 0A0AAzz$ K  $ $ . .+2C2C Cxx~ Kr directoryinputsc  tjj|s& tj|t d|dnt d|dt |jD]t\}\}}tj||}ttjj|d|dd 5}|j|jd d d vy #t $rt d|dYwxYw#1swYxYw) zOutput input tensors of test data to a directory Args: directory (str): path of a directory inputs (Dict[str, np.ndarray]): map from input name to value z#Successfully created the directory  zCreation of the directory z failedzWarning: directory z$ existed. Files will be overwritten.input_.pbwbN)ospathexistsmkdirprintOSError enumerateitemsr from_arrayopenjoinwriteSerializeToString)r8r9indexnamertensorfiles routput_test_datarPs 77>>) $ F HHY  7 {!D E #I;.RST(8|d((t4 "'',,yF5'*=> E JJv//1 2F E 9 C .ykA B CF EsC&; D&DDD test_casesverbose random_seedc |Jtjj|tj|g} t|D]} t ||||}|j |i}|rt |||||j <|rt|||| | | ||j <|rt| dk(r td|| j|| S)aCreate given number of input data for testing Args: batch_size (int): batch size sequence_length (int): sequence length test_cases (int): number of test cases dictionary_size (int): vocabulary size of dictionary for input_ids verbose (bool): print more information or not random_seed (int): random seed input_ids (TensorProto): graph input of input IDs segment_ids (TensorProto): graph input of token type IDs input_mask (TensorProto): graph input of attention mask average_sequence_length (int): average sequence length excluding paddings random_sequence_length (bool): whether use uniform random number for sequence length mask_type (int): mask type 1 is mask index; 2 is 2D mask; 3 is key len, cumulated lengths of query and key Returns: List[Dict[str,numpy.ndarray]]: list of test cases, where each test case is a dictionary with input name as key and a tensor as value rzExample inputs) rrseedr1rrMr$r7lenrCappend)r r rQr rRrSrr r+r&r,r- all_inputs _test_caseinput_1r9s rfake_test_datar[sD   IINN; KK JJ' %i_o^..'* '<[*Ve'fF;## $ &:J9PRhjs'F:?? # s:!+ "F +&!( rrUc dt|||| ||||||| | } t| |k7r td| S)aCreate given number of input data for testing Args: batch_size (int): batch size sequence_length (int): sequence length test_cases (int): number of test cases seed (int): random seed verbose (bool): print more information or not input_ids (TensorProto): graph input of input IDs segment_ids (TensorProto): graph input of token type IDs input_mask (TensorProto): graph input of attention mask average_sequence_length (int): average sequence length excluding paddings random_sequence_length (bool): whether use uniform random number for sequence length mask_type (int): mask type 1 is mask index; 2 is 2D mask; 3 is key len, cumulated lengths of query and key Returns: List[Dict[str,numpy.ndarray]]: list of test cases, where each test case is a dictionary with input name as key and a tensor as value z$Failed to create test data for test.)r[rVrC) r r rQrUrRrr r+r&r,r-r rXs rgenerate_test_datar]sPB   J :*$ 45 rc|t|jk\ry|j|}|j|}|A|j||}|-|jdk(r|j|jd}|S)NCastr)rVinputfind_graph_input get_parentop_type) onnx_model embed_node input_indexr` graph_input parent_nodes rget_graph_input_from_embed_noderi$sc***++   [ )E--e4K ++J D  "{':':f'D$55k6G6G6JKK rrdinput_ids_namesegment_ids_nameinput_mask_namecp|j}||j|}|td|d}|r!|j|}|td|d}|r!|j|}|td|d|rdndz|rdndz}t||k7rtd|dt||||fSt|dk7rtdt||j d } t| dk(rh| d} t || d}t || d}t || d }|(|D]#} | j j} d | vs"| }%| td |||fSd}d}d}|D]0} | j j} d | vr| }$d | vsd| vr| }/| }2|r |r|r|||fStd)a Find graph inputs for BERT model. First, we will deduce inputs from EmbedLayerNormalization node. If not found, we will guess the meaning of graph inputs based on naming. Args: onnx_model (OnnxModel): onnx model object input_ids_name (str, optional): Name of graph input for input IDs. Defaults to None. segment_ids_name (str, optional): Name of graph input for segment IDs. Defaults to None. input_mask_name (str, optional): Name of graph input for attention mask. Defaults to None. Raises: ValueError: Graph does not have input named of input_ids_name or segment_ids_name or input_mask_name ValueError: Expected graph input number does not match with specified input_ids_name, segment_ids_name and input_mask_name Returns: Tuple[Optional[np.ndarray], Optional[np.ndarray], Optional[np.ndarray]]: input tensors of input_ids, segment_ids and input_mask Nz Graph does not have input named r(rzExpect the graph to have z inputs. Got r/z'Expect the graph to have 3 inputs. Got EmbedLayerNormalizationmaskz#Failed to find attention mask inputtokensegmentz?Fail to assign 3 inputs. You might try rename the graph inputs.)'get_graph_inputs_excluding_initializersra ValueErrorrVget_nodes_by_op_typerirMlower) rdrjrkrl graph_inputsrr r+expected_inputs embed_nodesrer`input_name_lowers rfind_bert_inputsr{1s>4EEGL!//?  ??OPQ Q $556FGK" #CDTCU!VWW #44_EJ! #COCT!UVVKqQ7 1PQR |  /88IWZ[gWhVijk k+z11 <AB3|CTBUVWW112KLK ;1 ^ 3J AN 5j*aP 4ZQO  %#(::#3#3#5 --!&J&  BC C+z11IKJ ::++- % %J ' '98H+HKI[Z+z11 V WWr onnx_filect}t|d5}|j|jdddt |}t ||||S#1swY"xYw)aFind graph inputs for BERT model. First, we will deduce inputs from EmbedLayerNormalization node. If not found, we will guess the meaning of graph inputs based on naming. Args: onnx_file (str): onnx model path input_ids_name (str, optional): Name of graph input for input IDs. Defaults to None. segment_ids_name (str, optional): Name of graph input for segment IDs. Defaults to None. input_mask_name (str, optional): Name of graph input for attention mask. Defaults to None. Returns: Tuple[Optional[np.ndarray], Optional[np.ndarray], Optional[np.ndarray]]: input tensors of input_ids, segment_ids and input_mask rbN)rrHParseFromStringreadrr{)r|rjrkrlmodelrOrds rget_bert_inputsrsW( LE i $ diik* 5!J J8H/ ZZ  s AA!cttj}|jddtd|jddtdd|jd dtd d |jd dtd d|jddtdd|jddtdd|jddtdd|jddtd d|jddtdd|jdddd|j d|jdddd |j d!|jd"d#d$td%&|jd'd(ddd)|j d*|jd+dtd,d-|j }|S).Nz--modelTzbert onnx model path.)requiredrhelpz --output_dirFz4output test data path. Default is current directory.)rrdefaultrz --batch_sizer(zbatch size of inputz--sequence_lengthz maximum sequence length of inputz--input_ids_namezinput name for input idsz--segment_ids_namezinput name for segment idsz--input_mask_namezinput name for attention maskz --samplesz$number of test cases to be generatedz--seedr/z random seedz --verbose store_truezprint verbose information)ractionr)rRz--only_input_tensorsz-only save input tensors and no output tensors)only_input_tensorsz-az--average_sequence_lengthz)average sequence length excluding padding)rrrz-rz--random_sequence_lengthz3use uniform random instead of fixed sequence length)r,z --mask_typer)z^mask type: (1: mask index, 2: raw 2D mask, 3: key lengths, cumulated lengths of query and key))argparseArgumentParser add_argumentstrint set_defaults parse_args)parserargss rparse_argumentsrs'  $ $ &F  DsAXY   C  S!Rgh   /    '    )    ,    3  5sAMZ  (   &  <   51  #  8   " B  u5   m     D Krr output_dirrc t|||| \}}}t||||||||| | | }t|D]=\}}tjj |dt |z}t||?| ryddl}d|jvrddgndg}|j||}|jDcgc]}|j}}t|D]\}}tjj |dt |z}|j||}t|D]\}}tjt!j"|||}t%tjj |d|dd 5}|j'|j)dddycc}w#1swYxYw) aICreate test data for a model, and save test data to a directory. Args: model (str): path of ONNX bert model output_dir (str): output directory batch_size (int): batch size sequence_length (int): sequence length test_cases (int): number of test cases seed (int): random seed verbose (bool): whether print more information input_ids_name (str): graph input name of input_ids segment_ids_name (str): graph input name of segment_ids input_mask_name (str): graph input name of input_mask only_input_tensors (bool): only save input tensors, average_sequence_length (int): average sequence length excluding paddings random_sequence_length (bool): whether use uniform random number for sequence length mask_type(int): mask type test_data_set_NrCUDAExecutionProviderCPUExecutionProvider) providersoutput_r=r>)rr]rEr?r@rIrrP onnxruntimeget_available_providersInferenceSession get_outputsrMrunrrGrasarrayrHrJrK)rrr r rQrUrRrjrkrlrr&r,r-rr r+rXr3r9r8rrsessionoutput output_namesresult output_name tensor_resultrOs rcreate_and_save_test_datar sD*9P`bq)r&I{J#  Jz* 6GGLL-=A-FG F++ #k&I&I&K K !"89$ % **5I*FG.5.A.A.CD.CFFKK.CLDz* 6GGLL-=A-FG \62' 5NA{(33BJJvay4I;WMbggll9s#.>?F$ =::<=GF6+EGFs8F4 F99G ct}|jdkr|j|_|j}|Yt |j }t jj|jd|jd|j}|t |}|jddn tdt|j ||j|j|j|j|j |j"|j$|j&|j(|j|j*|j,td|y)Nrbatch__seq_T)parentsexist_okz7Directory existed. test data files will be overwritten.z Test data is saved to directory:)rr&r rrrr?r@rIparentr rBrCrsamplesrUrRrjrkrlrr,r-)rrpr@s rmainrYs  D ##q('+';';$J  WW\\!((fT__4EU4K_K_J`,ab J 4$ / GH           $$ ## " ,j9r__main__)r))i')NNN)!rr?rpathlibrnumpyronnxrrrrdrrndarrayrr$r*boolr7rdictrPr[r]ritupler{rrrr__name__rrrs 66 (+>ATWZZ<{VY^`^h^h6B3BB  FFFF! F ! F  FZZFR33T#rzz/-B3.7777 7  7  7777!7!77L!1111  1  1  111!1!111h "&#'"& YXYX$JYXDjYX4Z YX  2:: bjj4/d1B BC YX|"&#'"& [[$J[Dj[4Z [  2:: bjj4/d1B BC [8aHI> I>I>I> I>  I>  I>I>$JI>DjI>4ZI>I>!I>!I>I>X$:N zFr