o kiJ @sUdZddlZddlZddlmZddlmZddlZddlmZddl m Z m Z ddl m Z ddlmZd d lmZejZejZd Zeejjd fed <dagaeeegdfeefed<eZeZ e a!e"ejdrwejj#Z$n de%de%fddZ$e"ejdrejj&Z'n de%de%fddZ'ddZ(ddZ)ddZ*de+fddZ,djd d!Z-Gd"d#d#e.Z/de+fd$d%Z0de+fd&d'Z1dkdeddfd(d)Z2de%fd*d+Z3de%fd,d-Z4dkdedefd.d/Z5dkdedefd0d1Z6 dld4edBd5ed6e%ddfd7d8Z7de8eeffd9d:Z9d;ee%e%e%e%gdfddfddeddfdIdJZ?dkdede8eeffdKdLZ@GdMddZAGdNdOdOZBdPdQZCdFedBdeBfdRdFZDdndedefdTdUZEdeefdVdWZFdndXededdfdYdZZGd[eeddfd\d]ZHd^e%ddfd_d`ZId^e%ddfdadbZJde%fdcd^ZKde%fdddeZLde%fdfdgZMd dhlNTd dhlOTgdiZPdS)ozH This package enables an interface for accessing MTIA backend in python N)Callable)Any)Tensor) is_fbcodeis_prod)_LazySeedTracker)Device)_get_device_index.default_generatorsF _queued_calls_mtia_exchangeDevicedevicereturncC|dkrdStdNrz)PyTorch was compiled without MTIA support RuntimeErrorrr r ]/var/www/addictedbytheproject.nl/epg/venv/lib/python3.10/site-packages/torch/mtia/__init__.py_exchange_device)r_mtia_maybeExchangeDevicecCrrrrr r r_maybe_exchange_device3rrcCs tdSN) _lazy_initr r r rinit9 rcCs tot S)z9Return whether PyTorch's MTIA state has been initialized.) _initialized_is_in_bad_forkr r r ris_initialized=s r"cKstMtr|WdS|ddrt|tn |ddr.t|tnt |tfWddSWddSWddS1sRwYdS)Nseed_allFseed) _initialization_lockr"get_lazy_seed_trackerqueue_seed_all traceback format_stack queue_seedr append)callablekwargsr r r _lazy_callBs  "r/cC tjSr)torch_C_mtia_isInBadForkr r r rr!Prr!c CsFtsttdr dSttr WddStr tdts'tdtrCt rCz ddl m }| Wn t yBYnwtjdt_tddtDz1tD]'\}}z|WqZty}zd t|d d |}t||d}~wwWttdnttdwdaWddS1swYdS) Nis_initializingzwCannot re-initialize MTIA in forked subprocess. To use MTIA with multiprocessing, you must use the 'spawn' start methodzTorch not compiled with MTIA enabled. Ensure you have `import mtia.host_runtime.torch_mtia.dynamic_library` in your python src file and include `//mtia/host_runtime/torch_mtia:torch_mtia` as your target dependency!r)cxx_resource_managerTcss|]}|r|VqdSrr ).0callsr r r sz_lazy_init..z6MTIA call failed lazily at initialization with error: z( MTIA call was originally invoked at: )r"hasattr_tlsr%r!r _is_compiledAssertionErrorrrlibfb.py.cxx_resourcesr5installModuleNotFoundErrorr1r2 _mtia_initr4r extendr' get_calls ExceptionstrjoinDeferredMtiaCallErrordelattrr )r5 queued_callorig_tracebackemsgr r rrTsP           "rc@s eZdZdS)rGN)__name__ __module__ __qualname__r r r rrGsrGcCr0)z*Return true if compiled with MTIA support.)r1r2 _mtia_isBuiltr r r rr< r<cCstsdStdkS)z'Return true if MTIA device is availableFr)r< device_countr r r r is_availables rScCs:tj| tjWdS1swYdS)z?Waits for all jobs in all streams on a MTIA device to complete.N)r1mtiarr2_mtia_deviceSynchronizerr r r synchronizes$rVcCr0)z,Return the number of MTIA devices available.)r1r2_mtia_getDeviceCountr r r rrR rRcCr0)z0Return the index of a currently selected device.)r1r2%_accelerator_hooks_get_current_devicer r r rcurrent_devicerQrZcCtjt|ddS)aSReturn the currently selected :class:`Stream` for a given device. Args: device (torch.device or int, optional): selected device. Returns the currently selected :class:`Stream` for the current device, given by :func:`~torch.mtia.current_device`, if :attr:`device` is ``None`` (default). Toptional)r1r2_mtia_getCurrentStreamr rr r rcurrent_stream r_cCr[)a=Return the default :class:`Stream` for a given device. Args: device (torch.device or int, optional): selected device. Returns the default :class:`Stream` for the current device, given by :func:`~torch.mtia.current_device`, if :attr:`device` is ``None`` (default). Tr\)r1r2_mtia_getDefaultStreamr rr r rdefault_streamr`rballpythonenabledstacks max_entriescCstsdStj|||dS)aEnable/Disable the memory profiler on MTIA allocator Args: enabled (all or state, optional) selected device. Returns statistics for the current device, given by current_device(), if device is None (default). stacks ("python" or "cpp", optional). Select the stack trace to record. max_entries (int, optional). Maximum number of entries to record. N)r"r1r2_mtia_recordMemoryHistory)rerfrgr r rrecord_memory_historysricCr0)z4Return a dictionary of MTIA memory allocator history)r1r2_mtia_memorySnapshotr r r rsnapshotrXrkobservercCstj|dS)z9Attach an out-of-memory observer to MTIA memory allocatorN)r1r2_mtia_attachOutOfMemoryObserver)rlr r rattach_out_of_memory_observersrnTincluding_emulationcCsdS)zLReturn a bool indicating if the current MTIA device supports dtype bfloat16.Tr )ror r ris_bf16_supportedsrpcCr[)aReturn capability of a given device as a tuple of (major version, minor version). Args: device (torch.device or int, optional) selected device. Returns statistics for the current device, given by current_device(), if device is None (default). Tr\)r1r2_mtia_getDeviceCapabilityr rr r rget_device_capabilityrrcCr0)zEmpty the MTIA device cache.)r1r2_mtia_emptyCacher r r r empty_cacherQrustreamcCs|durdStj|dS)aSet the current stream. This is a wrapper API to set the stream. Usage of this function is discouraged in favor of the ``stream`` context manager. Args: stream (Stream): selected stream. This function is a no-op if this argument is ``None``. N)r1r2_mtia_setCurrentStreamrvr r r set_streams rycCs$t|}|dkrtj|dSdS)zSet the current device. Args: device (torch.device or int): selected device. This function is a no-op if this argument is negative. rN)r r1r2%_accelerator_hooks_set_current_devicerr r r set_devicesr{cCr[)zReturn a dictionary of MTIA device properties Args: device (torch.device or int, optional) selected device. Returns statistics for the current device, given by current_device(), if device is None (default). Tr\)r1r2_mtia_getDevicePropertiesr rr r rget_device_propertiesrsr}c@s<eZdZdZdefddZddZdededefd d Zd S) rzContext-manager that changes the selected device. Args: device (torch.device or int): device index to select. It's a no-op if this argument is a negative integer or ``None``. cCst|dd|_d|_dS)NTr\r)r idxprev_idx)selfrr r r__init__.s zdevice.__init__cCstj|j|_dSr)r1r2(_accelerator_hooks_maybe_exchange_devicer~r)rr r r __enter__2szdevice.__enter__typevaluer)cCstj|j|_dS)NF)r1r2rrr~)rrrr)r r r__exit__5szdevice.__exit__N)rMrNrO__doc__rrrrr r r rr&s c@sNeZdZUdZedBed<dedBfddZddZd ed ed efd d Z dS) StreamContextaContext-manager that selects a given stream. All MTIA kernels queued within its context will be enqueued on a selected stream. Args: Stream (Stream): selected stream. This manager is a no-op if it's ``None``. .. note:: Streams are per-device. N cur_streamrvcCstd|_||_tdd|_tjs|jdurd|_tjs dntjd|_ tjs1d|_ dStjd|_ dS)NTr) rrvr r~r1jit is_scriptingrTrbsrc_prev_streamdst_prev_stream)rrvr r rrHs     zStreamContext.__init__cCs|j}|dus |jdkrdStjd|_|jj|jkr9t|jtj|j|_Wdn1s4wYtj|dSNr) rvr~r1rTr_rrrry)rrr r rrWs zStreamContext.__enter__rrr)cCsJ|j}|dus |jdkrdS|jj|jkrtj|jtj|jdSr)rvr~rrr1rTryr)rrrr)rr r rrfs zStreamContext.__exit__) rMrNrOrStream__annotations__rrrrr r r rr:s  rcCstj|||dS)zset stream specified by the stream id, device index and device type Args: stream_id (int): stream id in stream pool device_index (int): device index in topo device_type (int): enum device type N)r1r2_mtia_setStream) stream_id device_index device_typer r r_set_stream_by_idtrsrcCst|S)a,Wrap around the Context-manager StreamContext that selects a given stream. Arguments: stream (Stream): selected stream. This manager is a no-op if it's ``None``. .. note:: In eager mode stream is of type Stream class while in JIT it doesn't support torch.mtia.stream )rrxr r rrvsrTcCs0tt|dd}|durt}t|}|S)a[Returns the random number generator state of the specified MTIA device as a ByteTensor. Args: device (torch.device or int, optional): The device to return the RNG state of. Default: ``'mtia'`` (i.e., ``torch.device('mtia')``, the current mtia device). .. warning:: This function eagerly initializes MTIA. Tr\N)rr rZr get_state)rr~default_generatorr r r get_rng_states  rcCsddttD}|S)zRReturns a list of ByteTensor representing the random number states of all devices.cSsg|]}t|qSr )r)r6ir r r sz%get_rng_state_all..)rangerR)resultsr r rget_rng_state_allsr new_statecsrtstjjtjdWdn1swYt|dddur,tfdd}t|dS)a8Sets the random number generator state of the specified MTIA device. Args: new_state (torch.ByteTensor): The desired state device (torch.device or int, optional): The device to set the RNG state. Default: ``'mtia'`` (i.e., ``torch.device('mtia')``, the current mtia device). ) memory_formatNTr\cst}|dSr)r set_state)rr~rr rcbszset_rng_state..cb) r"r1r2_DisableFuncTorchclonecontiguous_formatr rZr/)rrrr rr set_rng_states   r new_statescCs t|D] \}}t||qdS)zSets the random number generator state of all devices. Args: new_states (Iterable of torch.ByteTensor): The desired state for each device. N) enumerater)rrstater r rset_rng_state_alls rr$c$tfdd}t|dddS)aSets the seed for generating random numbers for the current MTIA device. It's safe to call this function if MTIA is not available; in that case, it is silently ignored. Args: seed (int): The desired seed. .. warning:: If you are working with a multi-GPU model, this function is insufficient to get determinism. To seed all GPUs, use :func:`manual_seed_all`. cst}t|}|dSr)rZr manual_seedr~rr$r rrszmanual_seed..cbTrNintr/r$rr rrrs rcr)zSets the seed for generating random numbers on all MTIA devices. It's safe to call this function if MTIA is not available; in that case, it is silently ignored. Args: seed (int): The desired seed. cs&ttD] }t|}|qdSr)rrRr r)rrrr rrs zmanual_seed_all..cbTr#Nrrr rrmanual_seed_alls rcCdd}t|ddS)axSets the seed for generating random numbers to a random number for the current MTIA device. It's safe to call this function if MTIA is not available; in that case, it is silently ignored. .. warning:: If you are working with a multi-GPU model, this function will only initialize the seed on one GPU. To initialize all GPUs, use :func:`seed_all`. cSst}t|}|Sr)rZr r$rr r rrszseed..cbTrr/rr r rr$s cCr)zSets the seed for generating random numbers to a random number on all MTIA devices. It's safe to call this function if MTIA is not available; in that case, it is silently ignored. cSs@d}d}ttD]}t|}|s|}d}q ||q |S)NrFT)rrRr r$r) random_seedseededrrr r rrs zseed_all..cbTrrrr r rr#s r#cCstt}t|S)zReturns the current random seed of the current MTIA device. .. warning:: This function eagerly initializes MTIA. )rrZr initial_seed)r~r r rrs r)*)#rrSr"rVrRrZr_rb memory_statsmax_memory_allocatedmemory_allocatedreset_peak_memory_statsrrr}rirkrnrur{ryrvrrrrrrrr$r#rrp MTIAGraphgraphgraph_pool_handle)rNr)rcrdr)T)rT)Qr threadingr)collections.abcrtypingrr1rtorch._environmentrr torch._utilsr torch.typesr_utilsr Eventrr tupler2 Generatorrr r listrElocalr;Lockr%r'r:rrrrrrr"r/boolr!rrDrGr<rSrVrRrZr_rbridictrkrnrprrruryr{r}rrrrvrrrrrrr$r#rmemory mtia_graph__all__r r r rs           >       :