o ei @s^UddlZddlmmZddlmZddlmZgZe e e d<ej j GdddZdS)N)Tensor)2_scripted_functional_optimizer_deprecation_warning__all__c@seZdZ         ddeededededed ed ed ed ed efddZdededBfddZdeedBfddZ dS)_FunctionalSGD{Gz?Fparamslrmomentum dampening weight_decaynesterovmaximizeforeachfused_allow_empty_param_listc Cs|tdd||||d|_||_||_||_| |_tjt tj t t tj ffi|_ t |dkr7| s7tdd|i|_dS)N) stacklevel)r r r r rz%optimizer got an empty parameter listr)rdefaultsr rrrtorchjitannotatedictrstrstatelen ValueError param_group) selfrr r r r r rrrrrp/var/www/addictedbytheproject.nl/epg/venv/lib/python3.10/site-packages/torch/distributed/optim/functional_sgd.py__init__s $z_FunctionalSGD.__init__paramgradNc Cs|jd}|jd}|jd}|jd}|g}g}g} d} |durK| ||jr+d} ||jvr5i|j|<|j|} d| vrD|dn|| dttj|| ||||||j|j | |j |j ddd Wdn1sqwY|j|} |d } | dur| | d<dSdS) z[Similar to self.step, but operates on a single parameter and its gradient. r r r r FNTmomentum_buffer r r r r r rhas_sparse_gradrr grad_scale found_infr) rappend is_sparserrno_gradFsgdr rrr) rr"r#r r r r rmomentum_buffer_listgradsr&rr$rrr step_param:sR            z_FunctionalSGD.step_param gradientscCs|jd}g}g}g}|jd}|jd}|jd}|jd} t|t|kr:tddt|dd t|d } t||D]7\} } | durx|| || | jrXd } | |jvrbi|j| <|j| } d | vrq|dqA|| d qAt t j ||||||| |j |j | |j|jddd Wdn1swYt|D]\}}|j|} ||}|dur|| d <qdS)Nrr r r r zEthe gradients passed in does not equal to the size of the parameters!zParams length: z. zGradients length: FTr$r%)rrrrzipr)r*rrr+r,r-r rrr enumerate)rr1rparams_with_gradr/r.r r r r r&r"gradientripr$rrr steplsn              z_FunctionalSGD.step) rrrrFFFFF) __name__ __module__ __qualname__listrfloatboolr!r0r8rrrr rsB   !2r)rtorch.optim._functionaloptim _functionalr,r,torch.distributed.optim._deprecation_warningrrr<r__annotations__rscriptrrrrr s