o oi; @sddlZddlZddlZddlmZmZmZmZddlZddl m Z edZ ddZ ddej d ed ed ed ej f d dZGdddZde d e fddZGdddZdS)N)AnyOptionalTypeVarUnion)tqdmTcCs&zddl}W|Stytdw)NrzVideo decoding capabilities were removed from torchvision and migrated to TorchCodec. Please install TorchCodec following instructions at https://github.com/pytorch/torchcodec#installing-torchcodec) torchcodec ImportError)r r j/var/www/addictedbytheproject.nl/epg/venv/lib/python3.10/site-packages/torchvision/datasets/video_utils.py_get_torchcodec s  r tensorsizestepdilationreturncCs|dkrtd||d}|}||||f}|||dd|d|f}|ddkr:d|f}t|||S)a similar to tensor.unfold, but with the dilation and specialized for 1d tensors Returns all consecutive windows of `size` elements, with `step` between windows. The distance between each element in a window is given by `dilation`. rz*tensor should have 1 dimension instead of r)dim ValueErrorstridenumeltorch as_strided)rrrro_strider new_stridenew_sizer r r unfolds    rc@sVeZdZdZdeeddfddZdefddZd ede eee e ffd d Z dS) _VideoTimestampsDatasetz Dataset used to parallelize the reading of the timestamps of a list of videos, given their paths in the filesystem. Used in VideoClips and defined at top level, so it can be pickled when forking. video_pathsrNcCs ||_dSNr)selfrr r r __init__6 z _VideoTimestampsDataset.__init__cC t|jSrlenrr!r r r __len__9r#z_VideoTimestampsDataset.__len__idxcCs8t}|j|j|}|jj}|jj}tt||fSr) r decoders VideoDecoderrmetadata num_frames average_fpslistrange)r!r)r decoderr-fpsr r r __getitem__<s z#_VideoTimestampsDataset.__getitem__) __name__ __module__ __qualname____doc__r/strr"intr(tuplerfloatr3r r r r r-s &rxcCs|S)zH Dummy collate function to be used with _VideoTimestampsDataset r )r<r r r _collate_fnDsr=c@seZdZdZ            d=deeded ed eed ee ee fd ed edededededededdfddZ d>ddZ de ee fddfddZ ede ee ffddZdeeddfdd Ze d?d!ejd"ed#ed$eed eedeejeeeejfff d%d&Zd?d"ed#ed eeddfd'd(Zdefd)d*Zdefd+d,Zdefd-d.Zd/edeeeffd0d1Zed"ed2ed3edeeejffd4d5Zd/edeejeje ee feffd6d7Zde ee ffd8d9Zd:e ee fddfd;d<Z dS)@ VideoClipsaE Given a list of video files, computes all consecutive subvideos of size `clip_length_in_frames`, where the distance between each subvideo in the same video is defined by `frames_between_clips`. If `frame_rate` is specified, it will also resample all the videos to have the same frame rate, and the clips will refer to this frame rate. Creating this instance the first time is time-consuming, as it needs to decode all the videos in `video_paths`. It is recommended that you cache the results after instantiation of the class. Recreating the clips for different clip lengths is fast, and can be done with the `compute_clips` method. Args: video_paths (List[str]): paths to the video files clip_length_in_frames (int): size of a clip in number of frames frames_between_clips (int): step (in frames) between each clip frame_rate (float, optional): if specified, it will resample the video so that it has `frame_rate`, and then the clips will be defined on the resampled video num_workers (int): how many subprocesses to use for data loading. 0 means that the data will be loaded in the main process. (default: 0) output_format (str): The format of the output video tensors. Can be either "THWC" (default) or "TCHW". rNrTHWCrclip_length_in_framesframes_between_clips frame_rate_precomputed_metadata num_workers _video_width _video_height_video_min_dimension_video_max_dimension_audio_samples_audio_channels output_formatrcCs||_||_||_||_| |_| |_| |_| |_| |_ |j dvr*t d| d|dur3| n| || |||dS)N)r@TCHWz5output_format should be either 'THWC' or 'TCHW', got .)rrErFrGrHrIrJrKupperrLr_compute_frame_pts_init_from_metadata compute_clips)r!rrArBrCrDrErFrGrHrIrJrKrLr r r r"fs    zVideoClips.__init__csg|_g|_ddljjjt|jd|jt d}t t |d0}|D]$}| dt t|\}}fdd|D}|j||j|q#WddS1sSwYdS)Nrr?) batch_sizerE collate_fn)totalrcsg|] }j|jdqS)dtype) as_tensorlong).0ptsrr r sz1VideoClips._compute_frame_pts..) video_pts video_fpstorch.utils.datautilsdata DataLoaderrrrEr=rr&updater/zipextend)r!dlpbarbatch batch_pts batch_fpsr r\r rPs$  "zVideoClips._compute_frame_ptsr,cCsV|d|_t|jt|dksJ|d|_t|jt|dks$J|d|_dS)Nrr^r_)rr&r^r_)r!r,r r r rQs  zVideoClips._init_from_metadatacCs|j|j|jd}|S)Nrr^r_rl)r! _metadatar r r r,s zVideoClips.metadataindicescs~fdd|D}fdd|D}fdd|D}|||d}t|jjj|jjjjjj j j d S)Ncg|]}j|qSr r rZir'r r r]z%VideoClips.subset..cror )r^rpr'r r r]rrcror )r_rpr'r r r]rrrl) rArBrCrDrErFrGrHrIrJrKrL) typer-rrCrErFrGrHrIrJrKrL)r!rnrr^r_r,r r'r subsets,zVideoClips.subsetr^r-rr2c Cs|durd}|dur |}t|||}ttt|||}||}t|||}|s3t dt |t rC|gt|}||fSt|||}||fS)NrzThere aren't enough frames in the current video to get a clip for the given clip length and frames between clips. The video (and potentially others) will be skipped.) r&r>_resample_video_idxr9mathfloorrrwarningswarn isinstanceslice) r^r-rr2rC total_frames_idxsclipsidxsr r r compute_clips_for_videos"   z"VideoClips.compute_clips_for_videoc Cs||_||_||_g|_g|_t|j|jD]\}}||||||\}}|j ||j |qt dd|jD}| d |_dS)a Compute all consecutive sequences of clips from video_pts. Always returns clips of size `num_frames`, meaning that the last few frames in a video can potentially be dropped. Args: num_frames (int): number of frames for the clip step (int): distance between two clips frame_rate (int, optional): The frame rate cSg|]}t|qSr r&rZvr r r r]z,VideoClips.compute_clips..rN)r-rrCr~resampling_idxsrer^r_rappendrrXcumsumtolistcumulative_sizes) r!r-rrCr^r2r~r clip_lengthsr r r rRs  zVideoClips.compute_clipscCs|Sr) num_clipsr'r r r r(szVideoClips.__len__cCr$rr%r'r r r num_videosr#zVideoClips.num_videoscCs |jdS)zJ Number of subclips that are available in the video list. )rr'r r r rs zVideoClips.num_clipsr)cCs<t|j|}|dkr|}||fS||j|d}||fS)zw Converts a flattened representation of the indices into a video_idx, clip_idx representation. rr)bisect bisect_rightr)r!r) video_idxclip_idxr r r get_clip_location s zVideoClips.get_clip_location original_fpsnew_fpscCsL||}|rt|}tdd|Stj|tjd|}|tj}|S)NrV) is_integerr9r{rarangefloat32rwtoint64)r-rrrrr r r rus zVideoClips._resample_video_idxcCs||krtd|d|d||\}}|j|}|j||}t|d}t|d}t}|jdkr@dnd} |j j || d } | j t t ||d d j} | jj} || } |d | }z|j |}|j| |d }|j}Wntytjd tjd}Ynwd| i}|jdur|j||}t|tjr||d}| |} |j|d<t| |jksJ| jd|j| |||fS)a7 Gets a subclip from a list of videos. Args: idx (int): index of the subclip. Must be between 0 and num_clips(). Returns: video (Tensor) audio (Tensor) info (Dict) video_idx (int): index of the video in `video_paths` zIndex z out of range (z number of clips)rrr@NHWCNCHW)dimension_orderr)rn) start_seconds stop_seconds)rrrVr_Nz x )r IndexErrorrrr~r9itemr rLr*r+ get_frames_atr/r0rbr,r. AudioDecoderget_samples_played_in_range ExceptionremptyrrCrrzTensorr&r-shape)r!r)rr video_pathclip_pts start_idxend_idxr rr1videor2 start_secend_sec audio_decoder audio_samplesaudioinforesampling_idxr r r get_clip#s<          $ zVideoClips.get_clipcCsndd|jD}dd|jD}|rt|}|}|j}||d<||d<|d=|d=|d=d |d <|S) NcSrr rrr r r r]Wrz+VideoClips.__getstate__..cSsg|]}|tjqSr )rrr)rZr<r r r r]\svideo_pts_sizesr^r~rr_version)r^rcatnumpy__dict__copy)r!rr^dr r r __getstate__Vs  zVideoClips.__getstate__rcCsfd|vr ||_dStj|dtjd}tj||ddd}|d=||d<||_||j|j|jdS)Nrr^rVrr)r) rrrXrsplitrRr-rrC)r!rr^r r r __setstate__rszVideoClips.__setstate__) r?rNNrrrrrrrr@)rNr)!r4r5r6r7r/r8r9rr;dictrr"rPrQpropertyr,rt staticmethodrrr:rr{rrRr(rrrrurrrr r r r r>Ks      %  &* 3r>)r)rrvrxtypingrrrrrrarrr rr9rrr=r>r r r r s $