[Bug]: Stream gear imposes simplified aspect ratios, which does not seem necessary

[GuillaumeTong]

Description

When providing frames of unconventional aspect ratios, StreamGear enforces a simplified aspect ratio, Which forces ffmpeg to use non-square pixels, and distorts the output video.

Issue Checklist

• I have searched open or closed issues for my problem and found nothing related or helpful.
• I have read the Documentation and found nothing related to my problem.
• I've read the Issue Guidelines and wholeheartedly agree.

Expected behaviour

I expect StreamGear to faithfully stream my frames in any resolution they may come in, without distorting them.

Actual behaviour

As in title and description, my stream is getting distorted, while my application requires outputs to be perfectly aligned.

Steps to reproduce

You may reproduce a distorted steam and its reference image featuring a checkerboard pattern using the script in the "Python Code" section. The input 19:20 aspect ratio gets "rounded" to 9:10, which forces a non-square pixel ratio of SAR 18:19.
It may be difficult to see the 5% image skew on the resulting video alone with the naked eye, but you can try to align the output image on top of the video to see the width and height cannot both be matched.

Here is "checkerboard.png" and a snapshot of "checkerboard.m3u8" for reference (they mismatch in height):

---

Terminal log output

10:30:15 ::    Helper     ::   INFO   :: Running VidGear Version: 0.3.2
10:30:15 ::    Helper     ::  DEBUG   :: FFmpeg Windows Download Path: C:\Users\GUILLA~1\AppData\Local\Temp
10:30:15 ::    Helper     ::  DEBUG   :: Final FFmpeg Path: C:\Users\GUILLA~1\AppData\Local\Temp\ffmpeg-static-win64-gpl/bin/ffmpeg.exe
10:30:15 ::    Helper     ::  DEBUG   :: FFmpeg validity Test Passed!
10:30:15 ::    Helper     ::  DEBUG   :: Found valid FFmpeg Version: `b'n4.4.2-5-gaa28df74ab-20220925'` installed on this system
10:30:15 ::  StreamGear   ::  DEBUG   :: Found valid FFmpeg executables: `C:\Users\GUILLA~1\AppData\Local\Temp\ffmpeg-static-win64-gpl/bin/ffmpeg.exe`.
10:30:15 ::  StreamGear   ::   INFO   :: StreamGear will generate files for HLS HTTP streaming format.
10:30:15 ::  StreamGear   ::  DEBUG   :: Path:`C:\Users\guillaume\PycharmProjects\sandbox\streaming_prototype\checkerboard.m3u8` is sucessfully configured for streaming.
10:30:15 ::  StreamGear   ::   INFO   :: StreamGear has been successfully configured for Real-time Frames Mode.
10:30:15 ::  StreamGear   ::  DEBUG   :: InputFrame => Height:1000 Width:950 Channels:1
10:30:15 ::  StreamGear   :: WARNING  :: No valid audio_source provided. Disabling audio for streams!
10:30:15 ::  StreamGear   ::  DEBUG   :: Setting Input framerate: 10.0
10:30:15 ::  StreamGear   ::  DEBUG   :: Setting bit-per-pixels: 0.1 for this stream.
10:30:15 ::  StreamGear   ::  DEBUG   :: Setting GOP: 20 for this stream.
10:30:15 ::  StreamGear   :: WARNING  :: No `-streams` are provided!
10:30:15 ::  StreamGear   ::  DEBUG   :: User-Defined Output parameters: `-vcodec libx264 -vf format=yuv420p -aspect 9:10 -crf 20 -profile:v high -tune zerolatency -preset veryfast -map 0:v -s:v:0 950x1000 -b:v:0 950k -bf 1 -sc_threshold 0 -keyint_min 20 -g 20 -hls_segment_type mpegts -hls_list_size 0 -hls_playlist_type vod -hls_base_url  -allowed_extensions ALL -hls_segment_filename C:/Users/guillaume/PycharmProjects/sandbox/streaming_prototype\chunk-stream-%03d.ts -hls_allow_cache 0 -f hls`
10:30:15 ::  StreamGear   ::  DEBUG   :: Additional parameters: `None`
10:30:15 ::  StreamGear   :: CRITICAL :: Transcoding streaming chunks. Please wait...
ffmpeg version n4.4.2-5-gaa28df74ab-20220925 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 12.1.0 (crosstool-NG 1.25.0.55_3defb7b)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64 --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug --disable-w32threads --enable-pthreads --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --disable-libpulse --enable-libvmaf --enable-vulkan --disable-libxcb --disable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --disable-frei0r --enable-libglslang --enable-libgme --enable-libass --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librist --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libmfx --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --disable-libdrm --disable-vaapi --enable-libvidstab --disable-vulkan --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags= --extra-libs=-lgomp --extra-version=20220925
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, rawvideo, from 'pipe:':
  Duration: N/A, start: 0.000000, bitrate: 76000 kb/s
  Stream #0:0: Video: rawvideo (Y800 / 0x30303859), gray, 950x1000, 76000 kb/s, 10 tbr, 10 tbn, 10 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
[swscaler @ 00000199b353b3c0] Warning: data is not aligned! This can lead to a speed loss
[libx264 @ 00000199b352a080] using SAR=18/19
[libx264 @ 00000199b352a080] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 00000199b352a080] profile High, level 3.2, 4:2:0, 8-bit
[libx264 @ 00000199b352a080] 264 - core 164 - H.264/MPEG-4 AVC codec - Copyleft 2003-2022 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=2 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=12 lookahead_threads=12 sliced_threads=1 slices=12 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=1 b_pyramid=0 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=20 keyint_min=11 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=20.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 pb_ratio=1.30 aq=1:1.00
Output #0, hls, to 'C:/Users/guillaume/PycharmProjects/sandbox/streaming_prototype/checkerboard.m3u8':
  Metadata:
    encoder         : Lavf58.76.100
  Stream #0:0: Video: h264, yuv420p(tv, progressive), 950x1000 [SAR 18:19 DAR 9:10], q=2-31, 950 kb/s, 10 fps, 90k tbn
    Metadata:
      encoder         : Lavc58.134.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/950000 buffer size: 0 vbv_delay: N/A
[hls @ 00000199b3528900] Opening 'C:/Users/guillaume/PycharmProjects/sandbox/streaming_prototype\chunk-stream-000.ts' for writing
[hls @ 00000199b3528900] Opening 'C:/Users/guillaume/PycharmProjects/sandbox/streaming_prototype\chunk-stream-001.ts' for writing
[hls @ 00000199b3528900] Opening 'C:/Users/guillaume/PycharmProjects/sandbox/streaming_prototype\chunk-stream-002.ts' for writing
frame=   50 fps=0.0 q=10.0 Lsize=N/A time=00:00:04.90 bitrate=N/A speed=23.8x    
video:35kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[libx264 @ 00000199b352a080] frame I:3     Avg QP: 6.50  size:  6890
[libx264 @ 00000199b352a080] frame P:25    Avg QP: 5.89  size:   404
[libx264 @ 00000199b352a080] frame B:22    Avg QP: 5.47  size:   195
[libx264 @ 00000199b352a080] consecutive B-frames: 12.0% 88.0%
[libx264 @ 00000199b352a080] mb I  I16..4: 84.7%  0.7% 14.6%
[libx264 @ 00000199b352a080] mb P  I16..4:  0.7%  0.0%  0.0%  P16..4:  0.7%  0.0%  0.0%  0.0%  0.0%    skip:98.6%
[libx264 @ 00000199b352a080] mb B  I16..4:  0.0%  0.0%  0.0%  B16..8:  0.0%  0.0%  0.0%  direct: 0.0%  skip:100.0%  L0:33.3% L1:66.7% BI: 0.0%
[libx264 @ 00000199b352a080] 8x8 transform intra:0.7% inter:37.2%
[libx264 @ 00000199b352a080] coded y,uvDC,uvAC intra: 5.2% 0.0% 0.0% inter: 0.2% 0.0% 0.0%
[libx264 @ 00000199b352a080] i16 v,h,dc,p: 63% 26% 10%  0%
[libx264 @ 00000199b352a080] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 37%  7% 53%  3%  0%  0%  0%  0%  0%
[libx264 @ 00000199b352a080] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 46% 24% 30%  0%  0%  0%  0%  0%  0%
[libx264 @ 00000199b352a080] i8c dc,h,v,p: 100%  0%  0%  0%
[libx264 @ 00000199b352a080] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 00000199b352a080] kb/s:56.08
10:30:15 ::  StreamGear   :: CRITICAL :: Transcoding Ended. HLS Streaming assets are successfully generated at specified path.

Python Code(Optional)

import cv2
import numpy as np
from vidgear.gears import StreamGear

h, w = 20*50, 19*50
x = np.arange(h).reshape(h, 1)
y = np.arange(w).reshape(1, w)
grid = (x // 50 % 2) == (y // 50 % 2)
grid = grid.astype("uint8") * 255
cv2.imwrite("checkerboard.png", grid)

stream_params = {
    "-input_framerate": 10,
}
streamer = StreamGear(
    output="checkerboard.m3u8",
    format="hls",
    logging=True,
    **stream_params,
)
for i in range(50):
    streamer.stream(grid)
streamer.terminate()

VidGear Version

0.3.2

Python version

Python 3.9.18

OpenCV version

4.8.1

Operating System version

Microsoft Windows 10 Home, Version 10.0.19045 Build 19045

Any other Relevant Information?

I am fairly certain the issue is with these lines (in the current master branch):

vidgear/vidgear/gears/streamgear.py

Lines 427 to 430 in be6f3f0

	aspect_ratio = Fraction(
	self.__inputwidth / self.__inputheight
	).limit_denominator(10)
	output_parameters["-aspect"] = ":".join(str(aspect_ratio).split("/"))

Specifically, .limit_denominator(10). Is that strictly necessary? I wonder if one of these fixed could be applied:

1. Remove .limit_denominator(10) altogether
2. Try "rounding" based on the numerator, instead of the denominator only, to see which can give a closer approximation
3. Limit the denominator to a bigger number, e.g. 100

[abhiTronix]

@GuillaumeTong Hi, thanks for reporting the issue, and I agree keeping it 10 will cause distortion with large fraction values. I think removing 10 from limit_denominator will default to max_denominator=1000000 as mentioned in docs: https://docs.python.org/3/library/fractions.html#fractions.Fraction.limit_denominator which will take care of any ratios.

[abhiTronix]

This issue is fixed in commit d4243ab

[abhiTronix]

@GuillaumeTong Now, Streamgear API do not imposes -aspect parameter in its FFmpeg pipeline by default. Goodluck!