Overlay transparent over image does not work "invalid <lambda> value.

[jurassicjordan]

backgroundremover -i greg.mp4 -toi green.png -o supergreg.mov
usage: backgroundremover [-h] [-m {u2net,u2net_human_seg,u2netp}] [-a [ALPHA_MATTING]] [-af ALPHA_MATTING_FOREGROUND_THRESHOLD] [-ab ALPHA_MATTING_BACKGROUND_THRESHOLD] [-ae ALPHA_MATTING_ERODE_SIZE] [-az ALPHA_MATTING_BASE_SIZE]
[-wn WORKERNODES] [-gb GPUBATCHSIZE] [-fr FRAMERATE] [-fl FRAMELIMIT] [-mk [MATTEKEY]] [-tv [TRANSPARENTVIDEO]] [-tov [TRANSPARENTVIDEOOVERVIDEO]] [-toi [TRANSPARENTVIDEOOVERIMAGE]] [-tg [TRANSPARENTGIF]]
[-tgwb [TRANSPARENTGIFWITHBACKGROUND]] [-i [INPUT]] [-bi [BACKGROUNDIMAGE]] [-bv [BACKGROUNDVIDEO]] [-o [OUTPUT]]
backgroundremover: error: argument -toi/--transparentvideooverimage: invalid value: 'green.png'

also the example in the readme calls an mp4 file??

[lianping1985]

how to solve it

[cirosantilli]

The README is botched, the following avoids errors:

convert -size 512x512 xc:black black.png
backgroundremover -i in.mp4 -toi -bi black.png -o out.mp4

The problem is that -toi has:

    ap.add_argument(
        "-toi",
        "--transparentvideooverimage",
        nargs="?",
        const=True,
        default=False,
        type=lambda x: bool(strtobool(x)),
        help="Overlay transparent video over another image",
    )

so if a positional parameter follows it has to be something that

from distutils.util import strtobool

can convert to bool like True or False, not black.png.

I try passing the image via -bi because the code then does:

        elif args.transparentvideooverimage:
            utilities.transparentvideooverimage(os.path.abspath(args.output.name), os.path.abspath(args.backgroundimage.name),
                                                os.path.abspath(args.input.name),
                                                worker_nodes=args.workernodes,
                                                gpu_batchsize=args.gpubatchsize,
                                                model_name=args.model,
                                                frame_limit=args.framelimit,
                                                framerate=args.framerate)

so the image comes from args.backgroundimage.name which comes from -bi.

Unfortunately the above command does not work, as it produces a video with a single frame. Possibly a simple bug with the ffmpeg cli:

def transparentvideooverimage(output, overlay, file_path,
                         worker_nodes,
                         gpu_batchsize,
                         model_name,
                         frame_limit=-1,
                         prefetched_batches=4,
                         framerate=-1):
    temp_dir = tempfile.TemporaryDirectory()
    tmpdirname = Path(temp_dir.name)
    temp_file = os.path.abspath(os.path.join(tmpdirname, "matte.mp4"))
    matte_key(temp_file, file_path,
              worker_nodes,
              gpu_batchsize,
              model_name,
              frame_limit,
              prefetched_batches,
              framerate)
    print("Scale image")
    temp_image = os.path.abspath("%s/new.jpg" % tmpdirname)
    cmd = [
        'ffmpeg', '-y', '-i', overlay, '-i', file_path, '-filter_complex',
        'scale2ref[img][vid];[img]setsar=1;[vid]nullsink', '-q:v', '2', temp_image
    ]
    sp.run(cmd)
    print("Starting alphamerge")
    cmd = [
        'ffmpeg', '-y', '-i', temp_image, '-i', file_path, '-i', temp_file, '-filter_complex',
        '[0:v]scale2ref=oh*mdar:ih[bg];[1:v]scale2ref=oh*mdar:ih[fg];[bg][fg]overlay=(W-w)/2:(H-h)/2:shortest=1[out]',
        '-map', '[out]', '-shortest', output
    ]
    sp.run(cmd)
    print("Process finished")
    try:
        temp_dir.cleanup()
    except PermissionError:
        pass
    return

The shortest=1 seems suspicious, because it seems to be overlaying the background image to the foreground video and stopping at shortest, which would be the image of course at 1 frame.

[cirosantilli]

Just removing shortest=1 didn't work unfortunately. The video regains the correct length, but background was not removed. More thinking/debugging would be needed.

[nadermx]

This if I remember correctly is an actual issue with the ffmpeg command doing it. I'll try and check later but if someone wants to debug it's the actual ffmpeg command and not shortest, but a filter issue i think