Is Video Supported?

[ssmmoo1]

Can this model accept video as an input?

[gordonhu608]

Thank you for your interest in our work. Yes it also supported video input, see this line of code

BLIVA/bliva/models/bliva_vicuna7b.py

Lines 298 to 306 in b45425a

	if image.dim() == 5:
	inputs_llm, atts_llm = [], []
	add_inputs_llm, add_atts_llm = [], []
	for j in range(image.size(2)):
	this_frame = image[:,:,j,:,:]
	with self.maybe_autocast():
	frame_embeds = self.ln_vision(self.visual_encoder(this_frame))
	frame_features =self.visual_encoder.get_intermediate_layers(this_frame)[-2]