Want to create both video and audio clip from same reference (mp4)

[ojadeyemi]

For some reason, whenever I want to add an audio clip to an audio track, it doesn't work unless the target URL is mp3 and not mp4. But I would like the audio clip to reference the same video so that when I export the fcpxml and import it into Premier Pro, I can get both video and audio in my sequence.

# refrence of audio clip
audio_reference = otio.schema.ExternalReference(
    target_url=f"vids/plays.mp3", available_range=video_range #if target URL is plays.mp4, clip file isn't made
) 

audio_clip = otio.schema.Clip(
    name="plays.mp4", source_range=clip_range, media_reference=audio_reference
)

#get this when target URL = vids/plays.mp4 in XML output instead. *get the correct output when target is mp3* 
 <audio>
  <track>
    <clipitem id="clipitem-1"/>
  </track>
</audio>

---

On windows 11 using visual studio code.
Using python 3.12
OpenTimelineIO==0.17.0
OpenTimelineIO-Plugins==0.17.0
otio-fcpx-xml-adapter==1.0.0

[ojadeyemi]

I have read this comment but it didn't seem to help
#877 (comment)

Here is a sample file

import opentimelineio as otio


video_data = {
    "video_file_name": "plays.mp4",  # AAron goorden dunking
    "start_time": 285.013,
    "end_time": 295.014,
    "video_length": 600,
}

FRAME_RATE = 30

video_filename = video_data["video_file_name"]
start_time = video_data["start_time"] * FRAME_RATE
end_time = video_data["end_time"] * FRAME_RATE
video_length = video_data["video_length"] * FRAME_RATE
duration = end_time - start_time

# for now there's only one clip
total_duration = duration

global_start_time = otio.opentime.RationalTime(0, FRAME_RATE)

# Create new timeline
timeline = otio.schema.Timeline(name="playstimelinetest", global_start_time=global_start_time)

# Range of clip
clip_range = otio.opentime.TimeRange(
    start_time=otio.opentime.RationalTime(start_time, FRAME_RATE),
    duration=otio.opentime.RationalTime(duration, FRAME_RATE),
)

# full length of video
video_range = otio.opentime.TimeRange(
    start_time=otio.opentime.RationalTime(0, FRAME_RATE),
    duration=otio.opentime.RationalTime(video_length, FRAME_RATE),
)

# full length of track
track_range = otio.opentime.TimeRange(
    start_time=global_start_time,
    duration=otio.opentime.RationalTime(duration, FRAME_RATE),
)

# Create new track
video_track = otio.schema.Track(
    name="videotrack", kind="Video", source_range=track_range
)

# refrence of video clip
video_reference = otio.schema.ExternalReference(
    target_url=f"vids/{video_filename}", available_range=video_range
)

# Create video clip
video_clip = otio.schema.Clip(
    name="plays.mp4", source_range=clip_range, media_reference=video_reference
)

#Creat audio track
audio_track = otio.schema.Track(
    name="audiotrack", kind=otio.schema.Track.Kind.Audio, source_range=track_range
)

# refrence of audio clip
audio_reference = otio.schema.ExternalReference(
    target_url=f"vids/{video_filename}", available_range=video_range #doesnt seem to work
)

audio_clip = otio.schema.Clip(
    name="plays.mp4", source_range=clip_range, media_reference=audio_reference
)


# append clip to track line
video_track.append(video_clip)
audio_track.append(audio_clip)

# append track to timeline
timeline.tracks.append(video_track)
timeline.tracks.append(audio_track)


# convert timeline to xml
otio.adapters.write_to_file(timeline, filepath="test/test2.xml", adapter_name="fcp_xml")

[apetrynet]

Hi @ojadeyemi

I'm not sure how Premier responds to mp3 vs mp4, but I noticed something that might cause an issue.
I see you defined some lengths in a dict at beginning of the file. These seem to be in seconds.
Could you try to create your RationalTime instances using the otio.opentime.RationalTime.from_seconds() method like the example below and see if it helps?

clip_range = otio.opentime.TimeRange(
    start_time=otio.opentime.RationalTime.from_seconds(start_time, FRAME_RATE),
    duration=otio.opentime.RationalTime.from_seconds(duration, FRAME_RATE),
)

[ojadeyemi]

Hi update. That's all Premiere needed to work. I made some mistakes when importing it and made assumptions. (I exported the XML from Premiere and was trying to recreate it using Otio).

But it looks like as long as the audio is linked to the clip_id of the video clip..all is well. I will now try on multiple video clips and see what happens then.

Also thanks for the feedback I will apply the otio.opentime.RationalTime.from_seconds() method and see how that changes.