ffmpeg voodoo


Author: Dave
Date: 05.26.24 - 3:21am


Ok So ffmpeg is kind of magical. I have zero experience with audio and video editing really.

I want to take my old videos (like 100 of them) and swap out my voice with an AI voice.

I tried the elevenstudios.io dubbing feature where you submit a video and it does it automatically. It was horrible. Even when tried to use the studio feature to customize it it got worse.

Sooo...workflow is:
  1. extract the audio from the video
    ffmpeg -i "Vb6 Pcode For Loops Disasm.mp4" -vn -acodec copy ./forloops.mp4

  2. split it into < 5 minute clips
    ffmpeg -i forloops.mp4 -f segment -segment_time 299 -c copy parts/output%09d.mp4

  3. submit each clip to elevenlabs.io for translation (they do have an api, Usage sample)

  4. recombine the clips
    ffmpeg -f concat -i concat_files.txt -c copy final.mp3 -y

  5. reintegrate the clips back into the video
    ffmpeg -i "Vb6 Pcode For Loops Disasm.mp4" -i final.mp3 -c:v copy -map 0:v:0? -map 1:a:0 forloops_new.mp4

Ill be damned if ffmpeg on the command line cant do it all! (except the ai submission)

I am mostly posting this here so I dont loose it. I will probably write a GUI to manage the cmdlines and paths shortly. Im amazed though..Still lots of voodoo so thanks to everyone whos cmdlines i ripped off from google searches!

Came out well, you can see the output here. Check the boundaries between the clips at the 5 minute marks.

I wish the voices were a little better, but for now its acceptable.

Also to the ffmpeg authors, you guys are fricking ninjas.

So I ended up writing a quick gui tool to help with the processing. AI voice conversion of a video is now reduced to two clicks. (select video, then full auto to convert)

I have spent ~$200 on AI credits on this project plus another $800 for the vids I initially had professionally edited. I had been putting this off for a year, so glad its finally done even though it took an entire holiday weekend.







Comments: (0)

 
Leave Comment:
Name:
Email: (not shown)
Message: (Required)
Math Question: 17 + 34 = ? followed by the letter: D 



About Me
More Blogs
Main Site
Posts: (All)
2024 (2)
     ffmpeg voodoo
     RegJump Vb
2023 (9)
     VB6 Virtual Files
     File handles across dlls
     python abort / script timeout
     py4vb
     VB6 Python embed w/debugger
     python embedding
     VB6 IDE Enhancements
     No Sleep
     A2W no ATL
2022 (4)
     More VB6 - C data passing
     Vb6 Asm listing
     Byte Array C to VB6
     Planet Source Code DVDs
2021 (2)
     Obscure VB
     VB6 IDE SP6
2020 ( 4 )
2019 ( 5 )
2018 ( 6 )
2017 ( 6 )
2016 ( 22 )
2015 ( 15 )
2014 ( 25 )
2013 ( 4 )
2012 ( 10 )
2011 ( 7 )
2010 ( 11 )
2009 ( 3 )