Page 1 of 1

Rhubarb Auto Lip Sync

Posted: Wed Feb 10, 2021 5:53 am
by arglborps
I haven't tried it out yet, but there's an open source software for automated lip synching called Rhubarb that works with all major animation tools, and it seems quite powerful.
There's even preliminary support for non English languages.

https://github.com/DanielSWolf/rhubarb-lip-sync

The developer, Daniel Wolf, is very friendly and responsive when I pinged him on Twitter.

Rhubarb on Twitter
https://twitter.com/RhubarbLipSync

The Developer on Twitter
https://twitter.com/Daniel_S_Wolf

Re: Rhubarb Auto Lip Sync

Posted: Wed Feb 10, 2021 9:08 am
by DK
Says it works off the command line tool. Love to hear any reviews.

Cheers
D.K

Re: Rhubarb Auto Lip Sync

Posted: Wed Feb 10, 2021 1:09 pm
by Onionskin
I tried Rhubarb auto lip sync and it works well, if I have a lot of dialog scenes I would definitely go with Rhubarb.
Minus: not user friendly to work with command line tool, the best would be to have some third party tool for comfortable work, just like the Rhubarb plugin for After effects.

Re: Rhubarb Auto Lip Sync

Posted: Wed Feb 10, 2021 7:14 pm
by synthsin75
Someone could always write a Moho Lua frontend for this. Don't know if it's something I would want or need.

Hopefully one of the other scripters has enough interest to write it, but if not, maybe there's enough people/interest to fund getting it written.

Re: Rhubarb Auto Lip Sync

Posted: Thu Feb 11, 2021 9:47 am
by Onionskin
synthsin75 wrote: Wed Feb 10, 2021 7:14 pm Someone could always write a Moho Lua frontend for this. Don't know if it's something I would want or need.

Hopefully one of the other scripters has enough interest to write it, but if not, maybe there's enough people/interest to fund getting it written.
If that happens Papagayo is history

Re: Rhubarb Auto Lip Sync

Posted: Mon Jan 08, 2024 7:55 pm
by Yosemite Sam
Onionskin wrote: Thu Feb 11, 2021 9:47 am
synthsin75 wrote: Wed Feb 10, 2021 7:14 pm Someone could always write a Moho Lua frontend for this. Don't know if it's something I would want or need.

Hopefully one of the other scripters has enough interest to write it, but if not, maybe there's enough people/interest to fund getting it written.
If that happens Papagayo is history
I can't find anything on how to actually use this with Moho. I watched the spine video, but not sure how to use it with switched in Moho. Any advice?

Re: Rhubarb Auto Lip Sync

Posted: Tue Jan 09, 2024 8:42 am
by Onionskin
Yosemite Sam wrote: Mon Jan 08, 2024 7:55 pm I can't find anything on how to actually use this with Moho. I watched the spine video, but not sure how to use it with switched in Moho. Any advice?
Try here

https://wiki.visionaire-tracker.net/wiki/Lip_sync

Re: Rhubarb Auto Lip Sync

Posted: Thu Jan 11, 2024 1:39 pm
by Víctor Paredes
Hi, Moho already uses Rhubarb for auto lip sync. You can simply import the audio you want to use, have the Preston Blair's phonemes in a switch layer and then open the Switch layer properties, Switch tab, Select audio sync source.
Once you hit OK, Moho will generate the lip sync keyframes.

Re: Rhubarb Auto Lip Sync

Posted: Thu Jan 11, 2024 1:56 pm
by lucasfranca
Does any name layer work, or does it need to be sequential numerals, Victor?

Re: Rhubarb Auto Lip Sync

Posted: Thu Jan 11, 2024 4:40 pm
by Víctor Paredes
If you use the Preston Blair's phonemes (AI, O, E, U, etc., L, WQ, MBP, FV, rest) the lip sync will be as accurate as possible.
If you use any other numbers, Moho will use the audio volume instead and add keframes following the order of the layers inside the switch (the layer on top will be the louder and vice-versa)