Script Automatic Lipsync?

[slowtiger]

Automatic lipsync is a godsend in some situations. Just open the switch layer tab, open the audio tab, select the one audio file in the scene and it creates all the keyframes.

Well. Unless you have that switch layer in several versions/views, and lots of characters and lots of scenes.

Would it be possible to write a script like
"- search for switch layer named "mouth"
- open its audio tab
- assign the one audio file in the project to it
- search next switch layer named "mouth""
etc
and will Moho create the keys as expected?

[synthsin75]

Not possible by script. The only access we have is to assign a .dat file to a switch. Moho does nothing when trying to use that to apply an audio file.

So we'd be left with maybe copying keyframes. If the phonemes across all switches have the same names, that would work.

But if the phonemes across all switches have the same names, we could just use reference layers of a single switch to keep them synced with the auto-lipsync switch. As long as the sublayers have the same names, it doesn't matter what their content is.

[slowtiger]

Well, that settles that. Thank you!

But that reference layer is an idea I should try.

[slowtiger]

OK, the reference layer idea was nice.

Thinking: if I create a switch layer, put it inside the bone layer, import some images a1 a2 a3, then create a reference layer from this switch layer, put it elsewhere, re-name it exactly as the original layer, then creating automatic lipsynce in the original switch should be replicated in ist reference layer.

Result: This works.

Thinking: Now I exchange the images in the reference layers with other, which are named a1 a2 a3 as well, this should work too.

Result: No, it doesn't. It doesn't make a difference wether I just import those images or replace the souce image image by image: as soon as there's a different bitmap the reference is broken. It also doesn't work to create a reference layer from an empty switch layer and fill the switches later, in this case nothing happens at all.

Conclusion: Referencing from a switch layer only works with an exact copy and identical content of the switches, and all layers inside the reference switch need to show that "reference" arrow marker as well.

Oh, it was such a nice dream ...

[Greenlaw]

Hm...I think it shouldn't matter because the Switch layer itself only cares about the stacking order of the child layers, not the contents of the child layers. You might try breaking the reference connections for the Switch Layer's contents but keep the connection for the Switch Layer (where the switch keyframes are stored.)

[SimplSam]

OK, the reference layer idea was nice.
...
Oh, it was such a nice dream ...

From what I am reading you want multiple switch layers to share the same synchronised switching?

If so, can you not create the master lipsync switch and then copy the master switch keyframes to the replica switch layers - each with their unique content?

[slowtiger]

then copy the master switch keyframes to the replica switch layers

Exactly this is the step I want to avoid - to go inside several switch layers and do something there - it would take the same time as to open each switch and assign that audio file to it.

I think It shouldn't matter because the Switch layer itself only cares about the stacking order of the child layers, not the contents of the child layers.

That was my thought too, but I tested it in every direction and it simply doesn't work. Maybe a feature request? (Too late for this project ...)

[SimplSam]

Exactly this is the step I want to avoid - to go inside several switch layers and do something there - it would take the same time as to open each switch and assign that audio file to it.

OK. The keyframe copying can be scripted. Wes already has a Copy Layer Parameters tool. This would make a nice addition (hint hint).

I think It shouldn't matter because the Switch layer itself only cares about the stacking order of the child layers, not the contents of the child layers.

That was my thought too, but I tested it in every direction and it simply doesn't work. Maybe a feature request? (Too late for this project ...)

Yeah - Switch layers only care about Layer Names, order is not important (except the top layer which gets chosen if no valid child-layer is active).

[Greenlaw]

That was my thought too, but I tested it in every direction and it simply doesn't work. Maybe a feature request? (Too late for this project ...)

It really should work. Here's an example I threw together in Moho 13.5.5



The pose SBD opens and closes the hand, and the turn SBD turns the hand. Notice that pose drawings stay in sync as the hand turns.

On the right, you can see the three pose Switches and two of them are references, but the drawing layers are not referenced, they're all unique drawing layers. So it appears Moho just looks at the layer order, not the layer ID.

When I set up the two Smart Bone Actions, turn is animating the Switch Layer called hand to switch between hand_palm (original), hand_side (ref), and hand_back (ref), and pose is only animating hand_palm. Because hand_side and hand_back are references of the original hand_palm Switch Layer, they follow the animation of the hand_palm.

If this isn't working in your project, make sure the reference switch layers are synchronized to the original switch layer. You can do this by right-clicking each reference and selecting Sync All Channels To Original.

If you referenced the drawing layers, you can break the reference connection by choosing Break Layer Reference. Breaking the references here will prevent any unwanted modifications from occurring.

Here's the project if you want to pick it apart...

multiSwitchHands

Hope this helps.

[synthsin75]

Thinking: Now I exchange the images in the reference layers with other, which are named a1 a2 a3 as well, this should work too.

Result: No, it doesn't. It doesn't make a difference wether I just import those images or replace the souce image image by image: as soon as there's a different bitmap the reference is broken. It also doesn't work to create a reference layer from an empty switch layer and fill the switches later, in this case nothing happens at all.

Conclusion: Referencing from a switch layer only works with an exact copy and identical content of the switches, and all layers inside the reference switch need to show that "reference" arrow marker as well.

Oh, it was such a nice dream ...

No, I just verified that it does work. Maybe you missed the step of right-clicking the switch layer channel and selecting Sync Channel To Original? This works, even if you have extra layers and in a different order. This is because all the switch layer channel stores are the sublayer names that are keyframed. Doesn't matter the layer order or if there are extra, unkeyed layers. You can either change the content of each layer, or even delete all the layers and recreate them with different content. The only thing that matters is that for every switch sublayer keyframed in the original, there exists one with the exact same name in the reference.

Once synced with the new content, the reference will then automatically match the original.

[synthsin75]

OK. The keyframe copying can be scripted. Wes already has a Copy Layer Parameters tool. This would make a nice addition (hint hint).

Unless I've complete forgotten about another script I wrote, I don't think that one's mine.

[SimplSam]

OK - my mistake . It was written by Stan (SZ_CopyLayerParameters).

[Greenlaw]

The only thing that matters is that for every switch sublayer keyframed in the original, there exists one with the exact same name in the reference.

@Synthsin75,
Ooooh! I didn't realize this...probably because I'm so 'OCD' about naming everything that I never had a problem with this. Anyway, I tried changing every layer name and you're right, it breaks things and the setup goes crazy.

@everyone else,
So, yes, be sure to keep your layer names sync'd between Switch Layers. The layers can have totally different content but the names must be consistent.

[Greenlaw]

I tried shuffling the layer order inside two of the switches, and the SDB setup still worked correctly.

So, confirming what Wes wrote above: layer order does not matter but layer names do.

[synthsin75]

The only time layer order matters in a switch layer is when you're not using Preston Blair phonemes and Moho is just using audio amplitude to set the switches. The louder the audio, the higher in the layer order.