• Runtimes
  • VTubing update?

Actually I don't know where this ought to fall under. Is there an update to the VTuber use case using Spine? I see there's a test on getting facial movements in sync with a webcam and a Spine animation on Itch. Are there any updates to this?

Thanks!

  • Misaki ответили на это сообщение.
    Related Discussions
    ...
    • Изменено

    The development of using Spine for VTubing is ongoing, with several updates and improvements being made over time by SilverStraw. The Spine Vtuber Prototype has seen multiple iterations, with improvements in face tracking, animation smoothness, and the ability to handle multiple keyframes and face meshes.

    One of the key updates includes the ability to record motion captured from a camera and export it as a JSON file, which can then be imported into Spine. This feature is particularly useful for creating animations that sync with live recordings.

    For more detailed information and to follow the latest updates, you can check out the ongoing discussions and shared prototypes in the Spine forum threads, such as the Spine Pro Vtuber Working Prototype.

    • bapho ответили на это сообщение.

      bapho Just in case you are wondering what is going on within the Spine team, we have been testing Vtubing for some time, but for a while now, we have been too busy releasing v4.2, adding new runtimes, improving existing runtimes, etc., to make much progress. We have not lost interest in Vtubing and may eventually get back into it when we have more time.

      • mikedssx ответили на это сообщение.

        All right, thanks for the heads up!

        I looked over that and last update was May, so I thought it might have stalled or something.

        • T.Fly() ответили на это сообщение.
        • Misaki оценил это.

          bapho I don't think Silverstraw's efforts are official. It's his personal project afaik.

          9 дней спустя

          Yes, it's unofficial.

          • bapho ответили на это сообщение.

            SilverStraw
            Hey thanks for your work btw! Is this still an ongoing WIP?

            • SilverStraw ответили на это сообщение.

              bapho Yes. Had been busy with work; didn't have get the time to work on the project.

              • bapho ответили на это сообщение.
                6 дней спустя

                SilverStraw

                I see. I also just wanted to drop some word of encouragement 🙂. Any part of this needs testing?

                • SilverStraw ответили на это сообщение.

                  bapho I update it to the latest runtime version. Physics constraints probably need more testing as Esoteric had several revisions to fix bugs. I also wanted to test how well the AI model is able to guess the depth so I can add some draworder depth. Something like how to tell if the right hand or left hand is in front of each other.

                  12 дней спустя

                  Thanks for the update! Just got a little busy, but I'll try and find some time to check things out

                  7 дней спустя

                  @SilverStraw - I'm looking at the demo file, and this is still based on v4.1.x. Is there a 4.2.x specific version of this, or do I just export, reimport, and save as a 4.2.x file?

                  • SilverStraw ответили на это сообщение.

                    bapho I just updated the demo files. There should not be too many differences since I did not incorporate physics constraints yet.

                    11 дней спустя

                    Ok. And when you say AI, does this upload image data to be processed somewhere with AI computing, or it's just the local webapp powered by JS on browser that does the processing?

                    • SilverStraw ответили на это сообщение.

                      bapho It's a local web assembly that does the inferring while Spine-WebGL does the rendering.

                      3 месяца спустя

                      Misaki

                      I've been a long-time admirer of Spine and the incredible work Esoteric has done in the 2D animation space. However, I can’t help but feel that Spine has been missing out on a massive opportunity which is VTubing.

                      Live2D has dominated the market, largely because it was one of the first major tools to cater to VTubers. But Spine’s workflow is, in many ways, more intuitive and powerful, particularly when it comes to mesh deformation and smooth animation blending. The ability to easily rig a character with Spine and create high-quality animation sequences already makes it a perfect fit for VTubing.

                      The demand for VTuber models has skyrocketed over the years, yet the barrier to entry remains high due to the cost of Live2D rigging commissions. If Esoteric provided dedicated VTuber export tools, support for real-time facial tracking, and an easy way to integrate Spine models into streaming software, it could open the door to a whole new market. Many aspiring creators would likely opt for Spine over Live2D if it meant a more affordable and user-friendly path to becoming a VTuber.

                      I truly believe that investing in VTuber-focused features and marketing could bring Spine to an entirely new audience, increase license sales, and position it as a serious competitor in this growing industry. I’d love to hear thoughts from the community and, hopefully, the Esoteric team on whether this is something that could be explored in the future.

                      Perhaps bringing in a team specifically to develop VTuber-focused features for Spine could be a smart move. Once implemented, it could create a steady revenue stream that helps support Spine’s other core development areas. Seems like a win-win to me

                      What do you think?

                      • Misaki ответили на это сообщение.
                      • Harald оценил это.

                        mikedssx Thank you so much for taking the time to share your opinion. This is my personal opinion, not the consensus of the Spine team, but I agree with you. If Spine could provide functionality for vtubing, there would be a large number of users who would use it and it would be good for our revenue.

                        Ultimately, the problem is that we don't have the runtime development resources to immediately start developing the players and other things needed to make Spine more useful for Vtubing. As you wrote, the functionality of the editor is mostly sufficient to create models for vtubing, what is missing is the runtime or player to easily integrate Spine models into streaming software.
                        We update our software constantly and quickly, but unfortunately there are still users waiting for bug fixes or improvements. In order to continue to stabilize and update the existing runtimes, supporting vtubing might be too much for us.
                        Having said all that, in any case we are not giving up on Vtubing. If a generous contributor appears, as was the case with the Godot runtime, it may hasten the time when we can make it a reality.