MikalDev

Spine team, some of us C3 devs are working on integrating Spine-TS into Construct 3. We've created a C3 JS template project which rendered to a separate Canvas (and WebGL context) and then used that canvas as a texture for a new plugin. This worked well with desktop GPUs with limited testing. Igor also created a more full-featured plugin using the template and plugin as a base. Again this worked well on desktop GPUs with some limit testing.

However, we found that on mobile (at least iPhone GPUs), that the gl function (texsubimage2d()) to upload a new texture each frame took too long (apparently due to their tiled/pre-sort render architecture.) Perf impact of 60 fps -> 30 fps with just one Spine character due to the texture upload. So for mobile optimizations, back to the drawing board.

So here's where we are looking for a little help. Instead of rendering to a separate context we are thinking to render to the same Canvas / webgl context that is being used by C3, we have a couple of options: render directly to the canvas or render to a texture and use that texture for a C3 object (e.g. a custom plugin w/ it's dedicated texture.)

I have been able to attach Spine-TS to the C3 canvas (had to change one minor thing in Spine-TS, C3 is using a 'webgl2' context, not 'webgl' so added a check for that also - will submit a PR at some point for that, if we get something working.) Render to C3 from spine-ts to C3 canvas works and the plugin can control animations. However, we also need C3 to continue to render, but it seems like the Spine render is doing something to the webgl state that C3 does not expect, which is not too surprising. I have done some save/restore of webgl state around the spine-ts render using gl-state and added in some more on suggestion from C3 plugin devs (e.g. VAO save for webgl2), however, it still has issues. Are they any suggestions or hints from the Spine team on how to integrate Spine-TS in with another render engine using WebGL in the same context? Thanks for the help.

C3 Forum w/ template and plugin.
MikalDev
  • Сообщения: 20

badlogic

Cool! Can you give me a bit more information which of the spine-ts/webgl APIs you are using to render Spine models? Probably the easiest code for rendering to integrate is the SkeletonRenderer class, which uses PolygonBatcher to do the WebGL heavy lifting and state modifications. The batcher does:

1. Enable blending and change blend modes
2. Set the appropriate shaders
3. Set VBO (or VAO in your case) bindings
4. Set texture bindings

The batcher does not clean up any of these things, assuming that subsequent drawing code will rest these things to something else anyway.

I'm not familiar enough with C3's rendering setup, but I believe it shouldn't be to hard to restore the WebGL state C3 relies on after drawing a skeleton through our spine-ts rendering code.

Alternatively, you could modify SkeletonRenderer to use a custom batcher implementation that uses C3 APIs. Last time I checked, C3 couldn't render arbitrary triangle meshes though.
Аватара пользователя
badlogic

Mario
  • Сообщения: 2108

MikalDev

We got it working, saving appropriate state (C3 VAO, C3 Shaders, C3 blend mode) and creating and binding a new VAO for Spine use. Your list is definitely helpful to know what changes in Spine. SceneRenderer.drawSkeleton is being used (any comment on that vs SkeletonRender?).

C3 has the assumption that it owns the webgl state completely, so we need to do a little work to save and restore the state and also attach to the C3 canvas. Hopefully, the changes will go into another devs plugin soon and then there will be a plugin with decent perf (mobile and desktop) working and released after that. Currently, the C3 actions are basic to get started, playanimation, update bounds, etc.
MikalDev
  • Сообщения: 20

badlogic

Nice! SceneRenderer uses a SkeletonRenderer underneath and has APIs to draw stuff other than skeletons via WebGL. It's likely overkill for what you want to achieve. SkeletonRenderer and PolygonBatcher should be mostly all you need.

Keep us posted on this, sounds exciting! I'm especially interested how you import the Spine data into C3 via their SDK. When I last looked at C3's API it was not obvious how to get that going with all the files making up a skeleton (.json/.skel, .atlas, .png).
Аватара пользователя
badlogic

Mario
  • Сообщения: 2108

MikalDev

Thanks, I submitted a PR to Spine-TS (for webgl2 context support) and also a PR to the C3 plugin (that changed to render to texture vs separate canvas and texture upload.) They are definitely still some things to review (C3 effects work on, but some state needs to be worked out as it's not consistent, also low-quality full-screen scaling fails in C3.) The plugin also needs at least a few more features to be added (like picking a skeleton when multiple are available (I have hacked the plugin to allow that, the plugin needs a consistent way that uses ACEs.) There also some perf optimizations (batch Spine render for all Spine objects, so only need to save/restore webgl state once per frame.)

Thanks for the advice on SceneR vs SkeletonR, is there a performance difference, will take a look at it.

I see some intermittent PMA problems too, so working on getting some blend modes appropriately set before and after Spine WebGL render. Any hints would be good, I've read the Spine PMA faq, but still some issues, perhaps with blendFunc() or blendFuncSeparate().

I did have to change Spine-TS to handle resizing for the render to texture buffer rather than render to and resizing the C3 canvas directly. I added an 'issue' to Spine-TS regarding this. My changes are not backward compatible, so I'm not going to create a PR, but at least theses changes show that it will work.

In terms of C3 files, here's what I did in my original C3 JS template and the plugin uses it. It's now pretty easy, just need to import the three files into a C3 project 'FIles' folder, add the names to the Spine plugin properties and it works.
// Creating URI to access C3 project files:
this.pngURI = await globalThis.c3_runtimeInterface._localRuntime._assetManager.GetProjectFileUrl(this.pngPath);
this.atlasURI = await globalThis.c3_runtimeInterface._localRuntime._assetManager.GetProjectFileUrl(this.atlasPath);
this.jsonURI = await globalThis.c3_runtimeInterface._localRuntime._assetManager.GetProjectFileUrl(this.jsonPath);
If you want to try a quick demo, check out the *.c3addon from my branch/PR for the plugin and use the original demo project from the plugin:
https://github.com/MikalDev/c3_spine_plugin/tree/render-to-texture/dist
https://github.com/gritsenko/c3_spine_plugin/tree/master/sample_projects
(Note the current plugin only works with the skeleton / animation in the sample project.)

Igor is the plugin author (great work), their github repo is here:
https://github.com/gritsenko/c3_spine_plugin

C3 Test (w/ edited plugin to allow other skeletons), C3 outline effect on one of the orangegirl skeletons. Integrated into C3 layers, so each Spine object can be in a different C3 z-order and layer.



---

Thanks for the suggestion on reviewing the Skeleton Renderer vs the SceneRenderer, it looks like that will help clear up the resize difficulties and canvas interactions. We can also get the 'webgl2' context external to the Spine-TS, so I think no changes needed there for that either if not using SceneRenderer.

---

I did an update using skeletonRenderer instead of sceneRenderer, everything is cleaner and no changes needed for Spine-TS anymore. Thanks for pointing the way there.

There is a remaining issue around PMA though. I have added a flag/property for PMA for the Spine object, so I can apply it to the renderer:
this.renderer.premultipliedAlpha = this.premultipliedAlpha;
this.renderer.draw(this.batcher, this.skeletonInfo.skeleton);
Is the above correct?

However, for the orangegirl and greengirl meshes the small black borders still appear.

I am wondering if something could be happening on texture load. We are using an existing webgl context which most likely has alpha:true and we can't change an existing context when we get the context for Spine and I see this comment in one of the examples:
// Setup canvas and WebGL context. We pass alpha: false to canvas.getContext() so we don't use premultiplied alpha when
// loading textures. That is handled separately by PolygonBatcher.
Suggestions?
MikalDev
  • Сообщения: 20

badlogic

Yes, setting that field to true should trigger PMA blending. In line 163 of SkeletonRenderer we decide what blend modes to use based on that field value. Context alpha set to true shouldn't interfer. But loading textures can. Maybe C3 sets the pixel storage mode to UNPACK_PREMULTIPLIY_ALPHA_WEBGL? See https://developer.mozilla.org/en-US/docs/Web/API/WebGLRenderingContext/pixelStorei
Аватара пользователя
badlogic

Mario
  • Сообщения: 2108

MikalDev

Thanks for the help, reply, and information! That did fix the issue (C3 is setting it to 'true' in other areas.)

To minimize the impact of changing the WebGL state for Spine vs C3, I'm looking to find where I can 'restore' the state, but I am not sure where that is possible. Currently, I have done this but had to comment out the section that restored the state, since it caused the Spine texture load to have the old PMA issue. Is there another time where it's safe to restore the PMA state to 'true'? Perhaps the textureLoader is async/callbacks, so I need to find a sync point? I thought the GLTexture fcn() was where the texImage2D() call was to load the texture.
// Change PMA format to load spine texture
gl.pixelStorei(gl.UNPACK_PREMULTIPLY_ALPHA_WEBGL,false);
console.log("update:gl.UNPACK_PREMULTIPLY_ALPHA_WEBGL:"+gl.getParameter(gl.UNPACK_PREMULTIPLY_ALPHA_WEBGL));

var textureLoader = function(img) { return new spine.webgl.GLTexture(gl, img); };
this.assetManager.loadJson(this.DEMO_NAME, this.jsonURI);
this.assetManager.loadTexture(this.DEMO_NAME, textureLoader, this.pngURI);
this.assetManager.loadText(this.DEMO_NAME, this.atlasURI);
this.isSpineInitialized = true;

// Restore PMA format to C3 state
// XXX Can not be reset here, causes PMA texture load to be incorrect
// gl.pixelStorei(gl.UNPACK_PREMULTIPLY_ALPHA_WEBGL,true);
MikalDev
  • Сообщения: 20

badlogic

Ah, yes. loads through our AssetManager are indeed async. When you specify a URL to load, we use XMLHttpRequest for non-image data, and HTMLImageElement.src plus callbacks for images.

I think the easiest way to fix this is to wrap setting and unsetting the pixelStorei setting in a custom AssetManager implementation similar to the one we ship:

spine-runtimes/AssetManager.ts at 3.8

The call to new spine.webgl.GLTexture(context, image) needs to be wrapped with the calls to gl.pixelStorei. That should do the trick.
Аватара пользователя
badlogic

Mario
  • Сообщения: 2108

MikalDev

Thanks, will look into that.

The plugin seems to be working pretty well and is fairly stable (I just did a PR to fix a C3 issue with effects for the plugin.) Igor is looking at making it easier to use in the editor, so a useable stable release could be soon (with the basic functions of playing different animations, not all the API hooks yet.)

There is an open PR to fix effects as of this post, which hopefully merges soon.
https://github.com/gritsenko/c3_spine_plugin
MikalDev
  • Сообщения: 20

badlogic

This is awesome! Please keep us updated on the progress. We'll be happy to promote it once it's in a stable state.
Аватара пользователя
badlogic

Mario
  • Сообщения: 2108

MikalDev

We have done some more fixes, enabling effects to work correctly now. We need at least one more ACE/condition to signal the end of an animation, then the most basic mode will be useable.
MikalDev
  • Сообщения: 20

badlogic

Awesome! Is there anything from our end we can help with?
Аватара пользователя
badlogic

Mario
  • Сообщения: 2108

MikalDev

ACEs have been added by Igor, so the basics are now all working.

Check it out here:
https://gritsenko.github.io/c3_spine_plugin/

I still have some PMA errors (I think due to the async nature of loading vs C3 changing PMA format.)

I have not used TS before, so I am not sure how to do the overload that you are suggesting. I edited the build JS code to test but found that it did not work unless I put the gl.pixelStorei(gl.UNPACK_PREMULTIPLY_ALPHA_WEBGL,false) call right before the texture load, e.g. gl.texImage2D().

We would definitely prefer to use the Spine-TS webgl as is, so we can pull down new versions without modifications. Can you give me more information on how to do this (I am sure others could follow your example, but I don't quite understand what to do.)
GLTexture.prototype.update = function (useMipMaps) {
var gl = this.context.gl;
if (!this.texture) {
this.texture = this.context.gl.createTexture();
}
this.bind();
console.log("***update3:gl.UNPACK_PREMULTIPLY_ALPHA_WEBGL:"+gl.getParameter(gl.UNPACK_PREMULTIPLY_ALPHA_WEBGL));
gl.pixelStorei(gl.UNPACK_PREMULTIPLY_ALPHA_WEBGL,false);
console.log("***update4:gl.UNPACK_PREMULTIPLY_ALPHA_WEBGL:"+gl.getParameter(gl.UNPACK_PREMULTIPLY_ALPHA_WEBGL));
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, this._image);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, useMipMaps ? gl.LINEAR_MIPMAP_LINEAR : gl.LINEAR);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
if (useMipMaps)
gl.generateMipmap(gl.TEXTURE_2D);
};
MikalDev
  • Сообщения: 20

badlogic

I do you one better. I've added a global "flag" to GLTexture. Before anything is loaded or instantiated, just set it:
spine.GLTexture.DISABLE_UNPACK_PREMULTIPLIED_ALPHA_WEBGL = true;
In GLTexture.update() that flag is evaluated and it will call pixelStoreI() accordingly. This change is live on both the 3.8 and 3.9-beta branches.

Let me know if that works for you!
Аватара пользователя
badlogic

Mario
  • Сообщения: 2108

MikalDev

Thanks! That worked great. I appreciate the support. I added a new PR to the plugin which uses your updated version.

FYI for anyone who might also use this, the flag I actually used is (note the .webgl)
spine.webgl.GLTexture.DISABLE_UNPACK_PREMULTIPLIED_ALPHA_WEBGL = true;


---

Also just for fun, another Construct 3 example (the robot is a Spine character.)

MikalDev
  • Сообщения: 20

badlogic

Whoops, sorry for omitting .webgl. Awesome demo!
Аватара пользователя
badlogic

Mario
  • Сообщения: 2108

MikalDev

Hi, just wanted to let people know that the basic C3 Spine plugin is working and is available here:

https://gritsenko.github.io/c3_spine_plugin/

For issue or comments, please post here:

https://github.com/gritsenko/c3_spine_plugin/issues
MikalDev
  • Сообщения: 20

badlogic

Awesome! The website still says you use one Canvas per object, leading to perf issues on mobile. Is that still true?
Аватара пользователя
badlogic

Mario
  • Сообщения: 2108

MikalDev

Ah, no we render to texture in the same canvas context, so perf is much better than the one canvas per object approach. We’ll update the website.

---

Updated website.
MikalDev
  • Сообщения: 20


Вернуться в Runtimes