Esoteric Software

[brendan.vance]

Did you know that Unity's new UI system allows you to essentially pass a Mesh instance into a CanvasRenderer component and draw said mesh directly onto a Canvas (with proper layering and everything)?

I've been able to hack together a thing for drawing SkeletonRenderer output this way, though my solution is primitive; it doesn't support submesh separators, for instance, and there are of course annoying update order problems because I'm using a separate component to stick the mesh into the CanvasRenderer on late update.

The easier solution would be to code CanvasRenderer support directly into the SkeletonRenderer component, maybe using toggles so you can enable/disable the corresponding 3d MeshRenderer! Would y'all consider working on this?

[Pharan]

We could look into it.
If you've already started work on it, do share your findings or code!

Not really sure how it works yet but I've noticed Unity's UI API seems to be in constant flux, since 4.6 and with changes in 5.0, 5.1 and 5.2. I wonder if 5.3 has any more changes. I've had to fix some UI effects every time it breaks with an update.

Right now though, MeshRenderers can work with UI if you use Screen Space - Camera on the Canvas.

[brendan.vance]

We have investigated Screen Space Camera! The issue we encountered was that we wanted Spine rigs on our loading screen, which has to appear above everything else in every other scene (including Screen Space Overlay UI components, which I believe can ONLY be occluded by other Screen Space Overlay components).

This is what we're doing at the moment:

using UnityEngine;
using System.Collections;

[RequireComponent(typeof(CanvasRenderer)), ExecuteInEditMode]
public class MeshRendererToCanvasRenderer : MonoBehaviour
{
// Update is called once per frame
void LateUpdate()
{
var meshFilter = GetComponent<MeshFilter>();
var meshRenderer = GetComponent<MeshRenderer>();
var renderer = GetComponent<CanvasRenderer>();

Mesh mesh = meshFilter.sharedMesh;
renderer.SetMesh(mesh);

Material[] materials = meshRenderer.sharedMaterials;
if(renderer.materialCount != materials.Length) {
renderer.materialCount = materials.Length;
}

for(var i = 0; i < materials.Length; ++i) {
renderer.SetMaterial(materials[i], i);
}
}

void OnDisable()
{
var renderer = GetComponent<CanvasRenderer>();
renderer.SetMesh(null);
}
}

It works for our purposes right now, but there are quite a few issues with it:

1) SkeletonRenderer force-enables its corresponding MeshRenderer component every update, meaning there will be spine rigs both in the 3D scene AND on the Canvas unless we do a bunch of stuff with layers and camera masks (and of course, we'd have to do it with every camera in our many scenes)

2) We need to put this script into the back of Unity's annoying 'script execution order' to ensure its LateUpdate() method gets called after Spine's, which is not ideal

3) This code is pretty crude and I don't think accounts for fancy Spine features like submesh separators

I think the most elegant solution would be to go into SkeletonRenderer and program a sort of 'toggle' between using MeshRenderers vs CanvasRenderers; at that point, however, I would of course have to deal with syncing my code to future Spine Runtime updates. Thus, I am proposing it as a feature request!

[Pharan]

I don't think it's a good idea to port submesh separators as it is.

However, if the only thing CanvasRenderer needs is a mesh and materials, it should be easy.
Because pretty much the only thing SkeletonRenderer does is make a mesh, pick materials, and give those to MeshFilter and MeshRenderer. Making a version that passes those things to CanvasRenderer instead sounds trivial.

I'll try to work on an official-ish solution for this soon, then stick it in the official runtime shortly after we prove it works well.

[townbully]

Hello..

Would this allow spine animations to work with the UI mask element correctly when placed within a new style Unity UI?

I have a super simple Unity project that demonstrates the mask problem if it would be helpful to provide.

-Jeff

[Pharan]

I'm honestly not sure. I don't know who's looked into how masking works already.
And I read they're planning to change it too in a 5.x version.

[townbully]

I tried testing with my UI masked spineboy with the script above from Brendan added to it, execution order tinkering, etc, but no dice. Seems that even if the canvas renderer is using the mesh and materials, the spine meshrenderer must be enabled in which case it doesn't observe the mask.

[Pharan]

Here's SkeletonGraphic: https://gist.github.com/pharan/d06c771a696fbf7ec733
(EDIT: This SkeltonGraphic is the old implementation. The latest official implementation is now a Module and part of the official runtime when you download it.)

This is a naive port and merging of SkeletonRenderer and SkeletonAnimation into one script that works in Canvases, sans the incompatible features like submesh separators. You use it like the other Unity UI components and the scripting API is similar to SkeletonAnimation. I haven't ripped out the part of the rendering code that does submeshes. Ideally, it should outright reject skeletons that use multiple materials since it's just not compatible.
The single script should work by itself.

- This hasn't been thoroughly tested yet but I figured it'd tide you over for basic cases. It does work. The inspector works. It animates, updates. You can set freeze to true in case you want to stop it from updating the mesh, but don't want to clear the mesh inside CanvasRenderer.
- You can't use skeletons that use more than 1 material (this is a Canvas/CanvasRenderer limitation, afaik. SetMaterials(material, int) doesn't seem to work or make sense. no documentation on it either.)
- It has a few editor conveniences but no Instantiate shortcut from Project view. But all you usually need to do is assign it a SkeletonDataAsset and it'll start working.
- It uses the color property that comes with the Graphic class. Haven't seen if it interacts with the grouped alpha tweens that canvas elements have.
- It also uses the Reference Pixels Per Unit scale set on your Canvas's CanvasScaler.
- It doesn't use the Rect on the RectTransform since that actually involves having static bounds which Spine.Skeletons simply don't have. You can resize via the transform scale, and alignment is done by moving the RectTransform pivot.
- It also doesn't have support for Masking.

I'm busy these days so I don't think I'll have time to work on this much more anytime soon.
Masking involve some stuff about the stencil buffer and possibly some shader stuff. That's research I can't get into right now.
But if anyone wants to take a whack at switching it over to MaskableGraphic or whatever other features you need, feel free.

-- 11 Feb 2016 2:11 am --

SkeletonGraphic is going to get a new cleaner (temporarily less performant) implementation soon with compatibility with UnityEngine.UI Mask and RectMask2D.

Did some testing. Shader needed to check the stencil buffer for 1. But the MaskableGraphic class needs to do some other things to change each graphic's material/shader depending on whether or not it's a child of a Mask.



-- 22 Feb 2016 12:03 pm --

A couple of things about the new runtime, when I updated (downloaded a new copy from the github), it said this field doesnt exist


I just commented it out and haven't noticed it affecting anything in my game but just thought I'd tell you.

@Pharan

Running on Unity 5.3.1p3.

@Solgryn
The compilation error isn't making much sense since you're using 5.3.
It's supposed to be accessing this property, which DOES exist as it says right there in the docs: http://docs.unity3d.com/ScriptReference/UI.Graphic-raycastTarget.html
I'll try to look into this.

-- 22 Feb 2016 12:10 pm --

A note to everyone else, SkeletonGraphic is now part of the official runtime under Modules. You can find a sample scene in the Examples folder.
If you don't need it or it causes you problems, you can delete the whole folder named SkeletonGraphic without affecting anything else in your project.

I've already tested it some and squashed a few bugs here and there. And the instantiation shortcut is also already in the project menu if you right-click on a SkeletonDataAsset. Feel free to use it and report any problems.

It's only ever so slightly slower than how SkeletonAnimation does things. The difference is small enough that any slowdown you have is much more likely because of UnityEngine's current UI performance issues.

It has some small API differences.
- You'll have to access its Skeleton and AnimationState through SkeletonGraphic.Skeleton and SkeletonGraphic.AnimationState properties, unlike SkeletonAnimation which lets you access the fields directly.
- Its Skeleton color is controlled by SkeletonGraphic.color (inherited from UnityEngine.UI.Graphic.color) so changing the Skeleton's color directly will yield unpredictable results. But this means (at least it should) that you can use it with grouped and animated alpha fading in UI elements.
- As it was with the first implementation of SkeletonGraphic, because of the limitations of the UI system, you can't use skeletons that require more than one Unity Material. It just doesn't work.
- You can right-click on the component in the Inspector and you have the option to resize the RectTransform's rect to fit the currently rendered skeleton in the editor. Graphic design-wise, this is usually not a good idea since the skeleton's mesh usually has a bunch of transparent and uneven stuff sticking out of it, but it does make for a good starting point in some cases.

I'll be making a how-to-use video for it soonish.

[brendan.vance]

We have yet to update to the latest Spine runtime (we're very near to our ship date and it's getting crazy over here), but I can report that we are having some success with the crude method I proposed a while back (shoving meshes/materials into CanvasRenderer and hacking Spine not to force-enabled the MeshRenderer component).

I have not noticed an issue with multiple materials using this cruder technique, which has me concerned. I wonder if there might be differences between the version of Unity we're using (5.3.1p3) and the one you're using?

[Pharan]

There should have been no problems if your skeleton was only rendering one texture.

When I said "multiple materials", I meant using a material array.
The way you did it, you shoved the mesh into CanvasRenderer. What did you do with the Material array?

Anyway, you're busy with crunch with release approaching so.. good luck!

[brendan.vance]

I am straight-up duplicating whatever material array SkeletonRenderer creates (tho changing to a UI-specific shader), in the following fashion:

[RequireComponent(typeof(CanvasRenderer)), ExecuteInEditMode]
public class MeshRendererToCanvasRenderer : MonoBehaviour
{
public Material[] Materials;

// Update is called once per frame
void LateUpdate()
{
var meshFilter = GetComponent<MeshFilter>();
var meshRenderer = GetComponent<MeshRenderer>();
var renderer = GetComponent<CanvasRenderer>();

Mesh mesh = meshFilter.sharedMesh;
renderer.SetMesh(mesh);

if(Materials == null || meshRenderer.sharedMaterials.Length != Materials.Length || renderer.materialCount != Materials.Length) {
var shader = this.InjectResolveType<WorldConstants>().SpineCanvasShader;

Materials = new Material[meshRenderer.sharedMaterials.Length];
for(var i = 0; i < Materials.Length; ++i) {
Materials[i] = new Material(meshRenderer.sharedMaterials[i]);
Materials[i].shader = shader;
}

renderer.materialCount = Materials.Length;
}

for(var i = 0; i < Materials.Length; ++i) {
renderer.SetMaterial(Materials[i], i);
}
}

void OnDisable()
{
var renderer = GetComponent<CanvasRenderer>();
renderer.SetMesh(null);
renderer.Clear();
}
}

I haven't done extensive testing on mobile and such, but in the editor at least, it appears to work even with material arrays.

[Pharan]

I guess this calls for a separate of SkeletonGraphic that does multiple materials but gives up its MaskableGraphic inheritance 'cause that has a lot to it in terms of hooking into other UnityEngine.UI stuff, custom materials.
Good thing I already finished a version of the MeshGenerator that does pretty much what SkeletonRenderer does last week.

I'll just give them some unifying interface definitions so people can handle them similarly.

I should review that part where it force-enables MeshRenderer. I don't remember that part at all.

EDIT:
I see the part. Just checks if the MeshRenderer is enabled and doesn't generate the mesh if it's disabled. Seems fair to remove that check if you needed it to do invisible stuff like that.