Этот сайт лучше всего просматривать в современном браузере с включённым JavaScript.

Question about face scribbling feature

aaazsd

I'm working on a feature where players can scribble on defeated enemies' faces. After a player defeats an enemy, they can draw on the enemy's face.

First idea came up to my mind was implementing a raycast system that detects which attachment is hit at the brush position, returning the topmost attachment (assuming the collision area is not fully transparent).
Based on the raycast hit result, I can map from canvas coordinates to the slot/attachment coordinates.

My question:
Am I on the right track? I'm not aiming for very high performance or visual quality.

Any suggestions or alternative approaches would be appreciated. Thanks!

Spinebot

Based on my knowledge, your approach with raycasting to detect attachments is on the right track! Here are some suggestions for implementing this feature:

Your Raycast Approach:
Yes, this is a valid starting point. You can iterate through visible attachments in draw order and check if your brush position intersects with their bounds. For each attachment, you can get its bounds using the attachment's properties (x, y, width, height).

Alternative and Complementary Approaches:

Use a separate damage/scribble mask texture:
Instead of modifying the original atlas texture directly, create a lower-resolution grayscale mask texture. This mask can then be used as a blend factor between the normal texture and a "scribbled" version. This approach:
- Is more performant (lower resolution)
- Preserves the original atlas
- Allows for easier reset/cleanup
Mesh-based approach:
If your face attachments use meshes, you could overlay a separate mesh attachment with your scribble texture on top of the face. This would be easier to manage and wouldn't require complex texture manipulation.
Pre-defined scribble variations:
The simplest solution would be to prepare 3-4 pre-made "scribbled face" variations in Spine and swap attachments when the player defeats an enemy. You could either:
- Switch the attachment at the slot
- Overlay a damage/scribble attachment on top

Important Considerations:

Never modify the original atlas texture - always work with a copy, otherwise all skeleton instances will be affected
Consider the performance impact of texture modifications during gameplay
If you're instantiating multiple enemies, texture operations per instance could add up

For a feature that doesn't require very high performance or visual quality, starting with pre-made scribble variations as overlays would be the easiest to implement and maintain!

Harald

@aaazsd It also depends how complex your drawing-area / face is set up - is it a single attachment, or multiple? Does it move while/after being drawn to, i.e. does it still allow passive ragdoll-like interaction?

As spinebot mentioned, it would be a good idea to use a second (writable) mask-texture which is edited and used to overlay the color. Basically the full-fledged approach of implementing a raycasting solution might be good if you need to support arbitrary skeletons. You would then e.g. iterate the attachments of skeleton in the order of Skeleton.DrawOrder and then check for the transformed region whether it contains the given painting-position. Then you would need to get the UV coords of the corners of the attachment region and paint only into this region (to not bleed onto adjacent regions in the atlas).

Note that if you can modify your Spine project accordingly, you could create a single draw-only attachment which you then exclusively be used for paining to. Then you could just test a single region for point-inclusion instead of testing all slot's attachments, and update only a single small texture instead of an atlas where you paint only inside a limited rectangle.

aaazsd

I've completed the raycast system, custom shader, RenderTexture setup, and drawing with brush strokes.

However, I'm still working on the coordinate transformation from skeleton space to texture space (UV coordinates).

Currently, I have:

Screen position converted to skeleton space (I believe this is working, though not fully verified)
Iteration through slots by draw order to find the hit attachment

I need a function that can convert a point in skeleton space to texture space (UV coordinates) for a given attachment. This transformation cannot be static data, because the character is animating during the drawing procedure.

I would like to know: should I implement this transformation myself, or is there an existing API for this?

My closest guess would be attachment.ComputeWorldVertices(), which I found in the Generic Rendering section: https://esotericsoftware.com/spine-runtime-skeletons#Generic-rendering

However, I haven't figured out how to use this function for my use case.

The strokes are visible on the hair, but the transformation is currently incorrect.

Harald

aaazsd only now have i realized that I used ffd.

If you use a MeshAttachment and mesh deformation keys (vertex deformation animation), you need to test against each of the deformed triangles and get the UV coords from that result.

Please note that mesh deformation keys are costly and not recommended, instead weighted meshes are the recommended way. See the FAQ here.

aaazsd My closest guess would be attachment.ComputeWorldVertices(), which I found in the Generic Rendering section: https://esotericsoftware.com/spine-runtime-skeletons#Generic-rendering

There is no method provided out of the box to transform skeleton-space coords to UV space, you would need to write that yourself. ComputeWorldVertices is for updating the vertices, this is all automatically called where needed already, so won't help you much. What you would need to do instead is access the generated transformed mesh vertices to get the transformed skeleton-space coords, as they are written by SkeletonRenderer.MeshGenerator to a single vertex buffer and are stored no-where else like e.g. each Attachment.

You can register a callback to SkeletonRenderer.OnPostProcessVertices and then access the MeshGeneratorBuffers at the right time.

aaazsd I recalled that this transformation exists in shader.

No, the FFD mesh deformation animation does not happen in a shader but on the CPU, if that's what you meant.

aaazsd

only now have i realized that I used ffd.
The transformation is not per attachment but per triangular fragment.

aaazsd

I recalled that this transformation exists in shader.
My current drawing shader used uv. If I could work out the transformation in GPU. However, I lost the information of attachment of raycasted. And I need a way to get transformed circle/brush.
I am not sure if this workaround is perferable.

aaazsd

Thank you very much for your help. This information is extremely valuable.

I would have spent much longer on this feature if I hadn't gotten frustrated and turned to easier alternatives.

The core issue was understanding what's in the buffer. The method you provided is correct. However, please allow me to continue using my current approach—recalculating the relevant vertices each time. I understand there could be a better solution: finding the relationship between Attachment/triangle and buffer index (assuming the order is fixed, though unverified—I didn't spend more time investigating the code) and using the cached relationship along with the vertices buffer to retrieve data.

Regarding the mapping part, each triangle has a 3-point correspondence, which means it's an affine transformation. Barycentric coordinates are preserved under affine transformations. I didn't calculate the transformation matrix itself; instead, I calculated the transformed point directly, and the solution worked. I'm sure there are better solutions, but please allow me to move on and continue with the next feature.

I really appreciate your replies.

P.S. As for the shader issue, please forgive my recklessness.

aaazsd

https://esotericsoftware.com/forum/assets/files/2025-11-13/1763012222-246313-2025-11-12-22-59-22.mkv
I'm happy with the effect now. Though the strokes aren't perfect--you could tell from the clip.

Harald

@aaazsd Glad to hear you've figured it out! Thanks for getting back to us.