Creating a Metahuman Head from a Midjourney Images

We took on an exciting challenge: creating a #Metahuman head in Unreal Engine 5 using AI-generated facial images from #Midjourney . This unique project combined AI-generated art with real-time 3D modeling, pushing the boundaries of both fields.

First, I got one photo of the head from the front. I decided to make this head in Blender using the purchased Face Builder plugin. Accordingly, we needed additional high-quality photos - a front view, a side view from the left and right sides, and also from the back. 

Using Face Builder, I uploaded the head photos into the plugin. Next, we select the desired view (initially the front view), and add reference points to the base head mesh that appeared as soon as we started working with the plugin.

By moving the points while looking at the photo, we replicate the silhouette of the head's shape. We do the same from other angles, periodically checking the model for visual accuracy. It’s helpful if you have artistic talent — it can really come in handy here.

In the end, using all the angles from the generated photos — front view, left and right side views, as well as the back view — I initially ended up with a distorted head with significant shape inaccuracies. This is because when generating the secondary photos using the main one (front view), the AI creates other angles with distortions: sometimes it pulls the nose forward, sometimes it changes the shape of the eyes. These distortions are subtle, but if you look closely and start comparing, you’ll notice many inaccuracies.

If we had used photos of a real person, these distortions would have been eliminated, as the shape of a person’s head and face remains consistent from any angle. Therefore, I had to repeat the process, but this time I kept only two main angles: one front view and one side view. I used the back view as a reference for creating the texture later on. Following this approach, I ended up with an accurate head shape mesh.

Next, I baked the textures for the Metahuman head. It's important to note that the more photo angles you use, the smoother the overall texture will be.

Next, we transfer the head mesh into the engine, saving it as an FBX file. We open it in the engine as an FBX file, set as a Static Mesh. Using the Metahuman Identity tool, we load the generated mesh. In the selected view, we click "Track Markers" to detect control points for the mouth, eyes, and eyebrows. We then do the same for the side view, or you can get by with just the front view. After that, click "Metahuman Identity Solve." Once the engine processes it, we get the skeletal mesh of the head, which we then attach to the Metahuman body and upload it into the Metahuman browser editor.

Open the editor in the browser. Adjust the head as much as possible to match the original front view; an artistic talent can really come in handy here.

The head textures can be made roughly similar since we will be editing them further in Photoshop. Once done, we upload the generated Metahuman back into the Unreal Engine.

Next, we move to Photoshop to edit our texture by overlaying the Metahuman texture onto the texture from Blender. Don't forget about the fold textures — there are three of them, and they also need to be prepared for facial animation if you plan to use it later on.

Then, in the engine, we apply the finished textures to the Metahuman. And our head is ready.

I fully understand that this isn’t 100%, but I believe it’s a pretty good result — around 85-95% similarity. If it were a real person, the result would be better. Another drawback is that the Metahuman browser editor doesn’t allow for more precise and detailed sculpting of the head. All the sliders move according to predefined coordinates.

The Ai-human's name is T and he lives in social nets : https://x.com/agentic_t

Don't forget to subscribe to our socials:

Facebook

x.com

YouTube