The Workflows of Creating Game Ready Textures and Assets using Photogrammetry

A couple of month ago I wrote an article on how I was able to scan an entire castle for $70, and at the end I mentioned how I was porting that castle into the Unreal Engine to use in my game World Void. Working with a huge structure with millions of polygons can bring the most powerful software solution to its knees, and a lot of people have showed interest in the creative process that I am using to port the castle into a game engine. Making the castle game ready is a very tedious and long process, and before I can explain the work that goes into splitting the castle into game assets, I found it essential to first explain the challenges of manipulating 3D scans in general. Over the past few month I struggled to find a universal process that can be used on any mesh, in fact every mesh is different and requires a different remedy. Therefore in this article I will be going over multiple techniques that I stumbled upon, experimented with, and used in order to create the textures for the castle. I will be going over many solutions to delight a texture, and create a tile-able one.

Keep in mind that while 3D scans do make some assets creatiion tasks easier, it does come with its own challenges, and creating a proper texture from a 3D scan can sometimes take the same amount of time as to create a texture from scratch. The good news is however that this process becomes faster as you become more experienced with it.

A lot of research has gone through this article, and once again I would like to thank the 3D
Scanning User Group on Facebook, if it wasn’t for them I would never have learned about most of these tools and techniques. I would also like to thank Kenneth Brown from DICE, who was one of the speakers at GDC for a talk on photogrammetry and delighting techniques, who helped me when I got stuck trying to recreate the process he explained. Another thank you would go to Fedor Boytsov who suggested a few interesting links that were added to this article. Again a special thanks to Faisal Naamani for correcting some of my grammar mistakes. Finally, a quick shout out to all the people who read this article before publishing it, and provided me with some valuable feedback.

For these results it’s worth noting my computer specifications. I am using an Alienware laptop, if you are going to use a laptop for this intensive work, better make sure it’s a gaming laptop with a killer ventilation system. The laptop has 24GB of RAM installed, and when it comes to 3D scan you can never have enough RAM. A NVidia GTX 860M 2GB (Kepler), which is a good gaming GPU, but it’s not the highest end one which is sometimes not powerful enough for a 3D scan mesh, especially those 2GB of VRAM are definitely a bottleneck. And a 3.5GHz quad core i7.

Retopology and Mesh Editing Tools

Instant Meshes

This a very useful retopology tool that can generate good topologies for high poly meshes in a matter of seconds. You can download it for free from this website, which I also would encourage you to check out for other free cool utilities. On the download page you will find a quick video tutorial, and that is all you need to know to get up and running with this application.

The advantages of this tool are the following:

Here is an example of a 3D scan that I generated in Agisoft PhotoScan, notice how messy the wireframe is:

And here is a screenshot of the model in Instant Meshes after generating the new topology:

The most impressive thing is that the entire process of generating this topology took a few minutes in Instant Meshes, not to mention that I was also able to keep some details that normally would have been lost in conventional retopology tools:

However this solution is not perfect, and depending on how noisy the original mesh is, you might end up with a few erroneous polygons, but then again this issue is common with many software, and sometimes the best way to fix it would be through multiple iterations of retopology.

If you prefer another more manual retopology approach, you can also check out Sketch Retopo from the same site.

If you are already familiar with your own 3D modeling application, and don’t feel comfortable using 3^rd party application, I would recommend knowing your tools very well. Almost any 3D modeling application on the market has its own retopology tools built-in:

Autodesk Remake

This tool is made for 3D scans, it can open high poly scans, and I use it sometimes to select and remove some polygons from the mesh. I also use it to check the mesh for errors, like removing floating particles, and closing holes. Remake does offer some basic functionalities like extrude and bridge, but depending on the quality of your mesh, sometimes these functionalities will not work. Remake also offers retopology and Poly Cruncher functionalities in addition to texture baking options. This tool is not perfect and does crash sometimes, however it seems to do the job, and while the generated models are not always efficient, this tool is perfect for fast prototyping, and creating placeholder meshes. One thing to note, always have incremental saves, I used Remake to fix some Mesh issues, and it crashed during save, which corrupted the entire file, and I had to start over.

ZBrush

If you want to work with 3D Scans I would highly recommend learning ZBrush. ZBrush is an all in one solution, and if you learn it properly you’ll find that you will not be needing any other solution. The interface is not the greatest, especially if you do not own a touch screen, and the tools are not the most accurate, but they tend to deliver, not to mention that during my testing ZBrush was the only tool fully capable of handling millions of polygon. Moreover, ZBrush offers multiple tools for retopology like ZRemesher which I was never able to make it generate a mesh over 100k polygon (therefore you need to subdivide the mesh then reproject the details), Dynamesh which is great but will also close any hole in your mesh, and ZSphere the manual way of doing proper retopology work.

Highlight Removal

Like removing shadows from a texture, it is also possible to remove highlights. One way to do so would be to use a cross polarized photography rig, in fact the previous image was taken with such a rig, however if you did not use one when you captured your photos don’t worry there are other ways to get similar results. James Busby from Ten24 released a quick Nondestructive Image flattening and reflection removal video tutorial where he demonstrates a process to remove highlights from a texture which does not require any rig and can give decent results. In short this technique consist of selecting darker areas in a picture and then bumping their exposure a little. While he used this technique on a head shot, I found this technique to work well on rocks, especially the ones that have an overall similar coloration.

Delighting a Texture

In order to remove shadows from a texture there is a very accurate way to do so that requires a special rig.
Epic Games used this technique in order to create the assets for their open world demo. They even wrote two blog posts titled “Imperfection for Perfection” where they explained their choice of equipment, and their delighting process. In short this technique requires you to capture a panoramic HDR image of the environment around the scan, then by projecting this image on a sphere in your rendering application, and properly orienting it to your scan, you can replicate the lighting setup, which enables you to subtract the shadows.

However if you don’t have access to such a rig, another interesting method for delighting would be the one that was used in Star Wars Battlefront. The team behind the Star Wars game was kind enough to explain its workflow during GDC in a talk entitled “Photogrammetry-and-Star-Wars-Battlefront” which is available for free on the GDC Vault, and I would highly recommend you check it out, really go watch it. The talk contained a ton of useful information, however I will be focusing on their delighting process.

Step 1: Generate 3D scan

There are a lot of tutorials on how to create a 3D model in Agisoft, therefore I will only be going over it swiftly.