Introduction
I was working on porting a historical site into Unreal Engine 4 to use in the upcoming game World Void, and I was in charge of scanning a castle. I was waiting for a funding that kept being delayed, until I decided to finally just go out and do it, with no funding, and I did it, I scanned the castle! A few month has passed and I am still pumped about it! This article will discuss how I managed to do it. But first, I would like to take a moment to thank Milos Lukac, Jeffrey Ian Wilson, and the 3D Scanning User Group, who without their helps none of this would have been possible. And finally a special thanks to Faisal Naamani for correcting some of my grammar mistakes.
What's photogrammetry?
In short, photogrammetry is the process of creating 3D models from multiple images of the same object taken from different angles.
I will not be covering how to 3D scan an object, it is a topic that has been already covered a lot.
If you want to get started with photogrammetry, I would recommend this short YouTube video by James Candy. Although this video mentions Agisoft Photoscan, the explained concepts should work with any other photogrammetry software solution out there. You can also check The poor man’s guide to photogrammetry
A lot of artists have been using photogrammetry as part of their workflow, it has been used in the Kite Demo from Epic games, and in Star Wars Battlefront, and many countless others. However most of these studios have been scanning rocks and trees. Epic games wrote a blog post about how they created the assets for the open world demo. They even wrote a blog about the choice of equipment, and another explaining their delighting process. I would highly recommend checking them out if you haven't already.
Challenges
When scanning a large object like a castle, there are new challenges to keep in mind:
Picture count: A castle is bigger than a rock, therefore unless you want a low res scan, you will need to take more photos. Which means that you will need a bigger SD card. Once you get a bigger SD card, then you will start facing low battery issues, and it's a good idea to have an extra battery. This might sound simple, but it wasn't until I started taking 2k+ images in a session that I started facing battery issues. In fact the first time I faced that issue was in the middle of scanning the castle, luckily the security there was kind enough to let me charge the battery at his booth. Once the images were taken, you will need to make sure you also have enough storage on your computer. HDD right now are really cheap.
Longer time: It make sense, when you are taking more pictures, you need more time. The castle took me 8 hours straight to scan, which means you need a constant overcast/cloudy day that last for 8 hours. It's also a good practice to take all the required photos of one element of the castle before switching to the next one. That way if the sun light shifts a little, and make the software unable to align all the photos, you can always manually align each element using control points.
Processing power: With more photos to analyze you're going to need a lot of processing powers. Required computer specs do vary between different 3d scanning software, and I will go over that in a second, but here is what you need to know:
RAM, and lots of it, 64GB or more is good idea, however some 3d scanning software uses a cache system which enables them to process a scan even with a low amount of ram, but I would not recommend going lower than 32GB.
GPU, the faster the better. You don't need a lot of VRAM, 4GB is enough. I would recommend an NVidia GTX card, because some software do not support AMD.
CPU, the more core you have the better.
Hard to get angles: When scanning a rock you have the advantage of being able to take an image from every angle, however with a huge castle there are places that are hard to reach. Using a drone, would be a good idea, however I did not have access to one, and I had to get by without one, which I will discuss later in this article.
Complex structure: Unlike rock a castle have more complex shapes, not to mention that rocks do not have interiors? This made it even more challenging, and I had to make sure I covered everything. How did I do that? By pre-planning, and using on-field verification.
Know your target (Preplanning)
I am working on the Historical site of Byblos, a city in Lebanon that has been constantly occupied for over 10000 years. However the Castle itself dates back to 12th century. It is a relatively small castle compared to newer ones found in some European countries, however each wall of this castle is made of highly detailed rocks crafted by time itself.
But what I meant by knowing you target is that you need to take a pen and paper, sketch a top and side view of the castle, and try to split it into multiple element, try to figure out the order in which you will be taking the pictures, and mark down areas that requires more images.
Also errors during the scan can accumulate, and can create some shifted geometry, therefore always make sure to take a full circle around your target which forces the scan to correct itself. Open geometry might not be as accurate.
I also took the opportunity and took multiple images of the surroundings from the castle’s ledges, which resulted in adding the landscape to our scan.
Tip, please do not take multiple images of different angles from the same spot (panoramic images). This will mostly result in undesired errors.
A useful way of scanning small tunnels where it is hard to take images of all the angles due to the tight space, is to walk in a straight line, and take an image each step while facing the forward direction. However this technique provides less resolution, which might be fine depending on how much resolution you need.
Finally, it is highly likely that you miss a few angles when scanning, causing low overlap, which will lead to low resolution areas in your scan, therefore I would highly recommend an on-field verification. Reality Capture for example is a 3D scanning software that can check the overlap of huge numbers of images on a laptop in less than a minutes, and it will let you know which areas need more images, which is extremely useful.
The white dots represent the locations from which the photos were took. The blue dots represent the manually added ground control point that helps aligning some elements that the software was unable to align automatically. Notice how the castle's roof has multiple ground control points that is normal, since I did not have access to a drone, I wasn’t able to take images enough of the roof. Luckily I was able to manually align it, which might not be the case for larger castles. The software aligned 4000 photo in total.
What do you need?
If you are trying to keep the budget as low as possible, this is what you will need:
Camera (duh), a DSLR is a must, however, new smartphones like the Galaxy S7 have some pretty good camera, however, unless you have been doing 3D scanning for a living, stick to a DSLR. I did not have a DSLR, so I borrowed one from my aunt. It was the Nikon d3200, and since I was scanning a castle made of rocks, I did not need a sharper camera.
Lens, the camera came with a variable 18-55mm lens, while a fixed lens is recommended, as long as you use the limit range (18 or 55) and you stick with it during the entire scan, you should be fine.
Tripod, yes you need one, especially when you are in some low light situations like the inside of the castle, and you can't use a flash, and you have to take long exposure images.
Monopod, it's useful, but if you don't have shaky hands and with the right shutter speed you might get away without one. However it only cost 10$, so why not?
SD Card, I used a 64GB SD card. I also had me laptop with me on field just in case the card was full, I can just empty it on the laptop's hard drive.
What I didn't use
Battery, I did not have one, but if you are going to take a lot of pictures, a secondary or even a third battery is recommended.
Drone, it would make your life a ton easier, and if it’s an option you should definitely consider it. I did not have access to a drone, so I had to climb, and take images from some tricky locations. Luckily it worked, but how much are you willing to push your luck, especially with bigger castles?
Chrome Ball and Grey Ball, this rig is used to get the perfect color accuracy and accurate delighting process. It is tempting to want t