Fundamentals of Cupix Technology
When you upload a set of 360 photos or a 360 video, Cupix software automatically figures out camera locations (x, y and z coordinates) and orientations (rotational angles around the axis) of 360 photos, or the motion path in case of a 360 video, and creates the 3D model of the scene just by analyzing images. The magic under the hood is two cutting-edge technologies - the photogrammetry and the 3D reconstruction.
The main idea of the photogrammetry is to detect common features (a feature is a small portion of the image) among photos, typically tens of thousands of them between only two photos, and track the camera pose information by solving the problem of “What is the best position of each camera to make the common features appear in the images that way?” So, the core algorithm is a highly complicated optimization problem. The second part of the technology, 3D reconstruction, deals with how to create a clean (not messy) textured 3D mesh model from millions of unordered noisy 3D points, a.k.a point cloud, which tends to be generated from the photogrammetry processing.
Therefore, as you may be able to imagine, there are a few things to keep in mind as you take 360 photos and upload them to our server.
Common features are important
For obvious reasons, if there are insufficient similar pixels among photos, our software can hardly find the camera location, let alone the 3D model, and end up with NA or ANC cases. NA, or Not-Aligned, is the case when the location of a 360 photo is not able to be computed. ANC, or Aligned-but-Not-Connected, is the case when 360 photos are aligned as a group but not connected to the entire set.
Here are several reasons when common features become scarce and quick tips to avoid the situations.
Not enough photo overlap
The most general reason for the lack of common features and, consequently, the failure of the process is there are not enough photo overlaps. Please continue to read the How many photos are required? section for more details.
Actually no common features in the scene
- This is the most challenging case, and a typical example is single-colored walls with no deco.
- Add some features by placing posters or other colorful items in the scene.
The line of sight is blocked
- Take an additional photo on the threshold a door as well as one step before and after the door.
- Take more photos on narrow hallways.
Low lighting condition
- Take photos at daytime or turn on lights as much as possible.
- Bring portable lights when the site is really dark like a basement under construction.
- Make sure the camera and the tripod are secure.
- Stand still when you take photos handheld.
- In case of a video, walk a little slower than the average walking speed.
- Keep camera lens clean.
Best practices when taking photos
How many photos are required?
If you need just a ballpark estimate of the number of photos to take, please use these simple rules of thumb:
- One 360 photo for each 10~50 sqm (100~500 sqft) for open space.
- One 360 photo for each 3~5 sqm (30~50 sqft) for a house with rooms and hallways.
For instance, 20~40 photos are required to cover a 100 sqm (1,000 sqft), 2br/2ba house.
However, in practice, the number of photos may vary with the structure of the site, and so a better question would be where to take photos rather than how many photos to take.
Here are tips you need to follow when you take photos from a house.
- Take one photo under every doorway.
- Space photos by one step near doorways.
- Space photos by two steps in hallways.
- Space photos by three steps in rooms.
- Don’t take photos too close to the walls or furniture.
Do more photos make the 3D mesh cleaner?
Yes, you can expect the better quality 3D model if more photos are taken. However, because it would take exponentially more time to process them that you need to take the right balance between the mesh quality and resources you can spend.
How to hold the camera?
- Tripod or Monopod: Adjust the tripod to a height 4 to 5 ft. Maintain a constant height for a seamless transition.
- Selfie Stick: Place it overhead, keeping body out of the shot. Try not to move while capturing to decrease blurriness in photos.
- Helmet-mount or handheld: Make sure you keep your body out of the shot and stay still while taking photos.
- Telescoping rod: When you need to take semi-aerial photos or videos while driving a car, we suggest that you should use a 10 feet or taller telescoping rod to avoid the car to be shown in the bottom half of the photo.
The vertical angle of the photo is automatically compensated in the course of processing, and so you don’t need to make sure the camera is held strictly in the upright direction.
Interval Photos option generates the result faster and better
Interval Photos is a series of photos taken sequentially at every few steps. Our software assumes photos’ creation time or names are in the order of photo closeness and it can make the process much faster and reliable. Therefore, Interval Photos by a handheld or helmet mounted camera is the excellent choice when you need to cover a large site quickly.
To take full use of this option, you may need to take one of thee methods.
Stop, stand and take a photo at every 3~6 step as you keep walking following linear paths as if you were mopping the floor. Take more steps in opening areas and fewer steps around doorways.
Use the Time-lapse video option if the camera supports it. This is the best method in terms of ease of site capture if you don’t mind the less clear image quality. The camera creates a low frame rate, like 2 FPS or fewer, video file and our software detects the time-lapse mode automatically and process it as the Interval Photos. Unlike method 1, you don’t need to stop and take a shot, instead, you can keep walking at a constant speed, preferably at little slower than the average walking speed.
You can use the Interval Shooting option if the camera supports it. It does automatic shutter release at a set interval time like 4 or 8 seconds. You need to practice this technique to get used to the optimal walking speed because you can easily get blurry images when the camera takes shots while you are in motion.
GPS data helps process an outdoor scene much faster and better
Most 360 cameras support the GPS option, and so the camera can record GPS information in the metadata part of the photo file. Our software leverages the GPS info if you switch the option on when you upload 360 photos in several ways.
It uses the GPS data as the initial pose guess and makes the process much faster and more stable.
It uses the GPS data as the failover data as well, and even when our software fails to align a photo, it put it in the result 3D tour using the initial pose instead of creating NA or ANC sub-sections.
You can download the KML (Keyhole Markup Language) file in addition to the PLY and OBJ file, in which you can find the geolocations of both GPS data and our result.
Make sure the GPS signal is strong enough when you want to take advantage of this option. Never use this option for any indoor scene. Even for the outdoor case, keep camera intervals longer than the GPS accuracy range (typically 5~30 feet) Plus, if there are tall buildings around the site, GPS data may not be accurate enough to be used for the alignment.
There are dozens of off-the-shelf consumer grade 360 cameras in the market, and we would say that technically they are all compatible with Cupix software. However, a few of them perform better than others based on our experiments, and our top choices are Ricoh Theta, Insta360, Mi Sphere 360, Yi 360 cameras, and GoPro Fusion (not in the order of preference) at the time of writing.
One of the most critical manufacturer-specific issues from our software processing perspective is the amount of distortion introduced in the course of stitching images from multi-lens sensors, followed by the noise level of images.
Stitching Software Download Links
If you use a DLSR camera with a wide-angle lens and a panoramic tripod head, that would be perfectly fine for us to process correctly as long as photos are equirectangular which means the width to height aspect ratio is exactly 2:1.
You can capture the site very quickly and easily by taking a 360 video instead of many 360 photos. Our software automatically selects salient photo frames out of the video clip which can represent the scene the most effectively. Plus, the path of the movement is regenerated and displayed in the Web App which helps you align the 3D tour with a floor plan or 3D BIM model quickly. However, you should be aware that the quality of photos are not as great as still photos and the file size of a video can be very large.
Tips to keep in mind when you take a video.
- Keep your head up and walk at a little slower than the average walking speed.
- Try not to stop walking while recording a video.
- Try to keep the camera steady.
- Slow down near doorways and narrow hallways.