Fundamentals of Cupix Technology

Cupix creates 3D virtual tours from 360 camera photos or videos. When a set of 360 degree photos is uploaded to the Cupix online app, the software uses the images to determine the location and orientation of the camera in each picture. For 360 degree videos, the motion path of the video recorder is used instead. A 3D model of the site is automatically generated by analyzing the uploaded images. This is possible due to a combination of two cutting-edge technologies - photogrammetry and 3D reconstruction.

The principle of photogrammetry involves detecting common features between photos. A feature is a cluster of pixels from a small, unique portion of the image (ie: corners, furniture, paintings), and there are typically tens of thousands of these pixels between two photos. The camera location and tracking information is determined by repeatedly solving the problem of “What is the position of the camera in each picture to make common features between these two images appear as photographed?” This makes the core photogrammetry algorithm a highly complicated optimization problem. During image processing, millions of unordered and noisy 3D points are generated as well.

The second part of the technology, 3D reconstruction, deals with how to create a clean, textured 3D mesh model using this collection of 3D points, also called a point cloud, which is generated during photogrammetry. Therefore, there are a few guidelines to keep in mind as you take 360 photos to upload to our server.

Common Features are Important

If there are an insufficient number of similar pixels or features among uploaded photos, our software will not be able to determine the camera location or create a 3D model. This results in NA or ANC cases. NA, or Not-Aligned, is when the location of a 360 photo cannot be computed. ANC, or Aligned-but-Not-Connected, is when 360 photos are aligned as a group but are not connected to the entire set.

Listed below are some reasons that can lead to an insufficient number of common features in photos, as well as tips to avoid these situations.

Not enough photo overlap

The most frequent reason for a lack of common features and, consequently, the failure of the process is when there is not enough overlap between photos.

  • Read the section How many photos are required? below for more details.

No common features within the space

This is the most challenging case, and a typical example is when taking photos of single-colored walls with no decoration.

  • Add features by placing posters or other colorful items in the area to be photographed.

The line of sight is blocked

Areas where walls are close together frequently block useful features needed for spatial recognition or positioning from the camera’s view. This often occurs at doorways, entrances, built-in room dividers, and narrow spaces like hallways and small rooms.

  • For doorways and entrances, take an additional photo on the threshold of a door as well as one step before and after the door.
  • Take more photos in small spaces like narrow hallways.

Low lighting

  • Take photos during the day, or turn on as many lights as possible.
  • Bring portable lights to dark sites, like a basement under construction.

Blurred images

Blurred images can make it difficult for the software to identify common features. Here are some possible causes for blurred images, as well as tips to fix them.

  • If using a tripod or monopod, make sure the connection between the camera and tripod is secure.
  • For helmet-mounted and handheld photos, stand still while taking photos.
  • When recording videos, walk at a pace slower than your average walking speed. Avoid any shaking and sudden movements.
  • Keep the camera lens clean.

Best Practices When Taking Photos

How many photos are required?

If you need to estimate the number of photos to take, use the following guidelines:

  • For open spaces, one 360 photo for each 100-500 square feet (10-50 square meters).
  • For a house with rooms and hallways, one 360 photo for each 30-50 square feet (3-5 square meters).

For example, for a 1,000 square feet, 2 bed/2 bath house, approximately 20 to 40 photos are needed.

In practice, the number of needed photos may vary with the structure of the site, and so a better question to ask first would be where to take photos rather than how many photos to take.

Here are rules to follow when taking photos in a house.

  • Take one photo under every doorway.
  • Space photos by one step before and after doorways.
  • Space photos by two steps in hallways.
  • Space photos by three steps in rooms.
  • Do not take photos too close to the walls or furniture.
  • Take more photos around corners.

Do more photos make the 3D mesh cleaner?

The quality of the 3D model improves with an increasing number of photos. This is because each image provides an additional point cloud, which is used to accurately create the 3D mesh needed to generate the model. However, because each additional image requires exponentially more time to process, you will need to find the right balance between mesh quality and resources you can spend on your application.

How to hold the camera

  • Monopod or Tripod: Adjust the tripod to a height of 4 to 5 ft. Maintain a constant height for a seamless transition.
  • Selfie Stick: Hold the selfie stick directly over your head, keeping your body out of the shot. Move as little as possible while taking photos to decrease blurriness in photos.
  • Helmet-mount or handheld: Keep your body out of the shot and stay still while taking photos. If holding the camera, do not block the lens with your fingers, hands, or head.
  • Telescoping rod: When you need to take semi-aerial photos or videos while driving a car, we recommend using a telescoping rod at least 10 ft long to avoid showing the car in the bottom half of the photo.

Note: Tilting along the vertical axis of the photo is automatically compensated during processing, so the camera does not need to be held exactly upright.

Interval photos generate faster and better results

Interval Photos are a series of photos taken sequentially after a certain distance or amount of time. Our software assumes each photo location and proximity are indicated by each photo’s capture time and file name number, which can make processing much faster and more reliable. Therefore, interval photos taken by a handheld or helmet-mounted camera are an excellent choice when you need to capture a large site quickly.

To take full advantage of this option, use one of the following three methods:

  1. To manually take interval photos, stop, stand, and take a photo every 3 to 6 steps in open spaces. As you continue walking, follow linear paths as if you were mopping the floor. Take more steps in open spaces and fewer steps around doorways.

  2. Use the time-lapse video option if the camera supports it. This is the best method to use if a quick site capture is needed and crystal clear images are not required. The camera is set to a low frame rate of around 2 FPS or lower for the video file. Our software automatically determines that the video is in time-lapse mode and processes the file as interval photos instead. Unlike the first method, you do not need to stop each time a photo is taken. Instead, you can continue moving at a constant speed slower than the average walking speed.

  3. You can use the interval shooting option if the camera supports it. With interval shooting, the camera automatically takes a photo at a set interval of time. Practice this technique to familiarize yourself with the optimal walking pattern to avoid taking blurry images while you are in motion.

GPS data helps process an outdoor scene faster and more accurately

Most 360 cameras support GPS tracking, and cameras can record GPS information in the metadata of the photo file. If you turn on the GPS option during shooting, our software is able to use the GPS data in several ways when you upload the 360 photos:

  • Our software uses the GPS data to estimate the initial position of the camera, which makes 3D reconstruction of the site faster and more stable.

  • Our software also uses the GPS data as backup data. If our software is unable to align a photo correctly, the image will be inserted in the resulting 3D tour by using the initial position instead of creating NA or ANC sub-sections.

  • The Keyhole Markup Language (KML) file can be downloaded in addition to the PLY and OBJ files. The KML file contains the geolocations of both the GPS data and our processing results.

Tips:Make sure the GPS signal is strong enough if you want to use this option. Never use this option for indoor spaces. If you are taking outdoor photos, keep camera intervals longer than the GPS accuracy range (typically 5-30 feet). If there are tall buildings around the site, the GPS data may not be accurate enough to be used for photo alignment.


Videos

You can efficiently capture a site by recording a 360 video instead of taking multiple 360 photos. Our software automatically selects photo frames from the video clip that can effectively represent the space. The walking path is regenerated and displayed in the Cupix Web App, allowing you to quickly align the 3D tour with a floor plan or 3D BIM model. However, you should be aware that the photo quality will not be as clear as still photos, and the file size of a video can be very large.

Tips to keep in mind when recording a video

  • Keep your head up and walk a little slower than the average walking speed.
  • Do not stop walking while recording a video.
  • Keep the camera steady.
  • Slow down near doorways and in narrow hallways.