Previously powered by Photogrammetry, Neural Surface Reconstruction, and LiDAR, KIRI Engine now welcomes the new tech joining the application - 3D Gaussian Splatting!
What is 3D Gaussian Splatting (3DGS)?
3D Gaussian Splatting is a rendering technique that was created in the scientific field during the 90s. Its application to the visualization of scenes in real-time was presented last August at SIGGRAPH 2023, and it sounds promising and revolutionary.
Gaussian Splatting is a rasterization technique for real-time 3D reconstruction and rendering of images taken from multiple points of view easily by using your phone to record.
A set of Gaussians actually defines the 3D space, and machine learning calculates each Gaussian's parameters. This means data describing the scene and drawing them on the screen.
It is similar to triangle rasterization in computer graphics, but instead of triangles, they are Gaussian, and the following parameters describe this technology:
Position: where it is located (XYZ).
Covariance: how it stretches/scales (3×3 matrix).
Color: RGB.
Alpha: The transparency.
How does Gaussian Splatting work?
3DGS represents a 3D scene as tons of particles/Gaussians, and each 3D Gaussian comes with a position, orientation, scale, opacity, and color.
The particles are converted into 2D space in order to render them efficiently. The steps go like this:
Structure from Motion (SfM): It starts using this method to create a point cloud from images. This can be done using COLMAP, which is a general-purpose Structure-from-Motion (SfM) and Multi-View Stereo (MVS) pipeline with a graphical and command-line interface. It offers a wide range of features for the reconstruction of ordered and unordered image collections. The software is licensed under the new BSD license.
Convert to Gaussian: each point is converted to Gaussian, making rasterization possible. Nevertheless, only its position and color can be inferred from the SfM data.
Training: For high-quality results, it must be trained. Stochastic Gradient Descent, similar to a neural network, is used for this purpose.
1. Rasterise Gaussians in an image that uses differentiable Gaussian rasterization.
2. Calculate the loss as a function of the difference between the raster and real terrain images.
3. Adjust the Gaussian parameters according to the loss.
4. Differentiable Gaussian rasterization where each 2D Gaussian must be projected from the camera perspective, sorted by the depth, and, for each pixel, each Gaussian must be repeated and combined backward and forward.
In this informative video below, you can explore the KIRI Engine's capabilities using 3D Gaussian Splatting technology:
Download KIRI Engine now to have a glimpse of the future of 3D:
KIRI Engine on Android & KIRI Engine on iOS.
KIRI Innovations, the company behind KIRI Engine, was founded by three friends, Jack, Chris, and Peter in 2018. During their engineering studies in Hamilton, Canada, the team was determined to develop a 3D scanning solution that would be affordable to all.
