API

Mast3r inference

starster.Mast3rModel

alias of AsymmetricMASt3R

class starster.Scene(cache_dir: str | None = None, device='cuda')

Starst3r scene. Contains Mast3r and 3DGS reconstructions, and helper methods.

add_images(model, imgs: list[Tensor], conf_thres=1.5)

Add GT images to the scene. Solve camera pose and update dense points with Mast3r.

Parameters

model:

Model instance. starst3r.Mast3rModel.

imgs:

New GT images to add. Each img is shape (H, W, 3).

conf_thres:

Confidence threshold for Mast3r dense points.

c2w: Tensor

Camera-to-world matrix, shape (C, 4, 4).

dense_cols: list[Tensor]

Dense point colors from Mast3r reconstruction. From each camera view.

property dense_cols_flat

Dense colors concatenated from all cameras.

dense_pts: list[Tensor]

Dense point 3D coords from Mast3r reconstruction. From each camera view.

property dense_pts_flat

Dense points concatenated from all cameras.

imgs: list[Tensor]

GT images after scaling via Mast3r code.

intrinsics: Tensor

Camera intrinsic matrices, shape (C, 3, 3).

optim_params: dict[str, Any]

SLAM parameters passed to Mast3r scene optimization.

raw_imgs: list[Tensor]

GT images, unscaled. Each img is shape (H, W, 3).

property w2c: Tensor

World-to-camera transformation matrix (inverse of c2w).

3DGS refinement

starster.gs.init_3dgs(scene, init_scale=0.003, lr=0.001)

Initialize 3DGS splats and optims from Mast3r dense points.

starster.gs.render_3dgs(scene, w2c: Tensor, intrinsics: Tensor, width: int, height: int)

Render the splats from a set of camera views.

TODO currently width and height can only be the original image size.

Parameters

w2c:

World-to-camera matrices. Shape (N, 4, 4).

intrinsics:

Camera intrinsics. Shape (N, 3, 3).

width:

Image width.

height:

Image height.

Returns

Output of gsplat.rasterization.

Tuple of (render_img, render_alpha, info).

• render_img: Color image. Shape (N, H, W, 3).

starster.gs.render_3dgs_original(scene, width: int, height: int)

Render from camera views of original scene (scene.scene).

See render_views.

starster.gs.run_3dgs_optim(scene, iters: int, enable_pruning: bool = False, loss_ssim_fac=0.2, loss_opacity_fac=0.01, loss_scale_fac=0.01, verbose: bool = False) → list[float]

Run 3DGS optimization and pruning (optional) for a number of iterations.

Parameters

iters:

Number of iterations.

enable_pruning:

Enable pruning and densification via the gsplat pruning strategy.

verbose:

Enable tqdm progress bar.

Returns

List of losses at each iteration.

Image

starster.load_image(path: str | Path, size: int = 224) → Tensor

Load and process image from file.

Parameters

path:

Path to image.

size:

Resize longest edge to this.

Returns

Tensor. See process_image().

starster.load_images(paths: list[str | Path], size: int = 224) → list[Tensor]

Load a list of files.

Calls load_image() on each path.

starster.process_image(img: ndarray | Tensor, size: int) → Tensor

Preprocess image to pipeline requirements.

• Resizes longest edge of image to size.

• Crops (around center) H and W to a multiple of 8.

Parameters

img:

Shape (C,H,W), dtype uint8

Returns

Tensor, shape (C,H,W), dtype float32

Utils

starster.interp_se3(mat1: Tensor, mat2: Tensor, fac: float) → Tensor

Interpolate between two SE3 matrices.

Linear interpolation of SO3 and translation components. Normalizes and orthogonalizes SO3.

Parameters

mat1:

SE3 matrix 1, shape (4,4).

mat2:

SE3 matrix 2, shape (4,4).

fac:

Interpolation factor, 0 to 1.

Returns

SE3 matrix, shape (4,4).

starster.interp_se3_path(mat1: Tensor, mat2: Tensor, steps: int) → Tensor

Linear interp between two SE3 matrices with linearly increasing factor.

Parameters

mat1:

SE3 matrix 1, shape (4,4).

mat2:

SE3 matrix 2, shape (4,4).

steps:

Number of interpolation steps.

Returns

List of SE3 matrices, shape (steps, 4, 4).