About align cameras using first view

[1843744321mark]

It's a really great job. I am not sure why we need align cameras using first view, and cannot understand these formula. Could you please tell me a more detailed explaination about it? Thank you very much!
r = np.linalg.norm(tar_c2ws[0,:3,3])
ref_c2w = np.eye(4, dtype=np.float32).reshape(1,4,4)
ref_w2c = np.eye(4, dtype=np.float32).reshape(1,4,4)
ref_c2w[:,2,3], ref_w2c[:,2,3] = -r, r
transform_mats = ref_c2w @ tar_w2cs[:1]
tar_w2cs = tar_w2cs.copy() @ tar_c2ws[:1] @ ref_w2c
tar_c2ws = transform_mats @ tar_c2ws.copy()

[apchenstu]

Thanks! I thought it could provide better generalizability, but I didn't make an ablation on this. Now I feel it may provide better results if disabling this alignment, feel free to try ;)

[1843744321mark]

Thanks for your kindly reply! I meet another problem and hope for your reply. Why should we use permute(0,2,1) for w2c and camera Intrinsics?
def projection(grid, w2cs, ixts):
points = grid.reshape(1,-1,3) @ w2cs[:,:3,:3].permute(0,2,1) + w2cs[:,:3,3][:,None]
points = points @ ixts.permute(0,2,1)
points_xy = points[...,:2]/points[...,-1:]
return points_xy, points[...,-1:]

[apchenstu]

Assuming the points array (excluding the batch dimension) has a shape of [N x 3] and ixts is [3 x 3], the projection function ixts @ points.T can be transposed to points @ ixts.T. The same operation can be applied to the extrinsic transformation, eliminating the need to transpose the points—simply transpose the extrinsic or intrinsic matrix instead.