Rotation search and point cloud registration are two fundamental problems in robotics and computer vision, which aim to estimate the rotation and the transformation between the 3D vector sets and point clouds, respectively. Due to the presence of outliers, probably in very large numbers, among the putative vector or point correspondences in real-world applications, robust estimation is of great importance. In this paper, we present ICOS (Inlier searching using COmpatible Structures), a novel, efficient and highly robust solver for both the correspondence-based rotation search and point cloud registration problems. Specifically, we (i) propose and construct a series of compatible structures for the two problems where various invariants can be established, and (ii) design three time-efficient frameworks, the first for rotation search, the second for known-scale registration and the third for unknown-scale registration, to filter out outliers and seek inliers from the invariant-constrained random sampling based on the compatible structures proposed. In this manner, even with extreme outlier ratios, inliers can be sifted out and collected for solving the optimal rotation and transformation effectively, leading to our robust solver ICOS. Through plentiful experiments over standard datasets, we demonstrate that: (i) our solver ICOS is fast, accurate, robust against over 95% outliers with nearly 100% recall ratio of inliers for rotation search and both known-scale and unknown-scale registration, outperforming other state-of-the-art methods, and (ii) ICOS is practical for use in multiple real-world applications.