The translational and rotational dynamics of anisotropic optical nanoprobes revealed in single particle tracking (SPT) experiments offer molecular-level information about cellular activities. Here, we report an automated high-speed multidimensional SPT system integrated with a deep learning algorithm for tracking the 3D orientation of anisotropic gold nanoparticle probes in living cells with high localization precision (<10 nm) and temporal resolution (0.9 ms), overcoming the limitations of rotational tracking under low signal-to-noise ratio (S/N) conditions. This method can resolve the azimuth (0°–360°) and polar angles (0°–90°) with errors of less than 2° on the experimental and simulated data under S/N of ∼4. Even when the S/N approaches the limit of 1, this method still maintains better robustness and noise resistance than the conventional pattern matching methods. The usefulness of this multidimensional SPT system has been demonstrated with a study of the motions of cargos transported along the microtubules within living cells.