Abstract: In order to reduce the stringent volume and cost requirements for underwater unmanned autonomous vehicles (AUVs) when equipped with homing systems, this paper proposes a simple and feasible passive homing method. The proposed method can use only one sound pressure hydrophone to guide AUVs to approach the narrow-band sound source target with a known center frequency. The change in Doppler frequency shift caused by the relative motion between the AUV and target is used to control tracking behaviors. Based on the reinforcement learning Actor-Critic algorithm, the value function is constructed using the center frequency and signal-to-noise ratio parameters of the received signal obtained in real time. The target detection results, the target azimuth angle and the center frequency change rate parameters generated by the value function are utilized to construct the strategy function. The strategy function can combine historical actions to output the current optimal action. The training efficiency and the target tracking performance of the proposed method are studied through simulation experiments and lake trials. Results show that our passive homing method can be easily applied to underwater unmanned platforms to guide AUVs to more accurately track sound source targets with different speeds.