We prove that all$f$-divergences between univariate Cauchy distributions are symmetric. Furthermore, those$f$-divergences can be calculated as strictly increasing scalar functions of the chi-square divergence. We report a criterion which allows one to expand$f$-divergences as converging series of power chi divergences, and exemplifies the technique for some$f$-divergences between Cauchy distributions. In contrast with the univariate case, we show that the$f$-divergences between multivariate Cauchy densities are in general asymmetric although symmetric when the Cauchy scale matrices coincide. Then we prove that the square roots of the Kullback-Leibler and Bhattacharyya divergences between univariate Cauchy distributions yield complete metric spaces. Finally, we show that the square root of the Kullback-Leibler divergence between univariate Cauchy distributions can be isometrically embedded into a Hilbert space.