A united-reference fractional counter (URFC) is proposed for multi-level time-to-digital converters (TDCs) used in direct time-of-flight (dToF) light detection and ranging (LiDAR) applications. The URFC combines the integer counter and the second-level interpolation in one functional block that derives both the integer part and the fractional part from the same synchronization result. Thus the proposed TDC has a more robust structure to deal with misalignment errors, and measurement results from different levels can be perfectly matched. High linearity and high precision can be guaranteed because of the united reference without calibration. The response time of the URFC decreases to hundreds of picoseconds, which makes it suitable for all output pulses from the avalanche photodiode. A TDC with the proposed URFC was implemented in 65nm standard CMOS technology with a detection range of$2.5~\mu \text{s}$($\sim 375\text{m}$) and a single shot precision of 27.63ps (~4.0mm). The power consumption is 51.4mW with a reference clock at 200MHz and a repetition rate at 999kHz. The INL and DNL are ±2.751 LSB and ±1.679 LSB, respectively.