We propose a method for joint estimation of a host vehicle state and a map of the road based on global navigation satellite system (GNSS) and camera measurements. We model the road using a spline representation described by a parameter vector having a Gaussian prior representing the uncertainty of the prior map. Both GNSS and camera measurements, such as lane-mark measurements, have noise characteristics that vary in time. To adapt to the changing noise levels and hence improve positioning performance, we combine the sensor information in an interacting multiple-model (IMM) setting to choose the best combination of the estimators with the vehicle state and the parameter vector of the map as the state vector. In a simulation study, we compare vehicle models with varying complexity, and on a real road segment we show that the proposed method can accurately adjust to changing noise conditions and correct for errors in the prior map.