Mobile devices which can capture and view pictures are becoming increasingly common in our life. The limitation of these small-form-factor devices makes the user experience of image browsing quite different from that on desktop PCs. In this paper, we first present a user study on how users interact with a mobile image browser with basic functions. We found that on small displays, users tend to use more zooming and scrolling actions in order to view interesting regions in detail. From this fact, we designed a new method to detect user interest maps and extract user attention objects from the image browsing log. This approach is more efficient than image-analysis based methods and can better represent users' actual interest. A smart image viewer was then developed based on user interest analysis. A second experiment was carried out to study how users behave with such a viewer. Experimental results demonstrate that the new smart features can improve the browsing efficiency and are a good compliment to traditional image browsers.