Recent expansions in multimedia devices for many applications, such as surveillance, self-driving cars, and healthcare, gather enormous amounts of real-time images for processing and inference. The images are first compressed using compression schemes, like joint photographics experts group (JPEG), before processing to reduce storage costs and additional power requirements for transmitting the captured data in this era of emerging ultra-wideband communication and human-body communication. The JPEG algorithm realizes image compression using simplistic matrix manipulations, making it preferable for hardware implementations. Furthermore, due to inherent error resilience and imperceptibility in images, JPEG can be approximated to reduce the required computation/processing power and area. This work demonstrates the first end-to-end approximation computing-based optimization of JPEG hardware using 1) an approximate division realized using bit-shift operators to reduce the complexity of the computationally intensive quantization block; 2) loop perforation; and 3) precision scaling on top of a multiplier-less fast discrete cosine transform (DCT) architecture to achieve an extremely energy-efficient JPEG compression unit which will be a perfect fit for power/bandwidth-limited scenario. Furthermore, a gradient descent-based heuristic composed of two conventional approximation strategies, i.e., precision scaling and loop perforation, is implemented for tuning the degree of approximation to tradeoff energy consumption with the quality degradation of the decoded image. The entire register-transfer level (RTL) design is coded in Verilog HDL, synthesized using the industry-standard tool, mapped to TSMC 65nm CMOS technology, and simulated using Cadence Spectre Simulator under 25 ° C, typical/typical (TT) corner. The approximate division approach in the quantization block achieved around 28% reduction in the active design area. The heuristic-based approximation technique combined with accelerator optimization achieves a significant energy reduction of 36% for a minimal image quality degradation of 2% sum of absolute difference (SAD). Simulation results also show that the proposed architecture consumes 15 uW at the DCT and quantization stages to compress a colored 480-p image at 6 frames/s.