In the Ming dynasty, Penghu was a transshipment station for exporting ceramics to Southeast Asia. However, the Dutch East India Company in Jakarta bypassed Penghu. Instead, the Dutch established an Asian trading base in Taiwan, eclipsing Penghu. During the Japanese occupation, to control trade in Taiwan's ports, only the four major harbors––Keelung, Danshui, Anping, and Kaohsiung––were opened, negatively impacting Taiwan residents' supply of daily necessities. Ultimately, eight more "special import and export ports" had to be opened, including Magong Port in Penghu. In 1906, the Magong Office of Anping Customs was completed in Tsukijimachi, outside the Magong Yingxun Gate. The two-story brick "watchtower" structure was Penghu’s first official Western-style building. In 1934, it was restructured as a Kaohsiung Customs affiliate, officially designated "Magong Office, Kaohsiung,” but locally known as Sue-koan, or "Tax Customs" (Taiwanese).

Another historical building worth visiting in the urban area is the Penghu Hospital. The old vestibule of today’s Penghu Hospital was requisitioned during the Japanese occupation and converted into a two-story wooden building, Penghu’s first Western-style hospital. In 1940, a cavern-like underground air-raid shelter was constructed and fitted with sound-transmission holes, ventilation holes, hospital beds, and notice boards, serving military defense and medical purposes. One of Taiwan's few remaining historic medical buildings, Penghu Hospital has witnessed the nation's medical history from the Japanese occupation to the present day. Penghu residents call it “the grand hospital"––it's a living record of the people’s collective memories.