Kaohsiung International Airport: Take the MRT Red Line directly from the airport to Formosa Boulevard/Central Park or Kaohsiung Main Station: roughly 20 minutes to the core and about NT$35–50. It is faster, cheaper and less hassle than a taxi unless you are arriving late with a group and heavy luggage.
Zuoying HSR Station: Use the MRT Red Line southbound to Kaohsiung Main Station (R11), Formosa Boulevard or Central Park; allow about 15–25 minutes and NT$20–25. Do not assume the HSR terminal is downtown: it is on the northern edge of the urban core. A taxi is roughly NT$250–350 and makes sense only for a hotel poorly served by MRT.
Kaohsiung Main Station: This is the central TRA railway station and is already on the MRT Red Line. It is the sensible arrival for conventional TRA services, including the East Coast route; use the Red Line or walk onward rather than taking a taxi by default.
Taiwan Taoyuan International Airport: If your international flight is substantially cheaper or better timed, take the Airport MRT to THSR Taoyuan, then a southbound Taiwan High Speed Rail train to Zuoying; the rail leg is about 1½–2 hours, plus the airport-to-HSR connection and transfer into town. It is a workable national-gateway route, not a quick airport transfer; KHH is vastly easier for Kaohsiung.