The randomized NCRN phase III ABC-02 trial provided level-A evidence for first-line chemotherapy with cisplatin and gemcitabine combination in advanced biliary cancer (ABC). This systematic literature review aims to evaluate the level of evidence for the use of second-line chemotherapy for patients with ABC in terms of overall survival (OS), response, toxicity and quality of life. Eligible studies were identified using Medline, ASCO, ESMO and the World Gastrointestinal Congress databases. Searches were last updated on 15 December 2013. Eligible studies reported survival and/or response data for patients with ABC receiving second-line systemic chemotherapy. This systematic review was registered in the PROSPERO database (No. CRD42013004205). Five hundred and fifty-eight studies were identified from the searches in Medline (n = 342), ASCO (n = 160), ESMO (n = 27) and World Gastrointestinal Congress (n = 29). Twenty-five studies were eligible: 14 phase II clinical trials, 9 retrospective analyses and 2 case reports. In total, data from 761 patients were reported with median number of patients included in each study of 22 (range 9–96). The mean OS was 7.2 months [95% confidence interval (CI) 6.2–8.2] [phase II: 6.6 (95% CI 5.1–8.1); retrospective analysis: 7.7 (95% CI 6.5–8.9)]. The mean progression-free survival (PFS), response rate (RR) and disease control rate were 3.2 months (95% CI 2.7–3.7), 7.7% (95% CI 4.6–10.9) and 49.5% (95% CI 41.4–57.7), respectively. The best correlations were between OS and PFS for all studies (r = 0.54; P = 0.01) and between OS and PFS (r = 0.61; P = 0.04) and OS and RR (r = 0.62; P = 0.03) for phase II studies, respectively. Biliary tract cancer is known to be a chemo-responsive disease. There is insufficient evidence (level C) to recommend a second-line chemotherapy schedule in ABC, although the available data suggest that a cohort of patients may benefit. Further prospective and randomized studies are needed to clarify the relative value of second-line chemotherapy in this setting.