Purpose: The purpose of this study was to evaluate the clinical benefits of nivolumab with/without ipilimumab combined with stereotactic body radiotherapy (SBRT) in patients with pretreated metastatic biliary tract cancer (mBTC). Patients and Methods: The study was a phase 2 randomized trial with Simon’s optimal two-stage design requiring 36 evaluable patients per group after second stage. Sixty-one patients were included from September 2018 to January 2022 and randomized (1:1) to receive SBRT (15 Gy × 1 on day 1 to a primary or metastatic lesion) and nivolumab (3 mg/kg intravenously on day 1 and every 2 weeks) with/without ipilimumab (1 mg/kg intravenously on day 1 and every 6 weeks). Primary endpoint was clinical benefit rate (CBR), defined as the percentage of patients with complete response, partial response, or stable disease. Decision to continue accrual into the second stage depended on the CBR from the first stage. Results: Forty-two patients received SBRT/nivolumab/ipilimumab with a CBR of 31.0% [95% confidence interval (CI), 17.6–47.1]. Five patients (11.9%) achieved partial response with median duration of 4.4 months (range, 1.1–21.5). Nineteen patients received SBRT/nivolumab. This group was closed after the initial stage based on a CBR of 10.5% (95% CI, 1.3–33.1). Adverse events were graded with National Cancer Institute Common Terminology Criteria for Adverse Events version 4.0. Grade ≥3 treatment-related adverse events occurred in 13 (31%) and 3 (16%) patients in the SBRT/nivolumab/ipilimumab and SBRT/nivolumab groups, respectively. One patient died from immune-related hepatitis in the SBRT/ nivolumab/ipilimumab group. Conclusions: Combining SBRT, nivolumab, and ipilimumab is well tolerated, feasible, and shows response in a subgroup of patients with mBTC.