The energy barrier heights between two recently reported Ta-based amorphous metals (TaWSi and TaNiSi), TaN, and atomic layer deposited Al2O3 and HfO2 insulators are measured in metal/insulator/metal (MIM) structures with Au top electrodes using internal photoemission (IPE) spectroscopy. For Al2O3, the Ta-based metal barrier heights, phi(Bn), increase with increasing metal work function, Phi(M), for TaN, TaWSi, and TaNiSi, respectively. For HfO2, however, the barrier heights are relatively constant for all three metals phi(Bn),(TaNiSi) approximate to phi(Bn.TaWSi) approximate to phi(Bn.TaN). The difference between HfO2 and Al2O3 is attributed to enhanced Fermi-level pinning due to a larger dielectric constant. The slope parameter, S, was found to be roughly 0.89 and 0.44-0.69 for Al2O3 and HfO2, respectively. For devices with a TaWSi bottom electrode, a comparison was also made between Al and Au top electrodes. Significantly, smaller barrier heights were obtained with an Au top electrode than with an Al top electrode, 0.6 eV and 0.8 eV lower for HfO2 and Al2O3, respectively. IPE energy barriers are found consistent with current-voltage asymmetry of MIM diodes, whereas Schottky model predictions of barrier heights were inconsistent.