Single- and double-differential inclusive dijet cross sections in neutral current deep inelastic ep scattering have been measured with the ZEUS detector using an integrated luminosity of 374 pb$^{-1}$. The measurement was performed at large values of the photon virtuality, Q$^2$, between 125 and 20000 GeV$^2$. The jets were reconstructed with the k$_T$ cluster algorithm in the Breit reference frame and selected by requiring their transverse energies in the Breit frame, E$_T^{B,jet}$, to be larger than 8 GeV. In addition, the invariant mass of the dijet system, M$_{jj}$, was required to be greater than 20 GeV. The cross sections are described by the predictions of next-to-leading-order QCD.