This article presents the results of two studies of Higgs boson properties using the $WW^*(\rightarrow e\nu\mu\nu)jj$ final state, based on a dataset corresponding to 36.1 fb$^{-1}$ of $\sqrt{s}=13$ TeV proton-proton collisions recorded by the ATLAS experiment at the Large Hadron Collider. The first study targets Higgs boson production via gluon-gluon fusion and constrains the CP properties of the effective Higgs-gluon interaction. Using angular distributions and the overall rate, a value of $\tan(\alpha) = 0.0 \pm 0.4 (\mathrm{stat.}) \pm 0.3 (\mathrm{syst.})$ is obtained for the tangent of the mixing angle for CP-even and CP-odd contributions. The second study exploits the vector-boson fusion production mechanism to probe the Higgs boson couplings to longitudinally and transversely polarised $W$ and $Z$ bosons in both the production and the decay of the Higgs boson; these couplings have not been directly constrained previously. The polarisation-dependent coupling-strength scale factors are defined as the ratios of the measured polarisation-dependent coupling strengths to those predicted by the Standard Model, and are determined using rate and kinematic information to be $a_\mathrm{L}=0.91^{+0.10}_{-0.18}$(stat.)$^{+0.09}_{-0.17}$(syst.) and $a_{\mathrm{T}}=1.2 \pm 0.4 $(stat.)$ ^{+0.2}_{-0.3} $(syst.). These coupling strengths are translated into pseudo-observables, resulting in $\kappa_{VV}= 0.91^{+0.10}_{-0.18}$(stat.)$^{+0.09}_{-0.17}$(syst.) and $\epsilon_{VV} =0.13^{+0.28}_{-0.20}$ (stat.)$^{+0.08}_{-0.10}$(syst.). All results are consistent with the Standard Model predictions.