I am not sure yet. Some kind of alternative extension of integrals which happens to make pathwise calculations ove stochastic integrals simple, in some sense. I am pretty sure they mean rough in the sense of approximate rather than the sense of not smooth. Or maybe both?
Seems to originate in a fairly impenetrable body of work by Lyons, e.g. T. Lyons (1994) but modern recommendations are to read Friz and Hairer (2020), available free online, as an introduction, which covers the simplest (?) case of Gaussian noise.
Hodgkinson, Roosta, and Mahoney (2021) makes use of rough path integrals to justify learning by the adjoint method in stochastic differential equations.
Chevyrev and Kormilitzin (2016) discusses path signatures in particular, which is something arising in the theory about which I know little.