Convolutions generated by self-adjoint unbounded operators