This research monograph develops an arithmetic analogue of the theory of ordinary differential equations: functions are replaced here by integer numbers, the derivative operator is replaced by a ``Fermat quotient operator'', and differential equations (viewed as functions on jet spaces) are replaced by ``arithmetic differential equations''. The main application of this theory concerns the construction and study of quotients of algebraic curves by correspondences with infinite orbits. Any such quotient reduces to a point in usual algebraic geometry. But many quotients as above cease to be trivial (and become quite interesting) if one enlarges algebraic geometry by using arithmetic differential equations in place of algebraic equations. The book partly follows a series of papers written by the author; however, a substantial part of the material presented here has never been published before. For most of the book the only prerequisites are the basic facts of algebraic geometry and number theory.