A datum is the reference framework that defines how latitude, longitude, and (for vertical datums) height values correspond to real positions on the Earth. A horizontal (geodetic) datum specifies an ellipsoid model of the Earth's shape, its size and orientation, and a set of fixed control points that tie the model to the ground.
Why it matters
Coordinates are meaningless without a datum. The same latitude/longitude pair refers to different ground positions under different datums, because each datum places its ellipsoid slightly differently relative to the Earth. Mixing datums without transforming between them is one of the most common causes of layers that "almost" line up but are offset by metres to hundreds of metres.
A concrete example
WGS84 (used by GPS, EPSG:4326) and NAD83 are based on nearly identical ellipsoids, but the North American plate has drifted relative to the global frame. By recent epochs the offset between WGS84 and NAD83 positions in North America is on the order of 1–2 metres and growing. For survey-grade work you must apply a proper datum transformation rather than treating the two as interchangeable. In Europe, ETRS89 is fixed to the stable part of the Eurasian plate, so it diverges from WGS84 over time as well.
Common pitfall
A datum is not the same as a projection or a coordinate reference system (CRS). The datum defines where the coordinate origin sits; the projection defines how the curved surface is flattened to a plane. A full CRS (identified by an EPSG code) bundles a datum, a coordinate system, and—if projected—a projection. Assuming two datasets share a datum just because both are "lat/long" is a frequent mistake.