Accuracy

Floating Point numbers are not an exact representation of numbers.

The reasons for this are twofold.

1. It is much more efficient for a computer to work with data that is a "fixed size". You can't cram all the infinite possible numbers into 32, 64, or any fixed number of bits.
2. For most systems, the inaccuracy is okay. When it is not, there are ways to do "real math" that we will cover much later.

For an explanation of the mechanics, I'll defer to this old Computerphile video.