What is bit depth?
Bit depth describes the level of detail for the color information stored in an image. The higher the bit depth an image has, the more colors it can store. The more colors that can be stored, the smoother and more accurate an image looks. The lower the bit depth is for an image, the more likely you are to see sharp delineations between color changes in the images. Sharp changes that don't look like an image in real life are referred to as banding.
In the image presented above you can see how increased bit depth (the image on the left) yields a smoother image, while less bit depth results in what is known as banding - where you can see separate bands of color instead of a smooth blend between the colors. With techniques like dithering, you can conceal banding with varying degrees of success and avoid going up to a higher and more data expensive quality level. It depends on the image, the technique you use, and how high quality you need the final output to be.
When discussing bit depth, you'll hear things like this codec or color space is 8-bit, or this is 10-bit, 16-bit or similar. If someone says this is 8-bit color, what they're saying is that there are 2^8 possible values for red, green and blue in the image. That means you can have up to 256 values per color. How many values are possible changes rapidly as you increase bit depth. For example 10-bit appears to be a slight increase from 8-bit until you realize that this is a way of saying there are 2^10 possible values (or 1024 values) per color (red, green, blue). It's a big quality increase, and therefore requires a lot more storage.
As you can see, high quality color can become expensive quickly. So it's important to determine the level of quality you need. For example, you want the highest bit depth possible when you're capturing video footage because this gives you the most choices later when you process it. You can always drop information to create images with lower bit depth, but you can't add color information that you don't have.