If you don’t write much or don’t care about writing style, then the distinction between “that” and “which” is not likely of interest to you. The rest of us are divided among three camps:
1. There is no distinction between the two, and anyone who posits a distinction is a pompous pedant.
2. Failing to observe the proper distinction is a grievous error.
3. Observing the distinction generally makes your writing crisper and clearer.
I am mostly in the third category. I’m certainly not enough of an authority to pronounce on the second.
What is the distinction? When debating whether to start a clause with one of the two words, restrictive clauses should be introduced by “that,” and dependent clauses should be introduced by “which.” In the latter case, “which” should always be preceded by a comma.
What’s all this “restrictive clause” and “dependent clause” mumbo jumbo? Restrictive clauses identify the subject of the clause. Dependent clauses add description about an already identified subject. Imagine a line of cars. I might say, “I like the car that is red.” Or I might say,” I like the car on the left end, which is red.” In the first example, I have identified the car by stating its color. In the second, the car is already identified by its position, and the reference to color is just an additional description.
I think my writing has improved by sticking with the distinction. In drafting legal documents, failing to observe it occasionally results in ambiguity. Outside that context, I still prefer the distinction. You may believe and act otherwise, but there will always be some who find your writing poorer for that. If you observe the distinction, few in category one are likely to notice.
That itself speaks well of the rule. It is noticeable only when you don’t follow it. You should want your writing to communicate ideas without distracting the reader by the tools you used to accomplish the task.