A mistake made by Ayn Rand and some libertarians is to think it's simple to decide what is and isn't force.
Consider these example situations:
1) A biker is forced to steer well around a car being washed or else be hit with some water.
2) A car door is opened 10 ft in front of a biker who steers around it. The biker didn't see the driver look first.
3) Some amateur bikers go slowly blocking the road. They choose not to go single file to let people pass.
4) A car doesn't use a blinker when it should.
5) A baseball field is built so that home runs which clear the fence could hit people at a nearby park.
In each case, it is not obvious if force is being initiated. That means the principle "don't initiate force" isn't a panacea, because reasonable people can disagree about what force is.
One of the crucial mechanisms of a peaceful, civilized society is that people in general make a reasonable effort to avoid doing anything in the grey zone which might be taken as force, but at the same time if someone does do something borderline bad they do their best to overlook it and not get upset. Almost everyone having a double standard in this way (aim for one standard, but accept things up to a worse standard from others) is really effective. People don't suceed at this every time, but it's a major source of confliction prevention.
Another mechanism of our society is to treat the same action differently depending on the person's intention. Was it an accident or intended? Was he trying to accomplish a legitimate purpose that people should accomplish, or not? If you hit someone while playing baseball that's one thing; if you just find a fence with people on the other side and start hitting balls over that's another.
It's important to be tolerant of impositions others impose on us, and to expect them, not to just draw a line and become hateful and aggrieved if anyone crosses it.
One underlying reason this attitude is effective is that conflict resolution and negotiation is expensive. I don't have in mind only courts, but even simply talking to a stranger, and making them understand the issue, can be hard. They don't know what's on your mind, or what kind of things you care about or expect from life, and they may well assume if you're talking to them it must be important and so misunderstand any small complaint.