All of the uses of "by" in this way in my dialect (or at least my idiolect--I think it's dialectical, though) are paired with verbs having to do with motion--"go by", "swing by", "run by"*...stuff like that. In all cases, the use of this construction means that a minimum of time was spent in whatever place the person is visiting. If I say "I swung by the store after work", for example, it means that I stopped, quickly got a few things, and left. If I ended up doing a through shopping trip I'd say that I "went to" or "stopped at" the store.
Prior to having read this thread, if someone had told me that they ate by someone I would have assumed that they meant that they ate while sitting next to them. It wouldn't have occurred to me that they meant anything else by it.
*"stop by" is the one exception to this that I can think of, and even there you could still argue that it had to do with motion, or at least the cessation of it.