Evidence suggests that because asbestos was used a lot in the past and the fibres occur naturally, we are all exposed to and inhale a low level of asbestos on a daily basis. It is very unlikely that the general public will be exposed to a high enough level of asbestos, over a sufficiently long period of time to cause adverse health effects.
If you are in contact with someone or something that has been in direct contact with asbestos, it is unlikely this will pose an elevated public health risk, especially if the exposure is only short-term.